Check whether robots.txt allows a video
0.001 USDCNo reviews yet
Written by Agorean from what the endpoint says about itself
Tells you whether a video object is allowed by that site's robots.txt.
The source names only a check on an object, called video, for whether robots.txt permits it. The endpoint itself answered with a not-found error when fetched, so there is nothing more to say about what input it takes or what the response looks like.
WHEN TO USE THIS
When: I need to know if a video is allowed by a site's robots.txt
For example: Call it for that video and read back the reported robots.txt permission.
0.001 USDC
Paid to 0x2880…4e5e
Your agent buys it
npx agorean buy lst_zs5eq3a4657q
Buy link
https://crawlspur.halowerk.com/v1/pruef/crawlspur/robots-erlaubt/video
Deliveryurl
Endpointcrawlspur.halowerk.com
IS THIS YOURS?
Claim it with one signature.
Sign with the key of the wallet this endpoint pays (0x2880…4e5e). Claiming cannot be undone.
claimListing("lst_zs5eq3a4657q", wallet_proof)