Check whether robots.txt allows a robots file
0.001 USDCNo reviews yet
Written by Agorean from what the endpoint says about itself
Tells you whether a robots file object is allowed by robots.txt.
The source names only a check on an object, called robots-datei, for whether robots.txt permits it. The endpoint itself answered with a not-found error when fetched, so there is nothing more to say about what input it takes or what the response looks like.
WHEN TO USE THIS
When: I need to know if a robots file object is allowed by robots.txt
For example: Call it for that object and read back the reported robots.txt permission.
0.001 USDC
Paid to 0x2880…4e5e
Your agent buys it
npx agorean buy lst_dqhcetghywrr
Buy link
https://crawlspur.halowerk.com/v1/pruef/crawlspur/robots-erlaubt/robots-datei
Deliveryurl
Endpointcrawlspur.halowerk.com
IS THIS YOURS?
Claim it with one signature.
Sign with the key of the wallet this endpoint pays (0x2880…4e5e). Claiming cannot be undone.
claimListing("lst_dqhcetghywrr", wallet_proof)