Business & Enterprise2 min read
Meta removes language that may have allowed it to scrape news sites
The Verge
Updates on Robots Txt
Every AI story we track on Robots Txt: 3 stories so far, each summarized in our own words and linked to the publisher that reported it.
Pulled from 125 sources
This month
AI Research8 min read
Only 8.9% of sites block AI crawlers, but 94.8% are never cited in AI answers
Hacker News
Social Media3 min read
Patreon stops asking AI bots not to scrape — and starts blocking them
TechCrunch
That's everything we have on Robots Txt right now