Agreeing to honor a single robots. txt signal doesn’t solve the bigger problem of scraping the entire internet already.

Source: [Inc.](https://www.inc.com/jason-aten/the-4-ai-giants-just-agreed-to-let-websites-opt-out-theyve-already-taken-the-data/91429311)

Sponsored