Reading List

AI Companies Ignoring Robots.txt from Michael Tsai RSS feed.

AI Companies Ignoring Robots.txt

Mark Sullivan: The AI search startup Perplexity is in hot water in the wake of a Wired investigation revealing that the startup has been crawling content from websites that don’t want to be crawled.[…]“Perplexity is not ignoring the Robot Exclusions Protocol and then lying about it,” said Perplexity cofounder and CEO Aravind Srinivas in a […]