OpenAI launches web crawling GPTBot, sparking blocking effort by website owners and creators
August 10, 2023
Without fanfare or an official announcement, ChatGPT maker OpenAI launched a new website crawling bot this week for scanning website content to train its large language models (LLMs). But after news of the bot broke, a revolt ensued, as website owners and creators quickly traded tips on how to block GPTBot from scraping their sites’ data.
When OpenAI added the GPTBot support page, it also…