| title | slug | description | author | date | last_modified_at | draft | category |
|---|---|---|---|---|---|---|---|
Prevent ChatGPT from Scraping Your Site |
no-gpt-scrapey-scrapey |
The goods you need to add to robots.txt to keep ChatGPT from consuming your work. |
2023-08-11T16:00:00:000Z |
2023-08-11T16:00:00:000Z |
false |
programming |
A short but sweet post. Ready?
OpenAI quietly published the crawler name/user agent for ChatGPT, creatively named GPTBot.
Since we know the user agent, now, we can effectively prevent it from crawling a site using robots.txt like so:
User-agent: GPTBot
Disallow: /
OpenAI was also generous enough to provide a list of IP ranges their crawler will connect from, so go ahead and add these to your firewall rules, too:
20.15.240.64/28
20.15.240.80/28
20.15.240.96/28
20.15.240.176/28
20.15.241.0/28
20.15.242.128/28
20.15.242.144/28
20.15.242.192/28
40.83.2.64/28