AI search
GPTBot
OpenAI's training crawler. Distinct from OAI-SearchBot.
Definition
GPTBot fetches content used in OpenAI model training. It is distinct from OAI-SearchBot (which fetches for ChatGPT search-time grounding) and ChatGPT-User (which fetches in real-time as a user asks a question). All three respect robots.txt rules under their respective User-agent strings.
More in AI search
Answer Engine Optimization (AEO)
Optimizing content to be cited inside AI-generated answers.
AI Overviews
Google's AI-generated answer block that appears above traditional search results.
llms.txt
A proposed file at site root that hints to AI crawlers what content to prioritize.
OAI-SearchBot
OpenAI's search crawler. Indexes content for ChatGPT search-time grounding.
Need this concept applied to your stack?
Glossary entries are intentionally short. Real engineering tradeoffs need a scoping call — bring the domain, the stack, and the question.