OAI-SearchBot, explained
By Matvei Belov, founder of Glowsearch · Updated
OAI-SearchBot is the crawler OpenAI uses to find pages for ChatGPT search. If it can't read your site, ChatGPT can't show your pages in its search answers. OpenAI says sites that opt out of OAI-SearchBot "will not be shown in ChatGPT search answers, though can still appear as navigational links."
The short version
- What it is: OpenAI's search crawler. Its robots.txt name is
OAI-SearchBot. - What it isn't: GPTBot, which collects pages for training. You can block GPTBot and still allow OAI-SearchBot.
- To appear in ChatGPT search: allow it in robots.txt, and let its published IP addresses through your firewall or CDN.
- How fast: OpenAI says robots.txt changes take about 24 hours to reach ChatGPT search.
- To check: use our free ChatGPT crawler check.
What OAI-SearchBot does
When someone asks ChatGPT a question it searches the web for, the pages it can show come partly from OAI-SearchBot. OpenAI describes it in one line: "OAI-SearchBot is for search." It's used "to surface websites in search results in ChatGPT's search features."
ChatGPT also gets results from third-party search providers, Bing among them. So a page can reach ChatGPT through Bing too. But OpenAI's advice for appearing in ChatGPT search is to allow OAI-SearchBot, and its own crawler is the part you control.
Its user agent
OpenAI publishes this example. The version number can change:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
When it fetches your robots.txt, it may add a robots.txt; marker before the link, so you can tell those requests apart in logs that don't record paths:
... compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot
OAI-SearchBot, GPTBot and ChatGPT-User
OpenAI runs several crawlers, each with its own robots.txt setting. These are the three people mix up:
| Crawler | What it's for | If you block it |
|---|---|---|
| OAI-SearchBot | Finding pages for ChatGPT search | Your pages stop appearing in ChatGPT search answers |
| GPTBot | Collecting pages that may be used to train OpenAI's models | Your content is kept out of training; search isn't affected |
| ChatGPT-User | Opening a page when a person asks ChatGPT to, or a custom GPT needs it | OpenAI says robots.txt "may not apply", and it isn't used for search |
If you allow both OAI-SearchBot and GPTBot, OpenAI may use one crawl for both, so you can see fewer visits than you'd expect. There's also OAI-AdsBot, which only visits pages submitted as ChatGPT ads. Can ChatGPT see my website? covers all four.
robots.txt rules
Allow ChatGPT search, block training. The most common choice:
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /
Allow ChatGPT search, but not some folders:
User-agent: OAI-SearchBot
Disallow: /admin/
Disallow: /cart/
Allow: /
Block OpenAI's search and training crawlers. Your pages then won't appear in ChatGPT search answers. This doesn't cover ChatGPT-User, which OpenAI says robots.txt rules may not apply to, or OAI-AdsBot, which only visits ad pages:
User-agent: OAI-SearchBot
Disallow: /
User-agent: GPTBot
Disallow: /
If your robots.txt has no rule naming OAI-SearchBot, the rules under User-agent: * apply to it. A blanket Disallow: / there blocks it too. Our robots.txt generator writes these rules for you.
Firewalls and CDNs
robots.txt is only half of it. A firewall, CDN or hosting plan can turn OAI-SearchBot away before it ever reads robots.txt. Cloudflare, for one, blocked AI crawlers by default on new domains from July 2025. Since September 15, 2026, new domains get presets that allow search crawlers, but a domain set up before then may still block them. Check the AI crawler controls in your domain's Security Settings.
OpenAI recommends "allowing requests from our published IP ranges", listed at openai.com/searchbot.json. Allow those addresses rather than the name alone, since anyone can send a request that calls itself OAI-SearchBot.
How to see it in your logs
Search your server's access log for the name:
grep "OAI-SearchBot" access.log
Then check that the addresses are really OpenAI's. This prints the published ranges, which you can compare with the IPs in your log:
curl -s https://openai.com/searchbot.json | jq -r '.prefixes[].ipv4Prefix'
A request that uses the name but comes from another address isn't OpenAI's.
You can also see the result rather than the crawl: visits from links in ChatGPT's answers arrive with utm_source=chatgpt.com, which you can filter on in Google Analytics or similar.
Does allowing it get you recommended?
It makes it possible, not likely. Allowing OAI-SearchBot means ChatGPT can cite your pages. Whether it names your business depends on what it finds about you across the web.
In September 2026 we asked ChatGPT 70 questions customers ask about local services, in six US cities. Its 209 answers linked to about 2.6 sites each. Of 2,351 businesses that appeared on the answers' maps with a website, 123 (5%) had their own site among the links. Being readable is the entry ticket; the rest is in how to get ChatGPT to recommend your business.
Common questions
In Vercel's December 2024 measurements, none of OpenAI's crawlers ran JavaScript. If your pages only show their text after scripts run, OAI-SearchBot may see an empty page. Open a page with JavaScript turned off to check.
OpenAI doesn't say. On a new or small site, visits can be rare. There's no form to request a crawl; make sure it's allowed and that your pages are in Bing too.
There's no submission form. Any public site can appear if OAI-SearchBot isn't blocked. Adding your site to Bing Webmaster Tools helps, since Bing is one of ChatGPT's search providers.
OpenAI treats them as separate settings. Blocking GPTBot keeps your content out of training and doesn't remove you from ChatGPT search.
OpenAI says a blocked page can still appear as a link if ChatGPT learns its address from another search provider. To keep it out, add a noindex tag and leave OAI-SearchBot allowed so it can read the tag.