Answers / Crawlers

If I block GPTBot, will ChatGPT still recommend my business?

Matt Turley, Continuum. Updated November 5, 2026.

The crawlers involved, which one matters for being recommended, and how to check yours.

The short answer

If I block GPTBot, will ChatGPT still recommend my business?

Possibly. OpenAI documents GPTBot, used for training, and OAI-SearchBot, which surfaces sites in the search results ChatGPT cites, as separate crawlers. Blocking one does not block the other. If you want to be cited in live answers, check that OAI-SearchBot is allowed, and check your robots.txt and firewall for both.

Two crawlers, two jobs

OpenAI’s crawler documentation names separate bots for separate purposes. GPTBot is the one OpenAI describes for training its models. OAI-SearchBot is the one that surfaces sites in the search results ChatGPT cites. They follow separate rules in robots.txt.

So blocking GPTBot is a choice about training. It is not, by itself, a block on appearing in live ChatGPT answers. Perplexity documents its own PerplexityBot as surfacing and linking sites in results and not for training, with robots.txt steps of its own; see how to get cited by Perplexity.

How to check yours

Open yoursite.com/robots.txt and read every rule that names a bot, or applies to all bots.
Look for a rule that blocks the search crawler by name, or a blanket block that catches it.
Check any firewall or bot-protection service in front of your site, which can block crawlers that robots.txt allows.
Re-read OpenAI’s documentation, because crawler names and rules change.

Allowing the crawler is necessary, not sufficient

A crawler that can read your page is the first gate. The tool still has to choose to name you, and in our research that depends on how widely other sites describe you. Once access is sorted, see what ChatGPT reads before it names a business.

Want to see what the AI tools say about you?

Enter your business and the free check asks four AI engines the questions your customers ask, then shows whether you were named. No signup. It is a first look; the paid reports repeat each question several times.

Run the free AI visibility check. Or book a call and talk it through.

Book a call →See all services and prices →

FAQ

Straight answers

Should I block AI training crawlers?

That is your call, based on how you feel about training use of your content. It is a separate decision from whether you want to be cited in live answers.

How long until a robots.txt change takes effect?

It varies by crawler. Perplexity says changes can take up to a day to be picked up. Check each vendor’s documentation and re-test later.

Can a firewall block AI crawlers without my knowing?

Yes. Bot-protection settings can challenge or block crawlers that robots.txt allows. If your site sits behind one, check its logs and allow the crawlers you want.

How do I know if a crawler visited?

Your server or CDN logs show requests by user-agent name. Search the logs for the crawler names in each vendor’s documentation.

Keep reading

Run the free AI visibility checkFree. Four AI engines, no signup.How to get cited by PerplexityThe crawler checks and the page changes that matter.How do I get my business mentioned in ChatGPT answers?What ChatGPT reads before it names a business.How do I rank in ChatGPT for my business?Named versus cited, and what moves each.Does llms.txt help me get recommended by AI?What we have and have not measured.AI visibility audit for small businessWhat a good audit checks and what it should cost.Does ChatGPT recommend my business?The AI Answer Check: three tools, three runs per question.

Sources

1. OpenAI's crawler documentation, OpenAI: Used for the split between OAI-SearchBot, which surfaces sites in the search results ChatGPT cites, and GPTBot, which OpenAI uses for training.
2. Perplexity's bot documentation, Perplexity: Used for the point that PerplexityBot surfaces and links sites in search results and is not used to train models.