Can Perplexity see your website?

How Perplexity finds websites

Perplexity answers with a web search by default, and cites the pages it used. Those pages come from its own index, built mainly by PerplexityBot. Perplexity says PerplexityBot exists to surface and link websites in its results, isn't used to train AI models, and recommends allowing it if you want to appear.

Its other crawler, Perplexity-User, opens a page when someone asks about it. Perplexity says it generally ignores robots.txt, because a person asked for the page.

Perplexity doesn't build its own foundation models, so there's no training crawler to block: blocking PerplexityBot is purely a choice about appearing in its answers.

What blocking actually does

Blocking PerplexityBot doesn't erase you from Perplexity. In its help center, Perplexity says it won't index the text of a site that disallows PerplexityBot, but may still index the domain, the headline and a brief factual summary.

Perplexity also says it works with third-party crawlers to build its index, and that it has updated its agreements so they respect robots.txt too, particularly for news publishers.

Why firewalls often block Perplexity

Perplexity has a history with bot protection. In August 2025, Cloudflare reported that when PerplexityBot was blocked, it saw requests using a generic Chrome browser name from addresses outside Perplexity's published lists. Cloudflare removed Perplexity from its verified bots and added rules to block the traffic. Perplexity disputed the findings.

The upshot for you: on Cloudflare and similar services, Perplexity's crawlers can be caught by default bot protection, even if your robots.txt welcomes them. Perplexity's own advice is an explicit allow rule that matches both the name and the address:

• Cloudflare: in Security, then WAF, add a custom rule where the user agent contains PerplexityBot or Perplexity-User, and the source address is in Perplexity's published ranges. Set the action to Allow.
• AWS WAF: create IP sets from the same lists, add string matches for the two user agents, and give the allow rule a higher priority than your blocking rules.

The addresses change, so Perplexity suggests refreshing them from its JSON files regularly rather than copying them once.

What this checks

• Whether your robots.txt lets PerplexityBot and Perplexity-User read your homepage, and which line decides it.
• Whether your server lets in a request that uses PerplexityBot's name. A block here usually means a firewall rule, not robots.txt. The real crawler comes from Perplexity's published addresses, so treat it as a sign to check your settings.
• Whether your homepage has its text in the HTML. In Vercel's December 2024 measurements, PerplexityBot didn't run JavaScript.

If Perplexity is blocked

Allow PerplexityBot in robots.txt:

User-agent: PerplexityBot
Allow: /

The robots.txt generator writes the whole file for you. Perplexity says changes can take up to 24 hours to show.

How to tell whether Perplexity is reading you

Your server logs. PerplexityBot's requests contain compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot. Check the address against perplexitybot.json: a request with the right name from the wrong address isn't Perplexity's declared crawler.

Perplexity itself. Ask it a question your page answers. Perplexity shows its sources by default, so you can see straight away whether you're among them.

What gets cited

Perplexity cites a handful of pages per answer, and they're often review sites, lists and forums rather than the businesses themselves. Being in its index gets you considered; being on the pages it already trusts for your questions gets you named. How AI assistants choose their sources explains why, and the AI visibility checklist covers what to do about it.