PerplexityBot
PerplexityBot is Perplexity's retrieval crawler, described in Perplexity's documentation as designed to surface and link websites in search results on Perplexity, and it respects robots.txt. Perplexity separately runs Perplexity-User, which fetches a page when a user's question requires it and which the documentation says generally ignores robots.txt because the request was initiated by a person. Neither crawls for model training.
Two agents, two rulebooks
PerplexityBot is the scheduled crawler that builds Perplexity's index. Perplexity's own guidance recommends allowing it in robots.txt if you want to appear in results, and confirms it honours the file.
Perplexity-User is different by design. When a person asks Perplexity a question that requires visiting a page, that fetch happens on the user's behalf, and Perplexity documents that it generally ignores robots.txt rules for this reason. The logic is the same one browsers operate under: robots.txt governs automated crawling, not a person following a link.
Perplexity states that neither agent crawls for model training, which distinguishes both from GPTBot and ClaudeBot.
What this means for access decisions
The practical implication is that blocking PerplexityBot reduces your presence in Perplexity's index but does not make your pages unreachable — a user asking about a specific URL can still trigger a fetch.
For a distributor the calculus is usually simple, because there is no training exposure to weigh. Perplexity is a retrieval-and-cite engine; being indexed is how you appear. The reasons to block are narrower than they are for training crawlers, and mostly come down to infrastructure load or a policy position on AI answers reusing your content without a click.
The citation surface
Perplexity is comparatively transparent about sources, which makes it a useful testbed. Answers carry visible numbered citations, so you can run a prompt panel against it and see which of your URLs get pulled, at a level of clarity other assistants do not offer.
Use that. If your PDPs never appear but your blog does, the retrieval step is finding your editorial content and not your product records, which usually points at thin attribute coverage or a rendering problem on the catalog templates specifically.
Frequently asked questions
Does PerplexityBot respect robots.txt?
Yes. Perplexity documents PerplexityBot as a crawler that surfaces and links websites in results and recommends allowing it in robots.txt. The separate Perplexity-User agent, which fetches pages in response to a user's question, generally ignores robots.txt because a person initiated the request.
Does Perplexity train models on my content?
Perplexity's documentation states that neither PerplexityBot nor Perplexity-User crawls for AI model training. That distinguishes both from OpenAI's GPTBot and Anthropic's ClaudeBot, which are explicitly training crawlers.
Should we allow PerplexityBot?
For most catalogs, yes. It is a retrieval crawler with no stated training use, so allowing it is what makes your products eligible to be cited in Perplexity answers. The reasons to block are mostly about infrastructure load or a policy stance on uncredited reuse.
Why is Perplexity useful for testing AI visibility?
Because its answers display numbered source citations openly, you can see exactly which of your URLs were retrieved. That makes it a clearer diagnostic surface than assistants that cite less explicitly, particularly for finding out whether your product pages or only your editorial content are being found.