Amazonbot
Amazonbot is Amazon's web crawler, used to fetch and index page content for Amazon's products and services, including eligibility for AI model training. Amazon operates it alongside two related, independently controlled user-agents — Amzn-SearchBot, which feeds search-adjacent experiences like Alexa and Rufus, and Amzn-User, which fetches pages on behalf of a specific user action rather than crawling generally — and neither of those two crawls content for AI training. All three identify themselves by name in their user-agent string, honor robots.txt, and do not support a Crawl-delay directive.
Amazon's three crawler identities
Amazon documents three distinct user-agents, each with a different scope:
| User-agent | Purpose | Used for AI training? |
|---|---|---|
| Amazonbot | General crawling to improve Amazon's products and services | Yes |
| Amzn-SearchBot | Feeds search-adjacent experiences — Alexa, Rufus | No |
| Amzn-User | Fetches a page for a specific user action, e.g. an Alexa query needing current information | No |
They're independent: blocking Amazonbot does not block Amzn-SearchBot or Amzn-User, and allowing Amzn-SearchBot does not make a site's content eligible for AI training. A site that wants Alexa and Rufus eligibility without opting into AI training needs to allow Amzn-SearchBot and Amzn-User while disallowing Amazonbot specifically — the same shape of control Applebot and Applebot-Extended offer on Apple's side, GPTBot and OAI-SearchBot on OpenAI's.
What it means for product pages
For a distributor or manufacturer, Amzn-SearchBot and Amzn-User access is what makes product content eligible to surface in Rufus (Amazon's shopping assistant) and Alexa shopping queries. That's a separate consideration from whether Amazon Marketplace listing data feeds those same experiences — Rufus draws on both structured catalog data submitted through Seller Central and, where allowed, content crawled directly from a brand's own web pages. Blocking Amazon's crawlers outright, deliberately or through an overly broad bot-blocking rule, removes that content source regardless of what's submitted through the marketplace feed.
Controlling access in robots.txt
The three user-agents take independent robots.txt rules:
User-agent: Amazonbot
Disallow: /
User-agent: Amzn-SearchBot
Allow: /
User-agent: Amzn-User
Allow: /
That configuration opts out of AI-model training crawl while keeping the site eligible for Alexa and Rufus search-adjacent experiences. None of the three honor a Crawl-delay directive; Amazon's crawlers adjust their own request rate rather than following a fixed delay set in robots.txt.
Frequently asked questions
What's the difference between Amazonbot and Amzn-SearchBot?
Amazonbot is the general-purpose crawler Amazon uses to improve its products and services, and its crawl is eligible for AI model training. Amzn-SearchBot specifically feeds search-adjacent experiences like Alexa and Rufus and, per Amazon, is not used to train generative AI models. They're separate user-agents with separate robots.txt controls.
Does blocking Amazonbot stop a site from appearing in Rufus or Alexa?
Not by itself. Rufus and Alexa shopping features draw on Amzn-SearchBot and Amzn-User, not Amazonbot. A site can disallow Amazonbot to opt out of AI training crawl while still allowing Amzn-SearchBot and Amzn-User for search-adjacent eligibility.
Does Amazonbot respect Crawl-delay in robots.txt?
No. None of Amazon's three documented user-agents — Amazonbot, Amzn-SearchBot, Amzn-User — honor a Crawl-delay directive. They adjust their request rate automatically rather than following a fixed delay.
How can a site verify a request actually came from Amazonbot?
By checking the user-agent string for the literal "Amazonbot" identifier and cross-referencing the requesting IP against Amazon's published IP ranges for the crawler, since user-agent strings alone can be spoofed. Amazon publishes the current IP list for verification alongside its crawler documentation.