Amazonbot
Amazon · Training · robots.txt: Yes
Crawls the web to improve Amazon's products and services; the content may be used to train Amazon AI models.
Block Amazonbot
User-agent: Amazonbot
Disallow: /Blocking Amazonbot stops this crawler; Amazon's search experiences (such as Alexa) are controlled separately by Amzn-SearchBot. To stay crawlable but keep a single page out of model training, Amazon documents the noarchive robots meta tag.
Allow Amazonbot
User-agent: Amazonbot
Allow: /An allow group only changes anything if a broader rule would block Amazonbot. Under the robots.txt standard (RFC 9309, section 2.2.1) a crawler follows the group that names it and uses the User-agent: * group only when no group does — so this group lets it in even if your * group says Disallow: /. Token matching is case-insensitive.
Put these lines in /robots.txt at the root of each host (each subdomain has its own file). robots.txt is a request to well-behaved crawlers, not access control.
Facts
- Operator
- Amazon
- Purpose
- Model training
Amazon's wording is a mix: Amazonbot is "used to improve our products and services" and "may be used to train Amazon AI models".
- Obeys robots.txt
- Yes. Amazon says automated crawling honours robots.txt user-agent and allow/disallow rules, reads robots.txt per host or uses a copy cached for up to 30 days, and behaves as if there is no robots.txt when the file cannot be fetched. It does not support Crawl-delay. It also honours rel=nofollow and the robots meta tags noarchive (which Amazon says means do not use the page for model training), noindex and none.
- In your server logs
- The user-agent string contains "Amazonbot/0.1".
- How to verify it
- Match the request's IP address against Amazon's published Amazonbot IP list.
Published list: developer.amazon.com/amazonbot/ip-addresses/ — 1292 IPv4 ranges when we fetched it, list dated 8 Sep 2026. Always match against the live list; operators update them.
What Amazon says
Quoted word for word from the operator's documentation.
“Amazonbot is used to improve our products and services. This helps us provide more accurate information to customers and may be used to train Amazon AI models.”
“Each user agent setting is independent of the others, and may take ~24 hours for our systems to reflect changes.”
“Automated crawling from these listed user agents respects the Robots Exclusion Protocol , honoring the user-agent and the allow/disallow directives. They will fetch host-level robots.txt files or use a cached copy from the last 30 days. When a file can’t be fetched, they will behave as if it does not exist.”
“When these user agents access web pages they respect the link-level rel=nofollow directive, and page level robots meta tags of noarchive (do not use the page for model training), noindex (do not index the page) and none (do not index the page). They do not support the crawl-delay directive.”
“Example User Agent String: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36”
“Published IP Addresses:”
Sources
- Amazon Developer: About Amazonbot — fetched and checked 2026-09-22
- Amazon Developer: Amazonbot IP addresses — fetched and checked 2026-09-22