Applebot-Extended
Apple · Training · robots.txt: Yes
A robots.txt control token, not a crawler: it decides whether pages Applebot has crawled may be used to train Apple's generative AI foundation models.
Block Applebot-Extended
User-agent: Applebot-Extended
Disallow: /Disallowing Applebot-Extended opts out of training Apple's foundation models, but Apple says your pages can still appear in its search results and the rule is not a ranking factor. To keep content out of AI-generated "broad world knowledge" answers in Siri and Search, Apple points to the nosnippet robots meta tag instead.
Allow Applebot-Extended
User-agent: Applebot-Extended
Allow: /An allow group only changes anything if a broader rule would block Applebot-Extended. Under the robots.txt standard (RFC 9309, section 2.2.1) a crawler follows the group that names it and uses the User-agent: * group only when no group does — so this group lets it in even if your * group says Disallow: /. Token matching is case-insensitive.
Put these lines in /robots.txt at the root of each host (each subdomain has its own file). robots.txt is a request to well-behaved crawlers, not access control.
Facts
- Operator
- Apple
- Purpose
- Model training
- Obeys robots.txt
- Yes. Applebot-Extended exists only as a robots.txt rule; Apple says it does not crawl webpages and is used only to decide how data crawled by Applebot is used.
- In your server logs
- None of its own: Apple says Applebot-Extended does not crawl webpages. The fetching is done by Applebot.
- How to verify it
- There is no Applebot-Extended traffic to verify. Applebot itself is identified by reverse DNS in the applebot.apple.com domain, or by matching the IP against Apple's published CIDR list.
Published list: search.developer.apple.com/applebot.json — 24 IPv4 ranges when we fetched it, list dated 15 Sep 2026. Always match against the live list; operators update them.
What Apple says
Quoted word for word from the operator's documentation.
“With Applebot-Extended, web publishers can choose to opt out of their website content being used to train Apple’s general purpose foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools.”
“Applebot-Extended does not crawl webpages. Webpages that disallow Applebot-Extended can still be included in search results. Applebot-Extended is only used to determine how to use the data crawled by the Applebot user agent.”
“Site rules for Applebot-Extended are not considered in ranking for Search.”
“Web publishers can opt out of their content being used in these broad world knowledge answers by applying the nosnippet meta tag to specific content.”
“Traffic coming from Applebot is generally identified by using reverse DNS in the *. applebot.apple.com domain.”
“Another way is to match the IP address with a CIDR prefix contained in the following JSON file: Applebot IP CIDRs.”
Sources
- Apple Support: About Applebot — fetched and checked 2026-09-22
- Apple: Applebot IP CIDRs (applebot.json) — fetched and checked 2026-09-22