Brightbot
Bright Data · Unconfirmed · robots.txt: Unconfirmed
Bright Data's data collection crawler, meant to be the main pipeline that collects web data for all of Bright Data's products and services.
Block Brightbot
User-agent: Brightbot
Disallow: /Unconfirmed by the operator Bright Data's documentation we checked does not say what blocking Brightbot changes beyond the crawler itself.
Allow Brightbot
User-agent: Brightbot
Allow: /An allow group only changes anything if a broader rule would block Brightbot. Under the robots.txt standard (RFC 9309, section 2.2.1) a crawler follows the group that names it and uses the User-agent: * group only when no group does — so this group lets it in even if your * group says Disallow: /. Token matching is case-insensitive.
Put these lines in /robots.txt at the root of each host (each subdomain has its own file). robots.txt is a request to well-behaved crawlers, not access control.
Already have a robots.txt? Paste it into the robots.txt checker to see whether it blocks Brightbot on a given path, and which line decides.
Who blocks it
Brightbot is not in the 2026-10-09 survey: it was added to this directory after we last read the most-visited sites' robots.txt files, so we have no count for it yet.
Facts
- Operator
- Bright Data
- Purpose
- Unconfirmed by the operator
Bright Data documents it as its data collection crawler and says its purpose is to be the main data collection pipeline for all of Bright Data's products and services. It does not say what those products use the data for.
- Obeys robots.txt
- Unconfirmed by the operator
- In your server logs
- Bright Data gives the user-agent as "Brightbot 1.0".
- How to verify it
- Bright Data says to identify it by two parameters together: the user-agent "Brightbot 1.0" and the source IP subnet 82.97.199.0/24, which it states on its Brightbot page.
What Bright Data says
Quoted word for word from the operator's documentation.
“Brightbot is Bright Data’s data collection crawler.”
“Its purpose is to be the main data collection pipeline for all of Bright Data’s products and services.”
“Brightbot can easily be identified by two parameters: User-Agent: "Brightbot 1.0" Source IP subnet 82.97.199.0/24 Using both will ensure accurate identification.”
Sources
- Bright Data: Brightbot — fetched and checked 2026-10-09