Cotoyogi
ROIS-DS · Unconfirmed · robots.txt: Yes
A research crawler run by the Center for Research and Development on Data Lake at ROIS-DS, collecting Japanese language data resources from the web.
Block Cotoyogi
User-agent: Cotoyogi
Disallow: /Unconfirmed by the operator ROIS-DS's documentation we checked does not say what blocking Cotoyogi changes beyond the crawler itself.
Allow Cotoyogi
User-agent: Cotoyogi
Allow: /An allow group only changes anything if a broader rule would block Cotoyogi. Under the robots.txt standard (RFC 9309, section 2.2.1) a crawler follows the group that names it and uses the User-agent: * group only when no group does — so this group lets it in even if your * group says Disallow: /. Token matching is case-insensitive.
Put these lines in /robots.txt at the root of each host (each subdomain has its own file). robots.txt is a request to well-behaved crawlers, not access control.
Already have a robots.txt? Paste it into the robots.txt checker to see whether it blocks Cotoyogi on a given path, and which line decides.
Who blocks it
Cotoyogi is not in the 2026-10-09 survey: it was added to this directory after we last read the most-visited sites' robots.txt files, so we have no count for it yet.
Facts
- Operator
- ROIS-DS
- Purpose
- Unconfirmed by the operator
ROIS-DS documents it as a crawler run by its Center for Research and Development on Data Lake for collecting Japanese language data resources. It does not say what the collected data is used for.
- Obeys robots.txt
- Yes. ROIS-DS gives robots.txt (RFC 9309) as the first way to stop Cotoyogi, with Disallow (including * and $ patterns) and Crawl-delay examples, and says a robots meta nofollow tag stops it following a page’s links.
- In your server logs
- The user-agent string contains "(compatible; Cotoyogi/4.0;".
- How to verify it
- ROIS-DS publishes the range of addresses it crawls from on its crawler page: 157.1.136.4 to 157.1.136.11.
What ROIS-DS says
Quoted word for word from the operator's documentation.
“Cotoyogi is a Web crawler (aka robot) operated at Center for Research and Development on Data Lake, ROIS-DS for collecting Japanese language data resources.”
“For example, the following forbids Cotoyogi to retrieve any content from your site.”
“If the repetitive accesses are annoying to you, please follow the Robots Exclusion methods or contact us as described below.”
“For example, the following directs Cotoyogi to access the site at most once per 30 seconds.”
“In a nutshell, if you put <META NAME="robots" CONTENT="nofollow"> in the HTML headers, Cotoyogi will not follow the links found in the documents.”
“For more details, please refer to RFC 9309.”
“Disallow accepts wildcard character " * " and end-of-path designator " $ " as well.”
“User agent string: Mozilla/5.0 (compatible; Cotoyogi/4.0;”
“Range of IP addresses: 157.1.136.4 - 157.1.136.11”
Sources
- ROIS-DS: About Cotoyogi Crawler — fetched and checked 2026-10-09