What an AI crawler that runs no JavaScript can read in your page
Paste your page's HTML source and see the text, headings, meta tags and structured data a crawler gets from the HTML alone, before any script runs. It works on pages that are not public yet. The HTML is parsed in this tab and nothing is fetched or sent.
Use view source (Ctrl+U, or Cmd+Option+U on a Mac), not the browser's Elements panel: the Elements panel shows the page after JavaScript has run.
Paste some HTML first.
The text a crawler reads
Headings outline
Title, description, canonical, language
Robots and per-crawler meta tags
Any <meta name> named robots, googlebot, bingbot or one of the 39 tokens in the directory (case does not matter), with each directive decoded.
No robots or per-crawler meta tags. Crawlers treat that as index, follow.
Structured data (JSON-LD)
No <script type="application/ld+json"> blocks.
Links
Why the HTML alone matters
A crawler that does not run JavaScript reads the HTML your server sends and nothing else. If your page is built in the browser (a React, Vue or Svelte app without server rendering), that HTML is an empty container and a list of scripts. The check above counts what is actually there.
What operators and measurements say about running JavaScript:
- Googlebot renders JavaScript, later. Google: "Once Google's resources allow, a headless Chromium renders the page and executes the JavaScript." Pages wait in a render queue that "may stay on this queue for a few seconds, but it can take longer than that." (Google Search Central, JavaScript SEO basics, last updated 4 Mar 2026, checked 24 Sep 2026.)
- Applebot may render. Apple: "Applebot may render the content of your website within a browser", and if JavaScript and CSS are blocked by robots.txt "it may not be able to render the content properly." (Apple, About Applebot, checked 24 Sep 2026.)
- Bingbot renders. Microsoft's guidelines ask sites to "Allow Bingbot to crawl and render content efficiently." (Bing Webmaster Guidelines, checked 24 Sep 2026.)
- OpenAI, Anthropic, Meta, ByteDance and Perplexity crawlers: measured not to render. Vercel's study of its own network, published 17 Dec 2024, found "none of the major AI crawlers currently render JavaScript", naming OAI-SearchBot, ChatGPT-User, GPTBot, ClaudeBot, Meta-ExternalAgent, Bytespider and PerplexityBot. It adds that ChatGPT's and Claude's crawlers "do fetch JavaScript files (ChatGPT: 11.50%, Claude: 23.84% of requests), they don't execute them." (Vercel, The rise of the AI crawler, read 24 Sep 2026.) That is a third party's measurement from 2024, not a statement by the operators, and crawlers change.
- Everyone else in the directory (Amazonbot, DuckAssistBot, the MistralAI agents, PetalBot, CCBot and the rest): whether they run JavaScript is not documented by the operator, as far as we have found, and we have read no published measurement. Assume they get the raw HTML.
The safe reading: if the words are not in the HTML, most AI crawlers do not get them, and Google gets them only after rendering.
What counts as readable text
Every text node inside <body>, in source order, with runs of spaces collapsed and each block element on its own line. Left out: <script>, <style>, <template>, <iframe>, <svg>, <canvas> and the <head>. <noscript> is shown on its own, because a client without JavaScript sees it and a browser with JavaScript never does. Text hidden with CSS is counted: it is in the HTML, and a crawler that runs no CSS reads it too.
Words are counted with the browser's Intl.Segmenter, which uses Unicode word boundaries and dictionaries for Chinese, Japanese and Thai, so 我们今天去公园散步 counts as 5 words, not 1. "Client-side" counts as 2 words and "don't" as 1.
The signs of a client-rendered shell
- An empty
#root,#app,#__nextor#__nuxtelement, the mount points React, Vue, Next.js and Nuxt fill in. - A
<body>holding only scripts and empty elements. - A
<noscript>that says to enable JavaScript. Create React App ships "You need to enable JavaScript to run this app." - Text that exists only inside a
__NEXT_DATA__,__NUXT_DATA__orwindow.__NUXT__script blob. A server-rendered Next.js page has the blob and the text in its HTML, which is fine; the check flags it only when the blob holds far more words than the page.
The fix is to send the content in the HTML: server-side rendering or static generation (Next.js, Nuxt, Astro, SvelteKit and Remix all do it), or prerendering for a plain single-page app.
What this check cannot see
- HTTP headers. An
X-Robots-Tag: noindexheader has the same effect as the meta tag but is not in the HTML. Check it withcurl -sIand your page's address. - robots.txt. If robots.txt blocks a page, a crawler never fetches it and never sees these meta tags. Google says such rules "will not be found and will therefore be ignored" (robots meta tag specification, last updated 24 Mar 2026, checked 24 Sep 2026). Check your robots.txt for that.
- What a rendering crawler adds. Googlebot, and by their operators' own words Applebot and Bingbot, may run your scripts later and see more than this page shows. This check shows the floor, not the ceiling.
- What your server sends a bot. View-source shows what your browser was sent. A bot-protection layer may send a crawler a challenge page instead. To see that, fetch the page with the crawler's user-agent, for example
curl -s -A "GPTBot" YOUR-URL, and paste the result. Each agent's full user-agent string is on its directory page. - Anything loaded later. Iframes, images, and content fetched by scripts are not followed. Nothing on this page makes a request for your markup.
- Whether a crawler honours a tag.
noaiandnoimageaiare conventions, not standards, and a meta tag named after a crawler works only if its operator reads it. Both are shown decoded, not promised.