Why AI assistants cannot read your React SPA

An AI crawler asks for your HTML and reads whatever comes back

Open a React single page application in a browser and you see a finished page. That page is not in the file the server sent. The server sent a shell: a <div id="root">, a script tag, a few meta tags. The browser downloads the bundle, executes it, calls your API and paints everything you can see.

GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User and PerplexityBot do none of that. They request the URL, read the bytes that come back, and move on. No JavaScript engine, no waiting for hydration, no API calls on your behalf. Anything absent from the raw HTML response does not exist for them.

Googlebot is the exception that confuses everyone. It does render JavaScript, on a delay and on a budget, which is why a client-rendered site can rank acceptably in Google and still be a blank page in every assistant your buyer actually asks. On one bilingual hotel site we took over in June 2026, Google had indexed only 4 of roughly 30 pages before we prerendered it.

What is missing from the raw HTML is usually more than people expect:

• Text that renders after hydration, including your H1 and first paragraph.
• Copy fetched in a useEffect from an API or a database at runtime.
• Tab and accordion panels that only mount when the panel is active.
• JSON-LD injected by a client-side helper instead of written at build time.
• Anything gated behind a loading spinner, a cookie banner or a scroll trigger.

Test your own site in two minutes

Do not take anyone's word for this, including ours. Point curl at your own page with a crawler user agent and count what comes back.

First, fetch the page as GPTBot sees it:

curl -s -A "GPTBot/1.2 (+https://openai.com/gptbot)" https://example.com/ -o crawler.html
wc -c crawler.html

Byte size lies, so measure text, not bytes. On one site we audited, the shell was 3,559 bytes and contained no readable content at all. This counts the characters a model could actually read:

python3 - <<'PY'
import re
h = open('crawler.html').read()
h = re.sub(r'(?is)<script.*?</script>', ' ', h)
h = re.sub(r'(?is)<style.*?</style>', ' ', h)
t = re.sub(r'(?s)<[^>]+>', ' ', h)
print(len(' '.join(t.split())), 'characters of readable text')
PY

A real content page lands in the thousands. An empty shell lands in the low hundreds, most of it your meta description. Then check that the page states what it is:

grep -Eio '<title>[^<]*|<h1[^>]*>[^<]*' crawler.html
grep -c 'application/ld+json' crawler.html

Next, make sure nothing at the edge is quietly refusing the bots. Bot protection and firewall rules return 403 or a JavaScript challenge that no AI fetcher can solve, and your analytics will never show it:

for ua in "Mozilla/5.0" "GPTBot/1.2" "OAI-SearchBot/1.0" "ClaudeBot/1.0" "PerplexityBot/1.0"; do
  printf '%-22s ' "$ua"
  curl -s -o /dev/null -w '%{http_code}\n' -A "$ua" https://example.com/
done

Every line should read 200. Finally, sweep the whole site instead of one lucky URL, and sort the worst pages to the top:

curl -s https://example.com/sitemap.xml | grep -oE '<loc>[^<]+' | cut -c6- |
while read -r url; do
  n=$(curl -s -A "OAI-SearchBot/1.0" "$url" | sed -E 's/<[^>]+>/ /g' | tr -s ' ' | wc -c)
  echo "$n  $url"
done | sort -n | head -20

What zero actually looks like

We rebuilt a Spanish premium tableware store in the summer of 2026. When we measured the old site the way you just did, the answer was not "thin content": across the 65 public URLs, the pages served 0 characters of readable text to AI crawlers. Product names, prices, series descriptions, shipping terms, all of it arrived only after JavaScript ran.

After the rebuild, the same sweep on 4 August 2026 returned 105,212 characters across the same 65 URLs. Nothing was written for robots. It is the same copy a human reads, moved from runtime into the HTML file. That is the whole trick, and it is the only measurement in this article that we ran ourselves.

The three ways to fix it

Static site generation, the right default

At build time you render every route to a real HTML file. With Vite and React, vite-react-ssg does this without leaving your stack: you export your routes, provide getStaticPaths for dynamic segments, and the build writes dist/pricing/index.html with the text already in it. The site stays a SPA once hydrated, so navigation still feels instant.

Use it when the route list is known at build time: marketing sites, service pages, blogs, documentation, catalogues. Content that changes daily is not a blocker, a scheduled deploy hook rebuilds it. We run the tableware catalogue this way, with a daily refresh.

Server-side rendering, when the page depends on the request

Next.js, Remix or a Vite SSR setup render the HTML per request. You need this when the page genuinely cannot be known in advance: live inventory across thousands of SKUs, search results, anything behind a login. The price is a running server, cache design and more ways to fail at 3am. Do not adopt SSR because it sounds more modern than SSG. Adopt it when a page's content depends on who is asking.

Crawler prerendering at the edge, the retrofit

When a large SPA cannot be restructured now, an edge middleware can serve crawlers a real HTML body while humans keep the app untouched. We use this on a corporate site with roughly 700 news articles: the middleware detects the bot, returns the H1, the article body and the JSON-LD, and everyone else gets the SPA.

Two conditions make this safe. The text served to bots must be the same text a human sees, verified by an automated parity test in CI, otherwise it is cloaking. And treat it as a bridge, not a destination, because you now maintain two rendering paths.

Which one

• Route known at build time, content changes daily or less: SSG.
• Content depends on the request or the user: SSR.
• Large existing SPA you cannot rebuild this quarter: edge prerender, with a parity test.

What does not fix it

• An llms.txt file. Consumer assistants do not request it and Google ignores it. It is useful for coding agents, not for visibility.
• JSON-LD on its own. In a 2026 Ahrefs study of 1,885 pages against 4,000 controls, added schema produced no meaningful uplift in AI Overviews, AI Mode or ChatGPT. Visible text is the primary layer, structured data mirrors it.
• Writing "for AI": chunking pages into fragments, AI-only markdown copies, invented schema types. Google's 2026 guidance says these do not help, and AI-only pages are a cloaking risk.
• nosnippet or max-snippet:0 on pages you want quoted. They remove the eligibility you are trying to earn.

Verify, then keep verifying

Run the sweep again after you deploy, and put a version of it in CI so a refactor cannot silently take the text back out of the HTML. Watch the server logs rather than your analytics: crawlers do not run JavaScript, so GA4 and Plausible never see them, and Vercel or Cloudflare logs filtered by user agent are the only honest record of whether the bots came back.

If you would rather have someone measure it and fix it with you, Polargate runs this as part of a paid, fixed-price Discovery Sprint. Either way, run the curl commands first. The number you get is not an opinion.