The fetched HTML and the rendered page are two different documents, and only one of them is guaranteed to be read. Teignmouth Tackle rebuilt its shop on a JavaScript framework in October 2025, and 340 product pages arrived as one empty div with a script tag. Every browser anybody tested showed a perfect page.
By January the shop had lost most of its search traffic and nobody could find a mistake, because there was not one to find in the ordinary sense. The pages worked. What arrived when a crawler asked for them was an empty container and an instruction to go and fetch something else.
The two documents, and how to see each
View source shows what arrived, and the inspector shows what the script built. That is the check, it takes ten seconds, and almost nobody does it because both windows show the same site and one of them is the one people already have open.
Rendering
One product page, in the four states it passes through
A browser shows the last frame and a crawler may only ever see the first, and the whole fault lived between the two.
- 01What the server sentAn empty div and a script tag. No product name, no price, no links.
- 02What the crawler stored on the first passThe same thing, because storing does not wait for rendering.
- 03What rendering would have producedQueued, undated, and not guaranteed to run at all.
- 04What every tester sawA finished page, in a browser, which runs the script immediately.
Each frame is a state rather than a step somebody performs. The third one is the only frame in this strip that may never happen, and nothing reports whether it did.
Rendering happens later and is not guaranteed to happen at all. A crawler fetches, queues the page for rendering, and comes back to it when there is capacity, which may be minutes or weeks and is not reported anywhere. A page whose content only exists after rendering is a page whose content is provisional.
What has to be in the delivered HTML
Put the words and the links in the HTML the server sends. The words, because that is what an index stores; the links, because that is what a crawler follows. Teignmouth's category pages built their product links in script, so the 340 products were not merely empty, they were unreachable: nothing in the delivered document pointed at any of them.
HTML delivery
What must arrive in the server HTML before any script runs
An index stores the words in the response and a crawler follows the anchor links in that same document.
- 01Sea rods and reels
- 02Descriptions and prices
- 03/product/shore-rod
- 04Invisible to raw fetch
- 01Category headingSent directly in the HTML response so the first pass reads the topic.
- 02Product wordsPlaced in the delivered markup so indexing does not wait for script execution.
- 03Anchor href linksStandard links that let crawlers discover all 340 product pages.
- 04Script onclick routingNavigating only via click handlers hides destinations from non-rendering crawlers.
Teignmouth built category links in script, leaving 340 products unreachable from the delivered markup.
A link that only exists as an onclick handler is not a link. This is the same rule in its most common form: a clickable element that navigates by running code is invisible to anything that does not run the code, and a great deal of framework routing is written that way by default.
The fix, and what it costs
Server side rendering or prerendering, and both are a build decision rather than a setting. Rendering on the server sends a finished document and hydrates it in the browser; prerendering builds the pages ahead of time and serves them as files. Teignmouth chose prerendering because the catalogue changes weekly rather than hourly, and the shop recovered 300 of the 340 within nine weeks of the fix.
Remediation
Pages recovered within nine weeks of switching to prerendering
Prerendering builds finished HTML files ahead of time, which fits a catalogue that updates weekly rather than hourly.
Product pages recovered out of 340
340 total products
Prerendering brought back 300 of the 340 pages; the remaining forty also had changed URLs from the same rebuild.
The forty that did not come back are the ones whose addresses had also changed in the rebuild, which is a separate fault with a separate remedy at moving a page without losing what it earned. Two problems shipped on the same day and only one of them is about script.
Where this stalls: everybody tested it in a browser
The stall is that the failure is invisible to every ordinary check. The site loads, the pages look right, the analytics record visits, and a person clicking around finds nothing wrong. Teignmouth's agency tested on three devices and two browsers and every test passed, because a browser is exactly the environment in which this fault does not appear.
Diagnosis
Why every browser check passes while crawlers see an empty shell
Testing inside a browser runs the script immediately, hiding the very gap that URL inspection and curl expose.
- Clean display across devicesTesting on three devices and two browsers runs script before painting anything.
- Recorded site visitsAnalytics scripts fire normally once a visitor browser builds the page.
- Empty container in curl outputFetches the raw server document with no rendering so missing product names show immediately.
- Rendered HTML in URL inspectionSearch Console shows what the crawler produced, exposing an empty div in about forty seconds.
Inspecting the raw response with curl and the crawled render in Search Console replaces guessing from missing traffic.
Test with the URL inspection tool, not with a browser. Search Console's inspection reports the rendered HTML the crawler actually produced, which is the only view that answers the question, and it is free. It replaced guessing from the absence of a page in results, and at Teignmouth it showed an empty div in about forty seconds.
curl fetches the delivered document with no rendering at all, free and already on the machine, and it replaces trusting a browser, which runs script before showing anybody anything. Running it against a product page and searching the output for the product name is the fastest version of this check and needs no account.
The parts that are not script
A framework does not make a site unindexable by itself, and saying so matters because the opposite belief leads people to rebuild working sites. What breaks is content that exists only after rendering. A site that delivers its words and links in HTML and uses script for behaviour is in no danger at all, and that is most sites built with care.
Architecture
Where JavaScript sits cleanly alongside indexable HTML
A framework causes no harm when words and links ship in HTML and script handles only interactive behaviour.
- Initial fetchWhether the delivered document is requested and read in the first place.Crawl
- Delivered HTML contentCore words and anchor links present in the server response before scripts execute.Baseline
- Script-driven behaviourClient-side enhancements layered on top without withholding essential content.Interactive
- Index storageWhether a crawled page is kept once the document has been fetched and read.Retention
Only content that exists solely after client rendering breaks; keeping words and links in HTML avoids rebuilding working sites.
Whether the delivered document is fetched in the first place is how a page is found in the first place, and whether it is kept once fetched is being stored, and why a crawled page may not be. This is the layer between the two, and it belongs with the rest of the layer that decides whether any of the rest counts.