Three systems stand between a page and a result. A crawler fetches, an index stores, and a ranker orders, and they run at different times. Marrowbone Press, a letterpress printer in Leicester, put a page about wedding invitations online on 4 February 2026. It was fetched on 6 February, stored on 19 February, and first appeared for anybody's query in the middle of March.
Those three dates are the mechanism. Nearly every question that gets called a ranking problem is really a question about which of the three a page failed at, and the three fail for unrelated reasons and are fixed by unrelated people.
What each of the three actually holds
The crawler holds a queue of addresses and whatever came back over HTTP. It decides nothing. It fetches, reads the links it finds, and puts more addresses in the queue.
One page, four weeks
The four things that happen to a page, in the only order they can happen in
Drawn as a staircase rather than a list because none of these can be taken out of turn: nothing is ordered that was never stored.
- 01Fetched6 February. A queue, a request, and a copy of what came back.
- 02Processed and stored19 February. Tokens pointing at documents, not the page anybody wrote.
- 03Matched to a queryOnly when somebody types. The index supplies the candidates.
- 04Ordered against the restMiddle of March, and computed again from scratch every time.
The staircase says nothing about the gaps. Two days to the first step and thirteen to the second, at one small site, on one page, once.
The index stores a processed copy, not the page you wrote. Markup is stripped, script is run or not run, the text is cut into tokens, and what is kept is a set of entries pointing from a word to the documents that contain it. The page as a reader sees it is not in there at all, which is why a paragraph that a browser assembles after the fetch can be missing from the stored copy while looking perfectly present on screen.
The ranker holds nothing about the page until somebody types. There is no stored score per page; the order is computed for one query at a time, from the candidates the index can supply for that query. A page does not have a ranking. It has a position, for one query, at one moment, for one person in one place.
Crawled and not indexed is not a crawling problem
Search Console separates the two, and the two lines mean opposite things. Crawled, currently not indexed says the fetch worked and the storing was declined: the page was read and judged not worth keeping. Discovered, currently not indexed says it was never fetched. Crawled and not indexed is a different report line from not crawled, and the work each one calls for shares nothing.
Search Console lines
Two index exclusions that require opposite fixes
Marrowbone had thirteen pages in the first column and forty-one in the second; treating them as one problem wastes weeks.
A crawler cannot value what it has not fetched, and rewriting an unlinked page changes nothing until a link reaches it.
Marrowbone had thirteen pages sitting on the first line and forty-one on the second. The thirteen needed writing. The forty-one needed a link from somewhere, because nothing on the site pointed at them and a crawler cannot want what it has not been shown.
The disallow that hides the noindex
340 old proof pages sat under a proofs directory and should never have been listed. Somebody added a noindex tag to every one of them and a disallow line to robots.txt on the same afternoon, on the reasonable theory that two locks hold better than one. They cancel each other. A disallow in robots.txt stops the crawl, so the noindex on the page is never read, and a page blocked in robots.txt can still be listed, with a URL and no description, because the address was found in a link somewhere else and nothing was ever fetched that could say otherwise.
340 proof pages
How robots.txt Disallow breaks the noindex removal chain
Putting two locks on the same directory cancels the inner one because the crawler stops before reading the page tag.
- Link found elsewhereSearch engine still discovers the /proofs/ URL from external references.
- robots.txt Disallow blocks crawlWeak link: crawler halts at the door, so the HTML response is never fetched.
- noindex tag sits unread340 proof pages remain indexed as bare URLs without descriptions.
- Disallow lifted on 2 March 2026Bots read the noindex tag and the pages dropped out over five weeks.
Allow the crawl first, let noindex clear the index, and add Disallow afterwards only if crawl budget demands it.
The order that works is the opposite of the intuitive one. Allow the crawl, let the noindex be read, wait for the pages to fall out, and only then disallow if there is a crawl reason to. Marrowbone took the disallow off on 2 March 2026 and the proof pages left the index over the following five weeks.
Where this stalls: one word for three systems
The stall is vocabulary. A page is not ranking, and that sentence covers a page never fetched, a page fetched and not stored, and a page stored and sitting twelfth. Three failures with nothing in common, described by one complaint, and the complaint sends everybody to the writing because the writing is the part that feels like work.
Diagnosis before rewriting
One complaint on the surface, three unrelated system failures below
Saying a page is not ranking sends everyone straight to editing copy before checking whether the page was ever fetched or stored.
- 'The page is not ranking'Single catch-all phrase that hides which system actually stopped
- Never fetched by crawlerUnlinked or blocked; rewriting copy changes nothing
- Fetched, declined by indexRead and judged too thin or redundant to store
- Stored, sitting twelfthIndexed and eligible, just outranked for that query
Checking the index first settles roughly a third of copy arguments before anyone opens an editor.
The tell is an argument about the wording of a page that nobody in the room has confirmed is in the index. That check takes forty seconds and it ends about a third of these arguments before they start.
What the results page is made of in 2026
The query letterpress wedding invitations leicester returns a generated answer of four sentences naming three sources, then the blue line, the breadcrumb, the snippet, and a box of four adjacent questions. Being named in the answer is a citation and a citation is not a position, which is a genuinely new distinction and the reason the fourth cluster below this one exists.
One query, 2026
What a results page is actually made of now
The first thing on it is not a result, and being inside it pays differently from being under it.
- 01A generated answer, four sentences, naming three sources
- 02Marrowbone Press | Letterpress Wedding Invitations
- 03marrowbonepress.co.uk / invitations / wedding
- 05Four adjacent questions
- 01The answer panelBeing named here is a citation, and a citation is not a position.
- 02The blue lineRewritten by the engine when it judges a heading on the page a closer fit.
- 03The breadcrumbBuilt from the address or from structured data, and never from the menu.
- 04The snippetTaken from the page itself whenever the description does not answer the query.
- 05The question boxEach one a query the engine believes sits next to this one.
Drawn from a spec rather than a screenshot, so it is still readable when the layout changes, which on this query it has twice since 2024.
Google Search Console's URL Inspection reports which of the three stages a given address has reached, including the stored copy the ranker will actually see. It is free, and it replaced searching for your own page and guessing from the absence. Bing Webmaster Tools is also free and worth having for one reason: it replaces the assumption that every engine sees the same thing, and where the two disagree, the disagreement is usually the crawl.
Neither one shows the generated answer being assembled. Marrowbone can see that the invitations page is indexed and can see the answer naming a competitor, and nothing available in 2026 connects those two facts.
The four parts of the mechanism
Getting fetched at all is how a page is found in the first place, and the forty-one unlinked pages are what its absence looks like. Being kept is being stored, and why a crawled page may not be, which is the thirteen.
Four layers and a control rule
How the full search mechanism stacks up
Every outcome on the results page depends on the layers underneath it, while timing between removal and re-indexing stays opaque.
- Generated answer citationDraws on the index like links do, but rewards a citation rather than a position.Answer panel
- Query-time orderingComputed fresh for one query, one moment, and one searcher; stored nowhere.Ranker
- Processing and storageDecides what stays; thirteen crawled pages were dropped and 340 proofs took five weeks to exit.Index
- Discovery and fetchingFinds linked addresses; forty-one unlinked pages never reached this stage.Crawler
- Crawl and index directivesrobots.txt and noindex govern whether any layer above is allowed to operate.Control layer
No tool reports how long a bulk removal will take; Marrowbone's 340 pages cleared one by one over five weeks.
The order itself is what actually decides the order, computed per query and stored nowhere. Being named above all of it is being cited when the answer is generated, and it runs on the index like everything else while paying out differently.
The robots.txt line and the noindex tag both belong to the layer that decides whether any of the rest counts. What the four sentences in the answer panel were answering, and whether anybody was going to click anything afterwards, is finding what people type, and what they mean by it.
The five weeks it took the proof pages to leave is the number nobody at Marrowbone can explain. Nothing published says how long it should take, the pages went one at a time rather than together, and whether the next 340 would take five weeks or five months is not knowable from anything the shop can see.