How to see your page the way a crawler does
You want to know what a search engine receives when it requests your page, not what your browser shows you.
Short answer
Your browser inspector shows the page after JavaScript has run, which is not what a first-pass crawler receives. To see what a crawler sees, read the raw HTML the server returned: view source, curl, or a tool that reports the served bytes. Then check that your headings, body text, internal links and structured data are actually in it.
The mistake that hides everything
Right-click, Inspect, and you are looking at the DOM after your browser downloaded, parsed and executed every script. That is the rendered page. A crawler on its first pass has none of that, and the difference between the two is exactly what you are trying to measure.
How to see the real thing
View source shows the bytes the server sent. So does curl. So does any tool that fetches the URL server-side and shows you the response. If the content appears in the inspector and not in view source, it is being added by JavaScript.
What to check once you have it
Your title and description. Your h1 and section headings. Enough body text that the page is about something. Internal links, because that is how the rest of your site is discovered. Structured data, if you have any. Canonical and robots directives.
The signals of a shell
An empty mount element with nothing inside it. A large framework payload that is content waiting to be rendered. A stack of scripts against almost no words. Any of these alone can be innocent; together they mean the page arrives later than the crawl does.
Links are the part that hurts most
Navigation rendered client-side means the first pass sees no links, so discovery of everything else waits on rendering too. This is how a site ends up with a handful of indexed pages and no obvious explanation, and it is invisible from a browser.
What a thin result does and does not mean
It does not mean Google cannot see your page: Google renders, on a second pass, on its own schedule. It does mean every crawler that does not execute JavaScript sees nothing, which now includes most AI answer engines, most link tools and most social preview fetchers.
Check the status code too
A page can return a 404 or a 500 while looking fine in a browser, particularly behind a CDN or a fallback route. The status is part of what the crawler receives, and a success code on an error page is its own problem.
Questions
- Is view source enough?
- For most checks, yes. A tool that fetches server-side additionally shows you the status code, the redirect chain and the response headers, which view source cannot.
- What if my page needs JavaScript to be useful?
- Interactivity can and should be JavaScript. The content that needs to rank, and the links that lead to the rest of your site, should be in the HTML. Those are separable concerns on almost every framework.
- How do I know if rendering is delaying indexing?
- Compare publish date against the first impressions in Search Console, for a client-rendered page and a server-rendered one. A consistent gap is your answer.
Measured, not asserted
Fetching a tool page from ihatepdf.cv as a crawler returns the whole page in the HTML: hundreds of words, seventeen headings, dozens of internal links and four structured data blocks, all present before a single script runs. That is not an accident, and it is part of why the site reached 100,542 users on search alone.
Free tool for this: Crawler View. No account, nothing uploaded.
Where this goes deeper
Every number on this page comes from one complete dataset: one product taken from zero to 100K+ users on search alone, with nothing spent on advertising. The full argument is Chapter 10 and Chapter 11 of the book. Five chapters are free to read.