Indexability Checklist: 12 Reasons Your Pages Aren’t Indexed

Indexability is whether a page is technically able to be stored in a search engine’s index. If a page is not indexable, it cannot rank for anything, no matter how good the writing is. This checklist walks through 12 common reasons pages stay out of Google’s index, how to spot each one, and what to change so your important pages get a fair chance.

This guide is part of our complete website SEO audit guide, which walks through every step of auditing a small site.

Indexability vs crawlability: the quick difference

The two terms are often used interchangeably, but they describe different steps:

• Crawlability is whether a search engine can find and fetch the page. Robots.txt rules, broken links and server errors affect this.
• Indexability is whether a fetched page is allowed, and judged worthy, to be added to the index. Noindex tags, canonical tags, duplicate content and quality all affect this.

A page has to pass both stages. Google explains the full process of crawling, indexing and serving in its overview of how Google Search works. For practical purposes, think of it as a chain: discover, crawl, render, index. A break anywhere in the chain keeps the page out of results.

How to check a page’s indexability first

Before working through the list, gather some evidence so you know which problems actually apply:

1. Search Console URL Inspection. Paste the URL in. It tells you whether the page is indexed, when it was last crawled, and the reason if it was excluded.
2. Page indexing report. In Search Console, the Pages report groups excluded URLs by reason, such as “Excluded by ‘noindex’ tag” or “Crawled – currently not indexed”.
3. A site crawl. The LiteScroll SEO Crawler checks up to 25 pages free and flags indexability alongside canonical configuration, titles and headings, so you can see at a glance which pages carry a noindex or point their canonical elsewhere.

With that evidence in hand, match each excluded page against the 12 reasons below.

The indexability checklist: 12 reasons pages aren’t indexed

1. A noindex tag is present

The most common cause, and the easiest to miss. A <meta name="robots" content="noindex"> tag in the head, or an X-Robots-Tag: noindex HTTP header, tells Google not to index the page. It often slips in from a staging site, a “discourage search engines” setting in WordPress, or a plugin option applied to a whole post type.

Fix: Remove the tag or header from pages you want indexed, then request indexing in Search Console.

2. Robots.txt blocks the URL

If robots.txt disallows a path, Google cannot crawl it, and so cannot see the content. Blocked URLs occasionally appear in results without a description, but they will not rank properly. Our guide to robots.txt vs noindex explains why blocking is not the same as excluding.

Fix: Remove or narrow the Disallow rule, then test the URL again so you know indexability has been restored.

3. The canonical points somewhere else

If a page’s canonical tag names a different URL, Google treats the page as a duplicate and indexes the other URL instead. Templates that set every canonical to the homepage cause this at scale.

Fix: Use self-referencing canonicals on unique pages. See our canonical URL guide for the full set of rules.

4. The page returns an error status

Indexability starts with the server response: pages must return a 200 status code to be indexed. A 404, 410 or 5xx response removes them. Soft 404s, where a page returns 200 but shows “nothing found” or very thin content, are also dropped.

Fix: Restore the content, redirect the URL to a relevant replacement, or make sure genuinely empty pages return a proper 404.

5. The URL redirects

A redirecting URL is never indexed itself; only its destination can be. If your sitemap or internal links point to redirecting URLs, you are asking Google to index addresses that cannot be indexed.

Fix: Update links and sitemap entries to the final URL.

6. Nothing links to the page

Orphan pages, with no internal links pointing to them, are hard for crawlers to discover and look unimportant when they are found.

Fix: Link to the page from relevant posts, category pages or navigation.

7. The page is missing from the XML sitemap

A sitemap is not required for indexing, but it helps discovery, especially on new sites with few backlinks.

Fix: Make sure the sitemap lists every canonical, indexable URL and nothing else, and submit it in Search Console.

8. Content depends on JavaScript that fails to render

If the main content only appears after scripts run, and those scripts are blocked, slow or broken, Google may index an almost empty page.

Fix: Use URL Inspection’s rendered HTML view to confirm the content is present. Server-render critical text where possible.

9. The content duplicates another page

Near-identical pages, such as location pages with only the city name changed, or tag archives that mirror categories, are often filtered out as duplicates.

Fix: Merge similar pages, make each one genuinely distinct, or canonicalise duplicates to the best version.

10. The content is thin or low value

“Crawled – currently not indexed” frequently means Google fetched the page and decided it was not worth storing. Short pages with little original information are common victims. This is where indexability stops being a purely technical question and becomes a content one.

Fix: Expand the page with specific, useful information, or consolidate it into a stronger page.

11. The page is new or the site has limited crawl attention

“Discovered – currently not indexed” means Google knows the URL exists but has not crawled it yet. On new or small sites this can simply take time.

Fix: Strengthen internal links to the page, keep the sitemap current and avoid publishing large batches of low-value URLs that dilute attention.

12. Access is restricted

Pages behind a login, a password-protected staging environment, or a firewall rule that blocks Googlebot cannot be crawled at all.

Fix: Check server and security plugin settings, and confirm that public pages are actually public.

Indexability quick-reference table

Use this table to map what you see in Search Console to the likely cause:

Note that “Alternate page with proper canonical tag” is often fine. It means the canonical is working as intended. Only investigate it if the excluded URL is one you actually want in search results.

Building an indexability routine

Indexability problems tend to creep back in when themes update, plugins change or new sections launch. A light routine catches them early:

• Monthly: Review the Search Console Pages report for new exclusion reasons or sudden jumps.
• After any site change: Crawl your key pages and confirm none have gained a noindex or a foreign canonical.
• When publishing: Spot-check the new page’s robots meta and canonical. The free LiteScroll SEO Inspector Chrome extension shows page metadata while you browse, which makes this a ten-second job.
• Quarterly: Look for orphan and thin pages and decide whether to improve, merge or retire them.

If you want a structured way to do the crawl part, our walkthrough on running a 25-page SEO audit in 15 minutes fits neatly into this routine.

Many indexability problems persist simply because small sites rely on heavyweight tools they rarely open. We explain the alternative in why lightweight SEO tools win for small websites, and debunk related misconceptions in seven technical SEO myths.

Next steps

Start with evidence, not guesses: inspect the URL in Search Console, crawl your key pages, and match each excluded URL to one of the 12 reasons above. Fix the technical blockers first (noindex, robots.txt, canonicals, status codes) because they are quick and absolute. Then work on discovery and content quality, which take longer but decide whether Google wants the page once it can index it. For more lightweight checks, visit LiteScroll.com.