Technical SEO FAQ · XML Sitemaps & Robots.txt
How do you troubleshoot sitemap URLs that Google isn't indexing?
For a sample:
- Inspect URL.
- Check HTTP status.
- Check robots.
- Check noindex.
- Check canonical.
- Check Google-selected canonical.
- Check content uniqueness.
- Check internal links.
- Check rendering.
- Check whether the URL belongs to a duplicate cluster.
A sitemap tells Google about a URL; it does not guarantee indexing.
Related questions
- How do sitemap errors help diagnose technical SEO problems?
- How do you design XML sitemaps for millions of URLs?
- How do you structure sitemap indexes for large websites?
- How would you safely change robots.txt on a high-traffic production website?
- Should canonicalized URLs appear in XML sitemaps?
- Should noindexed URLs appear in XML sitemaps?
- What are common mistakes in robots.txt?
- What happens if a URL is blocked by robots.txt but included in an XML sitemap?
- What should and shouldn't be included in an XML sitemap?
