Technical SEO FAQ · XML Sitemaps & Robots.txt
Should noindexed URLs appear in XML sitemaps?
Avoid treating the sitemap as an inventory of URLs that you explicitly don't want indexed.
The sitemap should generally represent the URLs you want search engines to discover and process as important pages.
If a page is intentionally noindexed, its inclusion in a sitemap sends a contradictory operational signal.
Related questions
- How do sitemap errors help diagnose technical SEO problems?
- How do you design XML sitemaps for millions of URLs?
- How do you structure sitemap indexes for large websites?
- How do you troubleshoot sitemap URLs that Google isn't indexing?
- How would you safely change robots.txt on a high-traffic production website?
- Should canonicalized URLs appear in XML sitemaps?
- What are common mistakes in robots.txt?
- What happens if a URL is blocked by robots.txt but included in an XML sitemap?
- What should and shouldn't be included in an XML sitemap?