Technical SEO FAQ · XML Sitemaps & Robots.txt
What should and shouldn't be included in an XML sitemap?
Include URLs you want search engines to discover and that represent the preferred versions of your content.
Avoid using the sitemap as a dumping ground for:
- redirects;
- obvious duplicates;
- noindex pages;
- broken URLs;
- irrelevant parameter variants.
A high-quality sitemap is a curated URL inventory.
Google describes sitemaps as a way to tell Google about pages that are new or updated.
Related questions
- How do sitemap errors help diagnose technical SEO problems?
- How do you design XML sitemaps for millions of URLs?
- How do you structure sitemap indexes for large websites?
- How do you troubleshoot sitemap URLs that Google isn't indexing?
- How would you safely change robots.txt on a high-traffic production website?
- Should canonicalized URLs appear in XML sitemaps?
- Should noindexed URLs appear in XML sitemaps?
- What are common mistakes in robots.txt?
- What happens if a URL is blocked by robots.txt but included in an XML sitemap?