sitemap.xml check

What is sitemap.xml?

A sitemap.xml file, or simply a sitemap, is a technical file that lists the pages you want search engines to see. People don’t need it: it’s a table of contents for search engine crawlers. Google defines it this way:

A sitemap usually lives at example.com/sitemap.xml, and its location is listed in robots.txt with a Sitemap: line. Most CMSs and SEO plugins generate the sitemap automatically and update it whenever new pages are added.

A single sitemap has size limits: no more than 50,000 URLs and no more than 50 MB uncompressed. Both the sitemaps.org protocol and Google’s documentation require this. Large websites split their list into several sitemaps and tie them together with an index file, which can reference up to 50,000 sitemaps (Google on large sitemaps).

Why does your website need sitemap.xml, and what can go wrong?

A sitemap helps search engines find pages faster, especially ones that are hard to reach through links: new products, deep catalog sections, fresh articles. Google is clear that it’s a hint, not a guarantee:

That said, Google notes that in most cases a website benefits from having a sitemap, and that large and new websites with few external links need one most. For an online store with thousands of products, a sitemap is often the fastest way to tell search engines about new arrivals.

The trouble is that sitemap problems don’t show up on the website itself. If half the pages drop out of the list, the website still opens normally — new pages just wait longer to be crawled. If the file is unreachable, the Sitemaps report in Search Console shows the status “Couldn’t fetch,” and if the XML has errors, you’ll see messages about them. Google only queues the URLs it was able to parse:

Another risk is inaccurate <lastmod> dates. Google uses this value only “if it’s consistently and verifiably (for example by comparing to the last modification of the page) accurate” (Google documentation). If a plugin stamps today’s date on every page, the hint stops working for your whole website.

When do you need a sitemap, and when can you skip it?

A sitemap is optional for small websites with a simple structure and highly recommended for large, new, and frequently updated ones. In its “Do I need a sitemap?” section, Google names size as the main criterion:

When you can skip a sitemap:

• A small website. A business card website, landing page, or portfolio with just a few pages.
• Thorough internal linking. Every page is easy to reach from the main menu or internal links — in practice, no more than 2–3 clicks from the home page. Google puts it this way: the crawler can reach all your important pages by following links from the home page.
• Infrequent changes. Your content doesn’t change often, and you don’t have many videos, images, or news articles that need to show up in search.

When you need a sitemap or should really have one:

• A large website. Online stores, media sites, and portals — from hundreds to tens of thousands of URLs. Google says that on large websites it’s harder to make sure every page is linked from at least one other page, so the crawler may miss new pages.
• A complex structure. Some pages are isolated with no links pointing to them, or there are no breadcrumbs — a navigation trail like “Home › Catalog › Coffee.”
• A new website. It doesn’t have many external links yet, and crawlers find pages precisely by following links from other websites.
• Dynamic content. Pages are added and changed often; a sitemap with honest <lastmod> dates tells search engines what’s worth checking.
• Lots of videos, images, or news. Google can use additional information about these files from your sitemap.
• You need fast indexing. So crawlers see new products or articles as soon as possible — and they can appear sooner both in search and in AI answers like AI Overviews, which draw on indexed pages.

The criteria for size, internal linking, new websites, and media files come from Google itself (Google documentation); the rest are practical guidelines.

Why do pages drop out of sitemap.xml?

The sitemap is almost always generated by your CMS or a plugin, so pages drop out of it after changes to settings or on the server:

• SEO plugin settings. After an update or a manual tweak, a whole content type — products, categories, blog posts — gets excluded from the sitemap.
• noindex pages. Most plugins leave pages blocked from indexing out of the sitemap, and that’s correct: Google recommends including only the URLs you want to see in search. But an accidental noindex on a template removes pages from both search and the sitemap at once.
• Broken XML. A PHP warning printed before the <?xml declaration, an unescaped & in a URL, or a file cut off halfway makes the sitemap fully or partly unreadable.
• Timeouts on large websites. A dynamic sitemap is built on the fly, and if the server can’t keep up, the search engine gets an error or an incomplete file.
• Bot protection and caching. A firewall or CDN serves an “Are you human?” challenge page instead of XML, or a cache serves an old version of the sitemap.
• A website migration. After a move to HTTPS or a new domain, the sitemap still lists old URLs or sits at the old path, and robots.txt points to a file that doesn’t exist.

How do you spot a sitemap.xml problem?

Most sitemap problems show up in Search Console or in the file itself, not on the website. Here are the typical symptoms and what to do about them:

How to check sitemap.xml yourself

You’ll need a browser and Google Search Console. Repeat the check after any CMS, plugin, or theme update:

1. Open example.com/robots.txt and find the Sitemap: line — that gives you the exact sitemap URL.
2. Open the sitemap in your browser. If it’s an index file, open each sitemap it lists. The browser should show an XML tree, not an error or an HTML page.
3. Press Ctrl+U and make sure the file starts with <?xml, with no text before it. Use Ctrl+F to count the <loc> entries — that’s roughly the number of URLs.
4. Spot-check a few important pages: are they in the sitemap, and do their URLs open without redirects?
5. In Search Console, open the Sitemaps report: it shows the status of the last fetch and the number of discovered pages for each sitemap.
6. Or use the FullCheckApp “sitemap.xml check” and get the results in a clear visual format.

A one-time check takes a few minutes. What’s harder is catching the moment the number of pages in your sitemap changes: for that, you’d have to write down the count every day and compare it with yesterday’s.

If you chose the last option, you’ve already made the right call: the check runs automatically, and you’ll be the first to know about any warning or critical issue.

What FullCheckApp checks

Once a day, at the time you choose, FullCheckApp opens your sitemap and counts how many pages it lists. The result is compared with the previous check, so a sharp drop in the page count shows up right away — not weeks later in Search Console.

• Page count — the number of URLs in the sitemap, compared with yesterday.
• Valid XML — if the file is broken, FullCheckApp won’t pass off the part it managed to read as a full count.
• Sitemap URL in robots.txt — whether the domain of the sitemap listed in the Sitemap: line matches your website’s domain.
• Page domains in the sitemap — whether the page URLs in sitemap.xml match your website’s domain. A foreign or old domain in the sitemap is a classic sign of an unfinished migration.
• Websites behind protection — FullCheckApp doesn’t pretend to be Googlebot. If Cloudflare or other protection blocks the request, FullCheckApp recommends allowlisting its IP addresses.

Report flags: green means OK, yellow means a warning, and red means critical. You decide how critical each check is, separately for each website. After each check, you get an email report, and critical issues are also sent to Telegram. The robots.txt file, where your sitemap URL is listed, is covered by a separate robots.txt check, and the Google page indexing check shows how Google itself sees your sitemap.

FAQ

Sources

1. Google Search Central — Learn about sitemapsdevelopers.google.com/search/docs/crawling-indexing/sitemaps/overview
2. Google Search Central — Build and submit a sitemapdevelopers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap
3. Google Search Central — Manage your sitemaps with sitemap index filesdevelopers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps
4. Search Console Help — Manage your sitemaps using the Sitemaps reportsupport.google.com/webmasters/answer/7451001?hl=en
5. sitemaps.org — Sitemaps XML formatwww.sitemaps.org/protocol.html
6. Bing Webmaster Blog — Keeping Content Discoverable with Sitemaps in AI Powered Searchblogs.bing.com/webmaster/July-2025/Keeping-Content-Discoverable-with-Sitemaps-in-AI-Powered-Search
7. Google Search Central — AI features and your websitedevelopers.google.com/search/docs/appearance/ai-features