When managing a site with several hundred pages, you always end up facing the same problem: buried content that neither visitors nor search engines can find. The sitemap, whether XML or HTML, directly addresses this situation. Understanding this file and knowing how to read it allows you to regain control over how a site is crawled, indexed, and used on a daily basis.
The lastmod tag in an XML sitemap: a signal often misused
Most guides on sitemaps list the available tags without specifying which ones really matter. In practice, Google has ignored the priority and changefreq tags for years. Only the lastmod value remains relevant, provided it is reliable.
A useful lastmod reflects the actual date of the last substantial modification of the content. If all the URLs in a sitemap display the same date (the one from the automatic generation of the file), the signal loses its credibility. Google eventually stops considering it for that domain.
This can be checked by directly opening the sitemap.xml file in a browser. If the lastmod dates are identical on pages that are clearly different, there is a configuration problem.
On WordPress, some SEO plugins regenerate the sitemap with every minor save, which overwrites the true dates. Disabling the automatic update of lastmod on minor changes (typo corrections, tag additions) is a practical precaution. To observe the structure of a well-organized sitemap, you can check the sitemap page of Entrepreneur Land, which illustrates a categorization by content types.
HTML Sitemap and User Navigation: A Forgotten Use by Large Sites
The XML sitemap is for bots. The HTML sitemap, on the other hand, is for humans. On a site with several hundred pages, menu navigation is not always sufficient. Secondary sections, resource pages, or old articles disappear from common user paths.
The W3C Web Sustainability Guidelines explicitly recommend a human-readable sitemap for large or complex sites. This page acts as an index: it allows a visitor to quickly locate a section they might not have found via the main menu.
The benefits are twofold:
- Users looking for a specific page can access it without going through the internal search, which is often underperforming on smaller sites
- Crawlers discover orphan URLs (pages without internal links pointing to them) thanks to the linking created by the HTML sitemap page
- Editorial teams can quickly identify outdated or duplicate content by browsing the complete list of published URLs
We often underestimate how much a well-maintained HTML sitemap reduces support requests (“I can’t find such and such a page”) on institutional or e-commerce sites.
Diagnosing an Indexing Problem from the Sitemap
When pages do not appear in search results, the usual reflex is to check the robots.txt file or noindex meta tags. The sitemap is often overlooked as a diagnostic tool.
Cross-referencing Sitemap and Google Search Console
Google Search Console indicates the number of URLs submitted via the sitemap and the number of URLs actually indexed. A significant discrepancy between these two figures signals a technical problem: 404 error pages, looping redirects, duplicate content, or excessively long server response times.
The concrete approach: open the “Sitemaps” section of the Search Console, note the submitted/indexed ratio, and then check the “Pages” report to identify the reasons for exclusion. This check takes a few minutes and avoids searching blindly.
Identifying URLs That Should Not Be in the Sitemap
An artificially inflated sitemap dilutes the signals sent to search engines. Internal search results pages, e-commerce filter pages, date archives, or tag pages with only one article have no place in an XML sitemap.
Cleaning up your sitemap improves crawl quality without changing a single line of content. On high-volume sites, this operation frees up crawl budget for pages that really matter.
Structure of the XML Sitemap File: Reading the Important Tags
An XML sitemap file contains a set of tags framed by the urlset structure. Each URL is declared in a url block, with at least the loc tag indicating the page’s address.
The truly useful tags boil down to two:
- loc: the full URL of the page, mandatory and non-negotiable
- lastmod: the date of the last modification, useful only if it is accurate and verifiable by Google
- The changefreq and priority tags exist in the specification but are ignored by Google, making them practically useless
For sites that publish specific videos or images, XML format extensions allow these contents to be declared in dedicated sitemaps. Feedback varies on the actual impact of these specialized sitemaps, but they remain relevant for sites where multimedia content constitutes the main activity.
A sitemap can contain up to a limited number of URLs per file. Beyond that, a sitemap index is created that points to multiple sitemap files. This two-level architecture is standard on e-commerce sites or online media.
Regularly consulting your sitemap, whether to check the consistency of lastmod dates, identify orphan URLs, or clean up unnecessary pages, remains a simple maintenance gesture with a direct impact on visibility. It is a file that is configured once and then forgotten, whereas it deserves at least a quarterly review.
