An e-commerce site with several hundred references has just revamped its main menu. The result on mobile: a three-level hamburger menu, buried categories, and product pages that Google no longer crawls. The solution chosen by the technical team is not spectacular, but it has resolved the situation: a visible and up-to-date sitemap.
HTML Sitemap and Mobile Crawl: What Google Really Indexes
Since mobile-first indexing has become widespread, hamburger menus and JavaScript mega-menus pose a concrete problem. Googlebot accesses the mobile version of the site, and if the JS rendering fails or if the navigation depth exceeds three clicks, some URLs remain invisible.
See also : How to Smoke a Puff?
Google specifies in its Search Central documentation that sitemaps help discover deep URLs that are inaccessible via mobile navigation. We are talking here about the XML sitemap submitted in the Search Console, but also the HTML sitemap, the famous site map intended for human visitors. The two complement each other: the XML feeds the bot, the HTML guides the user.
On a showcase site with ten pages, the interest remains limited. However, as soon as we exceed fifty pages (product sheets, blog articles, service pages by city), a sitemap becomes a safety net. You can see how the Cyber Business sitemap organizes its sections to make each page accessible in one click from a single entry point.
See also : How to Enhance Your Bike Security with a Decathlon Lock?
Sitemap and Indexing Coverage in the Search Console
A common symptom in the Search Console: dozens of URLs with the status “Discovered, currently not indexed.” This status means that Google knows the page but has not deemed it useful to crawl. The causes vary, but one of them is the lack of a strong internal link pointing to these pages.
Adding or updating an XML sitemap improves the indexing coverage of these forgotten sections. Recent audits via the Search Console confirm that the regular submission of a sitemap reduces the number of pages in this state, particularly on e-commerce sites with deep categories.
The HTML sitemap plays a complementary role. It creates a flat internal linking structure: each page of the site is linked to the map, and the map is linked to the homepage. For Googlebot, it’s a shortcut. For the user, it’s an emergency exit when the main menu is not sufficient.
What to Include in an HTML Sitemap (and What to Exclude)
Not all pages deserve to appear on the map. We include pages that provide value in natural SEO and navigation:
- Main category and subcategory pages, the ones that users search for via Google
- Blog articles or guides targeting long-tail queries
- Key service or product pages, especially those lacking internal links in the menu
We exclude pages that are of no interest to the visitor: internal search results pages, pagination pages, duplicate tag pages, terms and conditions pages (except in GDPR cases, see below). A sitemap that lists three thousand URLs without hierarchy serves no one.
Sitemap and GDPR Compliance: Direct Access to Legal Pages
This point often goes under the radar of web teams. The CNIL recommends simple and permanent access to legal notices and the privacy policy. Several guides from agencies specializing in digital law cite the sitemap as a quick access point to legal pages and GDPR rights forms.
In practice, during a compliance audit, a DPO checks that the user can reach the privacy policy page and the rights exercise form in a maximum of two clicks. The footer usually does this job, but on some sites (SPA, web applications, sites with dynamic footers), the sitemap offers an additional guarantee.
This is not a legal gimmick. On e-commerce sites subject to enhanced obligations (payment data, third-party cookies), this redundancy of access facilitates the demonstration of compliance during an audit.
Creating and Maintaining a Sitemap: Concrete Technical Constraints
The difficulty is not in creating a sitemap. The difficulty is in maintaining it. A static HTML sitemap becomes outdated as soon as a page is added or removed. Two approaches work:
- Automatic generation via a CMS plugin (on WordPress, several extensions rebuild the map with each publication)
- Server-side script generation, which queries the database and produces an up-to-date HTML page daily
- Synchronization with the XML sitemap: the HTML map takes the same structure as the XML file submitted to Google, ensuring consistency between what the bot and the user see
Synchronizing the HTML sitemap and the XML sitemap avoids inconsistencies in internal linking. If a page appears in the XML but not in the HTML (or vice versa), we send contradictory signals to search engines.
Update Frequency and Common Errors
On a site that publishes several pieces of content per week, a daily update of the map is reasonable. On a showcase site updated once a month, a weekly regeneration is sufficient.
The most common mistake: leaving broken links in the sitemap. A URL returning a 404 error from the map degrades the user experience and wastes crawl budget. Feedback varies on the ideal frequency of checks, but a monthly audit of the map’s links remains a good practice.
Another pitfall: placing the sitemap only in the footer without contextual links. If no page points to the map outside of the footer, its weight in internal linking remains low. A link from the custom 404 page (“Lost? Check our sitemap”) or from the secondary menu enhances its visibility.
The sitemap is neither a relic from the 2000s nor a tool reserved for large portals. On a medium-sized site with a mobile menu that hides sections, it remains the most direct way to ensure that every page is accessible, indexable, and compliant with legal requirements. The only prerequisite: that it is up to date.