Github Pages (free version) - sitemap.xml not being fetched by Google

[books-bittencourt]

🏷️ Discussion Type

Question

💬 Feature/Topic Area

Pages

Body

Many people are complaining about the sitemap of their websites hosted on Github Pages not being read/fetched by Google Search Console. None of the suggestions found on the internet work. Any NEW suggestions? Does anybody know why this is like this, why Github doesn't fix this?

Thanks.

[maheerCodes]

hey @books-bittencourt 👋 .
this is actually a known pain point with github pages ,,,a few angles worth checking that arent always covered in the common suggestions :

• check the actual content-type header github pages serves for your sitemap,,,run curl -I https://yoursite.github.io/sitemap.xml and look at the content-type. github pages sometimes serves .xml files as text/ plain instead of application /xml or text / xml,,,and google search console can silently fail to parse it if the content-type is wrong.this is a commmon but lesser known cause.
• verify the sitemap is actually reachable without a trailing redirect ,,,github pages handles custom domains with a redirect from the .github.io url to your custom domain,and if your sitemap submission url doesnt match exactly(http vs https ,www vs non-www) google can fail the fetch even though the file loads fine in a browser .
• double check your robots.txt isnt accidentally blocking /sitemap.xml itself,, sounds obvious but its a surprisinglyy common cause when people copy a robots.txt template without checking it,
• try resubmittting through a raw fetch test in search console's url inspection tool(not just the sitemaps tab),,, that gives you the actual http response google received ,which is way more diagnostic than the generic "couldnt fetch" status.

as for why github hasnt fixed this ,,, its likely because pages is a static file host at its core and doesnt control response headers granularly the way a custom server would,,,,,so its less a "bug" and more a structural limitation of the platform. a _headers style config (like netlify has)would solve this cleanly but github pages doesnt support that yet

worth trying the curl content-type check first since thats the most common silent culprit 💙

[DEVAL-020]

Hi,

The "Couldn't fetch" status in Google Search Console does not necessarily mean there is a problem with GitHub Pages itself. It only means that Google was unable to retrieve or process the sitemap when it attempted to access it.

A few things I would check first:

1. Open the sitemap URL directly in a browser and verify that:

   • It returns HTTP 200 OK.
   • It is valid XML.
   • It is not redirecting unexpectedly.
   • It is not returning an HTML page instead of XML.

2. Test the sitemap URL with:

   • Google's URL Inspection tool.
   • Third-party sitemap validators.

3. Verify that robots.txt is accessible and contains the correct sitemap reference, for example:
   Sitemap: https://your-domain.com/sitemap.xml

4. If you are using a custom domain with GitHub Pages:

   • Check DNS configuration.
   • Verify that HTTPS is enabled and working correctly.
   • Confirm that both the website and sitemap are reachable from the same canonical domain.

5. Check whether the sitemap exceeds Google's limits or contains malformed URLs, invalid dates, unsupported tags, or URLs that return errors.

Regarding GitHub Pages specifically, many reports of "Couldn't fetch" eventually turn out to be:

• Sitemap formatting issues.
• DNS or HTTPS misconfigurations.
• Temporary Google Search Console fetching problems.
• Domain migration or canonical URL mismatches.

Could you share:

• The website URL.
• The sitemap URL.
• Whether you are using github.io or a custom domain.
• The result of opening sitemap.xml directly in a browser.

Without the actual sitemap URL, it's difficult to determine whether the problem is with GitHub Pages, the sitemap itself, or Google Search Console's crawler.

[attipaci]

This is a pervasive problem with GitHub pages and GSC. While my non-GitHub pages sites fetch the sitemap successfully, and almost immediately, the GitHub pages (6 of them) are stuck in 'couldn't fetch' for many months now.

Interestingly, my older GitHub pages (with HTTPS not enforced) seem to be unaffected (or affected differently), which is also strange -- and seems to point to an issue on GitHub's end. For the older repos, the GSC processes the sitemap without the "couldn't fetch" error, but doesn't actually read the sitemap either. I.e., the sitemap status on GSC indicates 'Success' immediately after submitting a new sitemap, but the 'Last read' entry continues to be stuck on some ancient date, such as "Oct 13, 2024".

Bing has no issue with any of these sites (for the old or new repos) either, confirming that the sites and the sitemaps are generally functional -- it's just GSC that cannot access them.

Finally, it all used to work fine until about 1.5 years ago. This is something that broke since then.

It affects a lot of users (the searches for "Google search console github pages sitemap") are inundated with people reporting the problem. I really hope someone actually fixes this, rather than repeating the same useless suggestions that have been posted in different discussions a thousand times, and never actually resolved the issue for anyone...