How to provide a large ‘sitemap.xml’ with Next on Vercel

[eduludi]

We fetch a dynamic generated xml from our CMS API, and we return its content in a Next.js API route (that just works as a proxy).

In development this works as expected, but due the 4MB payload size limitation in Vercel functions we get a 500 error (our sitemap is really big ~12MB).

I have seen this official guide in Vercel, but looks like it's is meant for small/medium sites. And generating the sitemap.xml file after build isn't an option, as the pages and content come from out CDN.

Any ideas on how to solve this?

[amyegan]

Hi @eduludi. There are many ways you could handle it and the best option will always depend on your project and workflow.

Some of first things that come to my mind for most cases are Git Hooks and Husky. Something like that could allow you to build a new sitemap from your local env and include it in your commit. Downside is this wouldn't update the sitemap instantly each time CMS content is updated.

Another thing to look at is using Vercel with GitHub Actions which would allow you to generate your sitemap before deploying it with the rest of your site.

[eduludi]

Thanks for your answer @amyegan. The problem with these solutions is that we fetch the sitemap dynamically from our CMS: Generating it at build time doesn't makes sense for us.

[amyegan]

Ah, I misunderstood what you meant by "generating the sitemap.xml file after build isn't an option." Sorry about that! I'm glad you found a solution. Splitting up the large sitemap into smaller pieces is a good strategy for this. Thanks for coming back to share what you did to make it work!

[eduludi]

I solved it by splitting the site into multiple, smaller sitemaps as described here: https://developers.google.com/search/docs/advanced/sitemaps/large-sitemaps

This was possible for us because we can fetch from the CMS the sitemap in separated chunks, but I'm not sure how this will be solved with other CMSs / data sources.