New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Be able to split sitemaps #562

Open
johndoudou opened this Issue Dec 20, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@johndoudou
Copy link

johndoudou commented Dec 20, 2018

In case of large site generation (100k pages), sitemaps get huge and robots (for example, Google crawler) is not able to parse it all.

From the Google guidelines,

All formats limit a single sitemap to 50MB (uncompressed) and 50,000 URLs. If you have a larger file or more URLs, you will have to break your list into multiple sitemaps. You can optionally create a sitemap index file (a file that points to a list of sitemaps) and submit that single index file to Google.

It would be great that Zola limits the maximum number of URLs in a single sitemap (for example, defaulted to 20k) and splits the sitemap in multiple sitemaps if we reach this value ; and maybe be able to configure this value somewhere.


Another solution would be to give the pages, sections, tags, categories (https://www.getzola.org/documentation/templates/sitemap/) to any page named sitemap*.xml (with the wildcard, and not only sitemap.xml), so that we can implement some logic ourselves to be able to count the number of URLs displayed in each sitemap.
This solution has the advantage to allow custom logic, and not only the maximum number of URLs, to be implemented. However, this logic has to be written for every project, and is not implemented by default.

@Keats

This comment has been minimized.

Copy link
Collaborator

Keats commented Dec 20, 2018

I didn't know this was a thing :o

I'm wondering if there is some overlap with #365 and your second solution.
The logic to implement that in Zola automatically would be fairly simple I think and has the advantage of doing the right thing automatically.

@Keats Keats added the enhancement label Jan 2, 2019

@johndoudou

This comment has been minimized.

Copy link
Author

johndoudou commented Jan 22, 2019

Thanks for your reply !

Agreed, I think the first solution is better too.
Should I create "New issue" from this post or something ?

@Keats

This comment has been minimized.

Copy link
Collaborator

Keats commented Jan 22, 2019

Do you mean to create a pull request? Fork the repository, select the next branch and start adding some code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment