Generate a split sitemap (also fix robots.txt) #4639
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Fixes #4638 |
✅ Deploy Preview for volto ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
Passing run #4939 ↗︎
Details:
This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. |
|||||||||||||||
reebalazs
changed the title
reebalazs
changed the title
`sitemap.xml.gz` remains identical. `sitemap-index.xml` can be used instead as an index file, which will link to `sitemap1.xml.gz`, `sitemap2.xml.gz`, ... The default index size is 2000 which also considers the max file size to remain under Google's limit. (50k)
Although it's all the same for Google with or without this, it's more correct not to add the content encoding gzip header, as we just want to transfer the gzipped file as a binary and not consider the browser to decode it when downloaded. (The other option would be to leave the content encoding header but then just call the file as `.xml` without the `.gz` ending. That would however only result in larger file sizes when saved and would give no extra benefit. It would also lead to non-compatible changes.)
Replace http://backend from the robots.txt provided by the backend with the public facing url. Also, publish the index file instead of the single file that would be rejected by Google.
reebalazs
force-pushed
the
ree-split-sitemap
branch
from
0c4aec5
to
b18e3aa
Compare
sneridagh
pushed a commit
that referenced
this pull request
Apr 14, 2023
sneridagh
added a commit
that referenced
this pull request
Apr 14, 2023
sneridagh
added a commit
that referenced
this pull request
Apr 17, 2023
* master: (22 commits) Release changelog notes for 16.20.1 Release 17.0.0-alpha.5 Generate a split sitemap (also fix robots.txt) (#4639) Fix search block in edit mode re-queries multiple blocks with an empty search text (#4694) Fix Move to top of folder ordering in folder content view (#4691) Changelog Revert "Add current page parameter to the route in the listing and search block pagination (#4159)" (#4695) Release generate-volto 7.0.0-alpha.4 Force the resolution of the `react-error-overlay` package to `6.0.9` (#4687) Fix training links (#4635) Release 17.0.0-alpha.4 Release changelog notes for 16.20.0 (#4684) Update to latest backend versions (#4682) Support RelationList field with StaticCatalogVocabulary and SelectWidget. (#4614) Load a theme via a `theme` key in `volto.config.js` or in `package.json` (#4625) docs: improve creating view documentation (#4636) fix sitemap.xml.gz is not compressed #4622 (v2) (#4663) Make URL a literal string to fix broken link (#4667) Move developer guidelines to contributing #4665 (#4666) Update Volto contributing to align with and refer to the new Plone co… (#4634) ...
sneridagh
added a commit
that referenced
this pull request
May 10, 2023
* master: (83 commits) Apply suggestion from browser for password field (#4524) (fix):Object.normaliseMail: Cannot read properties of null (#4558) Fix link in Volto, remove from linkcheck ignore in Documentation v6.0 (#4742) added documentation regarding the static middleware #4518 (#4736) Closes issue #4567 (#4570) Fix whitespace in locales created by the generator (#4737) Tidy up from synch with 16.x.x (#4728) Release notes from 16.20.2, 16.20.3, 16.20.4 (#4729) Use new URL 6.docs.plone.org (#4726) Security upgrade for momentjs (#4715) Don't decode querystring while adding apiExpanders (#4719) (Synchronize redundant block id in listing block on pasting) Fix duplicating listing block (#4239) Translate add-on control panel (cleaned up PR) (#4620) Fix Move to top of folder ordering in folder content view by searchin… (#4709) Fix robot.txt - the sitemap link should respect x-forwarded headers (#4704) Refactor faulty reorder elements in ObjectBrowserList widget (#4703) Release changelog notes for 16.20.1 Release 17.0.0-alpha.5 Generate a split sitemap (also fix robots.txt) (#4639) Fix search block in edit mode re-queries multiple blocks with an empty search text (#4694) ...
sneridagh
added a commit
that referenced
this pull request
May 11, 2023
* master: (185 commits) fix: unresponsive add page (#4507) Apply suggestion from browser for password field (#4524) (fix):Object.normaliseMail: Cannot read properties of null (#4558) Fix link in Volto, remove from linkcheck ignore in Documentation v6.0 (#4742) added documentation regarding the static middleware #4518 (#4736) Closes issue #4567 (#4570) Fix whitespace in locales created by the generator (#4737) Tidy up from synch with 16.x.x (#4728) Release notes from 16.20.2, 16.20.3, 16.20.4 (#4729) Use new URL 6.docs.plone.org (#4726) Security upgrade for momentjs (#4715) Don't decode querystring while adding apiExpanders (#4719) (Synchronize redundant block id in listing block on pasting) Fix duplicating listing block (#4239) Translate add-on control panel (cleaned up PR) (#4620) Fix Move to top of folder ordering in folder content view by searchin… (#4709) Fix robot.txt - the sitemap link should respect x-forwarded headers (#4704) Refactor faulty reorder elements in ObjectBrowserList widget (#4703) Release changelog notes for 16.20.1 Release 17.0.0-alpha.5 Generate a split sitemap (also fix robots.txt) (#4639) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
sitemap.xml.gzremains identical.sitemap-index.xmlcan be used instead as an index file, which will link tositemap1.xml.gz,sitemap2.xml.gz, ...The default index size is 2000 which also considers the max file size to remain under Google's limit. (50k)
Also: