Don't follow type tables for incremental sitemaps #2393
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Incremental sitemaps ingest replication packets and follow the tables therein to find changes to pages in the sitemaps. If a type table is updated, like release_group_primary_type for example, we'd end up trying to fetch every page for every release group using a modified type. (It doesn't matter which columns were changed.) I can't think of a situation where changing a type would significantly affect the content of so many pages that we'd want search engines to re-index them all, so I'm having the code skip these type tables entirely.
reosarevok
approved these changes
Jan 24, 2022
reosarevok
added a commit
that referenced
this pull request
Jan 24, 2022
* master: Update POT files using the production database Update translations from Transifex MBS-12155: Filter out downvoted tags from JSON WS user-tags/genres (#2385) MBS-12161: Expand, not replace, FormHandler Text's validate (#2384) Don't follow type tables for incremental sitemaps (#2393) MBS-12132: Allow Apple Music music-video links for releases (#2375) MBS-12127: Allow batch-changing all blank artist credits (#2364) Delete CODE_OF_CONDUCT.md and use org default (#2390) Respond with Bad Request for "invalid session ID" (#2369) MBS-12137: Remove Gravatar (#2366) Disable broken Amazon cover art lookup Bump Flow to 0.169.0 Fix uninitialized value in string eq warning Fix use of uninitialized value in SetCoverArt Filter empty string IDs in Data::Edit::load_all Fix uninitialized value warning sending email Fix undef var warning on user tags page Fix warning submitting unlinked artist credits Fix track_count warning for medium edits Bump cookie to v0.4.1 Bump shell-quote to v1.7.3 Bump ShellJS to v0.8.5 MBS-12121: Actually show parent changes in EditRelationshipType (#2360) Remove unneeded Number() cast MBS-12164: Use formatCount also on tag cloud numbers MBS-12167: Use returnToCurrentPage in UserSubscriptions action (#2383) MBS-12159: Add support for Amazon.pl (#2380) MBS-12142: Use formatCount also on search result numbers Update package.json github URLs to use https Revert private to false Change private to false Bump Flow to 0.168.0 MBS-12133: Allow % in jazzmusicarchives URLs (#2370) MBS-11995: Test error messages in external links editor (#2290) MBS-7727: Restrict Soundcloud to only "get the music" rels (#2321) Use a valid SPX License expression More information available at https://spdx.org/licenses/ MBS-10715: Block amzn.to redirect link MBS-10716: Block apple.co redirect links Use texp where possible Use commaOnlyListText where possible Use bracketedText where possible Use addColonText where possible MBS-12049: Convert the edit index page to React Convert Edit::Open test to use real data Convert Edit::Show test to use real data Remove no longer needed tt functions for edit display Remove unused import
reosarevok
added a commit
that referenced
this pull request
Jan 31, 2022
* beta: Update POT files using the production database Update translations from Transifex Fix Data::Recording test flakiness Remove parallel processing on prove Increase default avatar size & remove bg/border Add a default editor avatar for display Fix regression in medium track count comparison (#2396) Update POT files using the production database Update translations from Transifex MBS-12155: Filter out downvoted tags from JSON WS user-tags/genres (#2385) MBS-12161: Expand, not replace, FormHandler Text's validate (#2384) Don't follow type tables for incremental sitemaps (#2393) MBS-12132: Allow Apple Music music-video links for releases (#2375) MBS-12127: Allow batch-changing all blank artist credits (#2364) MBS-11986: Block spammer profiles from being seen MBS-11986: Block accounts with spammer flag from logging in MBS-12128: Split negative and positive user "privileges" MBS-11986: New account flag: Spammer Delete CODE_OF_CONDUCT.md and use org default (#2390) Respond with Bad Request for "invalid session ID" (#2369) MBS-12137: Remove Gravatar (#2366) Disable broken Amazon cover art lookup Bump Flow to 0.169.0 Fix uninitialized value in string eq warning Fix use of uninitialized value in SetCoverArt Filter empty string IDs in Data::Edit::load_all Fix uninitialized value warning sending email Fix undef var warning on user tags page Fix warning submitting unlinked artist credits Fix track_count warning for medium edits Bump cookie to v0.4.1 Bump shell-quote to v1.7.3 Bump ShellJS to v0.8.5 MBS-12121: Actually show parent changes in EditRelationshipType (#2360) Remove unneeded Number() cast MBS-12164: Use formatCount also on tag cloud numbers MBS-12167: Use returnToCurrentPage in UserSubscriptions action (#2383) MBS-12159: Add support for Amazon.pl (#2380) MBS-12142: Use formatCount also on search result numbers Update package.json github URLs to use https Revert private to false Change private to false Bump Flow to 0.168.0 MBS-12133: Allow % in jazzmusicarchives URLs (#2370) MBS-11995: Test error messages in external links editor (#2290) MBS-7727: Restrict Soundcloud to only "get the music" rels (#2321) Use a valid SPX License expression More information available at https://spdx.org/licenses/ MBS-10715: Block amzn.to redirect link MBS-10716: Block apple.co redirect links Use texp where possible Use commaOnlyListText where possible Use bracketedText where possible Use addColonText where possible MBS-12049: Convert the edit index page to React Convert Edit::Open test to use real data Convert Edit::Show test to use real data Remove no longer needed tt functions for edit display Remove unused import
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Incremental sitemaps ingest replication packets and follow the tables therein to find changes to pages in the sitemaps. If a type table is updated, like release_group_primary_type for example, we'd end up trying to fetch every page for every release group using a modified type. (It doesn't matter which columns were changed.) I can't think of a situation where changing a type would significantly affect the content of so many pages that we'd want search engines to re-index them all, so I'm having the code skip these type tables entirely.