New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-23.2: restore: limit restore spans to 200 files #119883
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously we would limit restore spans to 384MB, but if a backup contained many small files, a large number of these files could be grouped into such a span before it hit the 384MB target. Unfortunately this meant opening a all of those files to process the restore span, leading to very large numbers of concurrent connections when restoring a backup consisting of tiny files. This patch limits the restore spans to 384MB or 200 files, whichever is hit first. Restores of backups with large numbers of tiny files may thus be slightly slower but will better limit their concurrent outbound connections. This breaks the test cases for experimental/disabled support for memory-monitored restores. Given we have no plans to actually enable this at this time, for the sake of ease of backporting this bugfix, these tests are simply skipped rather than introducing churn to them. #119836 tracks removing them and that functionality entirely instead, but again for the same of backporting is not done here. Release note (bug fix): fix a bug where RESTORE on certain backups would open a very large number of connections to the backup storage provider. Epic: none.
blathers-crl
bot
added
the
blathers-backport
This is a backport that Blathers created automatically.
label
Mar 4, 2024
blathers-crl
bot
force-pushed
the
blathers/backport-release-23.2-119840
branch
from
March 4, 2024 20:12
8a8474c
to
5846bf2
Compare
blathers-crl
bot
force-pushed
the
blathers/backport-release-23.2-119840
branch
from
March 4, 2024 20:12
be8fc3b
to
343e3e8
Compare
Thanks for opening a backport. Please check the backport criteria before merging:
If your backport adds new functionality, please ensure that the following additional criteria are satisfied:
Also, please add a brief release justification to the body of your PR to justify this |
blathers-crl
bot
added
the
backport
Label PR's that are backports to older release branches
label
Mar 4, 2024
jbowens
approved these changes
Mar 5, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
stevendanna
approved these changes
Mar 5, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
backport
Label PR's that are backports to older release branches
blathers-backport
This is a backport that Blathers created automatically.
O-robot
Originated from a bot.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #119840 on behalf of @dt.
/cc @cockroachdb/release
Previously we would limit restore spans to 384MB, but if a backup contained many small files, a large number of these files could be grouped into such a span before it hit the 384MB target. Unfortunately this caused the restore process to open all of those files to process the restore span, leading to very large numbers of concurrent connections when restoring a backup consisting of tiny files.
This patch limits the restore spans to 384MB or 200 files, whichever is hit first. Restores of backups with large numbers of tiny files may thus be slightly slower but will better limit their concurrent outbound connections.
Fixes #119785.
NB: This breaks the test cases for the experimental/disabled support for memory-monitored restores. Given we have no plans to actually enable these at this time, but do need to fix this bug in the normal restore path and backport such a fix, to reduce churn in said backports the tests of the disabled mode are simply skipped in this diff rather modified, in anticipation of removing them completely from the development branch, tracked in #119836.
Release note (bug fix): fix a bug where RESTORE on certain backups would open a very large number of concurrent connections to the backup storage provider.
Epic: none.
Release justification: bug fix.