release-22.2: backupccl: speed up listing in some backup callsites #93204
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #93072 on behalf of @adityamaru.
/cc @cockroachdb/release
There were a couple of calls to
List
that would list all the files in the backup destination bucket. In the presence of a large number of files, usually SST files in thedata/
folder, this listing could be very slow. This could manifest as a RESTORE command hanging during the planning phase when we go to resolve encryption information from the base backup bucket.To fix this, we add a
delimiter
to theList
call that will make skipping over all files in thedata/
directory much cheaper.Informs: https://github.com/cockroachlabs/support/issues/1937
Release note (bug fix): speedup slow listing calls that could manifest as restore queries hanging during execution, in the presence of several backup files
Release justification: low risk change with high impact for customers restoring from buckets with a large number of files