cherrypick-1.1: cli: simplify and improve dump performance #19400
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A significant percentage (near 50%) of CPU time during dump is spent converting strings returned from the SELECT into SQL-safe strings and writing that to the output. Much of this work can be done concurrently, which improves the performance, and gets it close to that of piping a `SELECT *` to a file. This is done using an errgroup and passing various stages over channels. We can also greatly simplify dump now since cockroach streams results. There's no need anymore to page through the data. See cockroachdb/docs#1674
Although this test is already passing, it is being added to master so we don't regress and so it can be backported to 1.1, where it is failing. See #18500
|
This change is |
|
Review status: 0 of 2 files reviewed at latest revision, all discussions resolved, all commit checks successful. Comments from Reviewable |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherrypick #18472 and #19325.
#18472 was initially not included in 1.1 because it was thought to only be a performance increase, and the release cycle was late enough that it wasn't worth the risk to include it. However #18500 showed that the 1.1 branch had a known bug, so now it may be important enough to include here.
We could have alternatively made a from-scratch change to just fix the bug in 1.1, but I think it's better to cherrypick a more thought out and tested solution instead.
Our other option is to not accept the risk of the code changes here, and document the bug as a known limitation. However I think the change is safe enough to include, and comes with a hefty performance improvement to boot.
Fixes #18500