backupccl: skip backing up excluded spans #108627
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR? 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
|
This change is |
|
thinking about how to test this (inspect traces vs installing a kvrequest filter) but putting the diff up in the meantime |
dt
force-pushed
the
skip-excluded-spans
branch
from
91d1a08
to
b4d4e96
Compare
|
test added |
adityamaru
reviewed
Aug 11, 2023
| @@ -9494,6 +9508,11 @@ func TestExcludeDataFromBackupAndRestore(t *testing.T) { | |||
| restoreDB.Exec(t, `RESTORE DATABASE data FROM LATEST IN $1`, localFoo) | |||
| require.Len(t, restoreDB.QueryStr(t, `SELECT * FROM data.foo`), 0) | |||
| require.Len(t, restoreDB.QueryStr(t, `SELECT * FROM data.bar`), 10) | |||
|
|
|||
| before := atomic.LoadInt64(&exportReqsAtomic) | |||
There was a problem hiding this comment.
so at this point data.foo is going to be empty as asserted above. Can we add a few rows to it before attempt a backup.
Also s/BACKUP TO/BACKUP INTO/g
Previously we sent export requests to all spans being backed up. The ranges for spans of tables that had set the flag to exclude data from backup would reply with an empty response, excluding their data, but the backup process still sent these ranges these requests. This changes the backup process to not send requests for spans from excluded tables, when performing database, table, or cluster backups. Backups of tenants will still send export requests to every range for the tenant span, and those ranges that host tables that are excluded will continue to reply with no data. This is done both as an optimization, and so that backups can succeed even when a table is unavailable, if, and only if, that table is excluded. Release note (ops change): BACKUP now skips contacting the ranges for tables on which exclude_data_from_backup is set, and can thus succeed even if an excluded table is unavailable. Epic: none.
dt
force-pushed
the
skip-excluded-spans
branch
from
b4d4e96
to
c5219e3
Compare
adityamaru
approved these changes
Aug 11, 2023
msbutler
requested changes
Aug 11, 2023
msbutler
approved these changes
Aug 11, 2023
|
TFTRs! bors r+ |
|
Build failed (retrying...): |
|
Build succeeded: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously we sent export requests to all spans being backed up. The ranges for spans of tables that had set the flag to exclude data from backup would reply with an empty response, excluding their data, but the backup process still sent these ranges these requests.
This changes the backup process to not send requests for spans from excluded tables, when performing database, table, or cluster backups. Backups of tenants will still send export requests to every range for the tenant span, and those ranges that host tables that are excluded will continue to reply with no data.
This is done both as an optimization, and so that backups can succeed even when a table is unavailable, if, and only if, that table is excluded.
Release note (ops change): BACKUP now skips contacting the ranges for tables on which exclude_data_from_backup is set, and can thus succeed even if an excluded table is unavailable.
Epic: none.