Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

systemschema: use ExcludeDataFromBackup on some tables #119947

Merged
merged 1 commit into from
Mar 7, 2024

Conversation

rafiss
Copy link
Collaborator

@rafiss rafiss commented Mar 5, 2024

Certain tables are updated frequently and contain data that should never be backed up or restored. Allowing backups on them is not desired, since that results in protected timestamps being applied on the table, which prevents GC of old data.

This commit excludes the lease, replication_constraint_stats, replication_stats, and tenant_usage tables from backups. These are the tables that explicitly have their gc.ttl configured to a lower value.

informs: #80866
Release note: None

Copy link

blathers-crl bot commented Mar 5, 2024

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@rafiss rafiss force-pushed the system-table-exclude-from-backup branch 4 times, most recently from b9d27cc to de39d70 Compare March 5, 2024 23:19
@rafiss rafiss marked this pull request as ready for review March 5, 2024 23:20
@rafiss rafiss requested a review from a team as a code owner March 5, 2024 23:20
@rafiss rafiss requested a review from fqazi March 5, 2024 23:20
@Arivijay
Copy link

Arivijay commented Mar 6, 2024

Hey @rafiss,
Any chance of this getting backported it to 23.2?

@rafiss
Copy link
Collaborator Author

rafiss commented Mar 7, 2024

It cannot be backported, since modifying system tables in this way is not possible in patch releases.

Copy link
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just one minor comment

Reviewed 23 of 23 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rafiss)


pkg/upgrade/upgrades/v24_1_system_database.go line 111 at r1 (raw file):

		if err := deps.DB.DescsTxn(ctx, func(ctx context.Context, txn descs.Txn) error {
			for _, tabName := range []catconstants.SystemTableName{
				catconstants.ReplicationConstraintStatsTableName,

Do we want an upgrade test to confirm this?

Certain tables are updated frequently and contain data that should never
be backed up or restored. Allowing backups on them is not desired, since
that results in protected timestamps being applied on the table, which
prevents GC of old data.

This commit excludes the lease, replication_constraint_stats,
replication_stats, and tenant_usage tables from backups. These are the
tables that explicitly have their gc.ttl configured to a lower value.

Release note: None
@rafiss rafiss force-pushed the system-table-exclude-from-backup branch from de39d70 to 3d87bec Compare March 7, 2024 17:56
Copy link
Collaborator Author

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @fqazi)


pkg/upgrade/upgrades/v24_1_system_database.go line 111 at r1 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

Do we want an upgrade test to confirm this?

i added a logic test

@rafiss rafiss requested a review from fqazi March 7, 2024 17:56
Copy link
Collaborator

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: great work!

Reviewed 2 of 4 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

@rafiss
Copy link
Collaborator Author

rafiss commented Mar 7, 2024

tftr!

bors r+

@craig
Copy link
Contributor

craig bot commented Mar 7, 2024

Build succeeded:

@craig craig bot merged commit e214c34 into cockroachdb:master Mar 7, 2024
16 of 18 checks passed
@rafiss rafiss deleted the system-table-exclude-from-backup branch March 8, 2024 06:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants