Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

raft topology: add garbage collection for internal CDC generations table #15323

Closed
patjed41 opened this issue Sep 8, 2023 · 2 comments · Fixed by #15413
Closed

raft topology: add garbage collection for internal CDC generations table #15323

patjed41 opened this issue Sep 8, 2023 · 2 comments · Fixed by #15413

Comments

@patjed41
Copy link
Contributor

patjed41 commented Sep 8, 2023

As discussed in #13962 (comment), we need a mechanism to clear obsolete CDC generation data because we send the entire contents of CDC_GENERATIONS_V3 as a part of the group 0 snapshot. We need to prevent these snapshots from endlessly growing as we introduce new generations over time.

The first step of implementing such a mechanism is making the CDC_GENERATIONS_V3 single-partition and ordered by timeuuids. This change enables the efficient clearing of old generations by inserting a range tombstone. #15163 has already introduced this change.

The second (and the last) step is actually clearing old generations. The proposed solution is to make the topology coordinator systematically insert a range tombstone covering all generations that are:

  • published,
  • not current (its timeuuid does not equal current_cdc_generation_uuid),
  • older than now() - 24 h, where now() is the current time point of the coordinator's clock.

The explanation for the two first requirements is that we cannot delete unpublished and current generations. The last requirement addresses the clock discrepancies with a large reserve. We must ensure all nodes' clocks are in the future compared to a generation we want to remove.

Tombstones can be inserted by the topology coordinator, but it seems like its CDC publisher fiber introduced in #15281 is a better candidate for this task. It can be adjusted to do it.

@patjed41
Copy link
Contributor Author

patjed41 commented Sep 8, 2023

cc @kbr-scylla

@avikivity
Copy link
Member

raft topology is still experimental, not backporting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants