Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

engineccl: use incremental, records-based format for data key rotations #70140

Open
jbowens opened this issue Sep 13, 2021 · 2 comments
Open
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-storage Storage Team
Projects

Comments

@jbowens
Copy link
Collaborator

jbowens commented Sep 13, 2021

The CockroachDB encryption-at-rest implementation uses a file called the 'data keys registry' for storing the encryption keys used to encrypt a store's files. When the active data key is rotated, a new file is added to a protocol buffer and the active file is replaced with a new one containing the updated protocol buffer. As a store accumulates data keys, the cost of rotation grows. Rotation requires a O(# data keys) write to disk. Currently, the encryption-at-rest implementation never removes data keys from the data keys registry, which exacerbates the impact of this rewrite (see #70138).

We should modify the data keys registry to use Pebble's record package to write incremental updates to the data keys registry during rotations. Since 21.2 this is the approach that the file registry uses to record which files are encrypted with which data keys.

Jira issue: CRDB-9958

Epic CRDB-16419

@jbowens jbowens added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-storage Relating to our storage engine (Pebble) on-disk storage. T-storage Storage Team labels Sep 13, 2021
@jbowens jbowens added this to Incoming in Storage via automation Sep 13, 2021
@jbowens
Copy link
Collaborator Author

jbowens commented Sep 13, 2021

NB: This is a similar issue as #55967, but not as impactful since data key registries are significantly less frequent than file creations, removals or renames.

@mwang1026 mwang1026 moved this from Incoming to To Do (investigations) in Storage May 2, 2022
@mwang1026 mwang1026 moved this from To Do (investigations) to Prioritized in Storage May 2, 2022
@nicktrav nicktrav moved this from Prioritized to Backlog in Storage Jan 23, 2023
Copy link

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
10 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-storage Storage Team
Projects
Storage
  
Backlog
Development

No branches or pull requests

1 participant