Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storageccl: provide a mechanism to verify the age of all data-keys required to decrypt data in the store for EAR. #80535

Open
data-matt opened this issue Apr 26, 2022 · 2 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-storage Storage Team X-nostale Marks an issue/pr that should be ignored by the stale bot
Projects

Comments

@data-matt
Copy link

data-matt commented Apr 26, 2022

Is your feature request related to a problem? Please describe.
When rotating data-keys, the background is, we rely on "storage engine churn" to de-crypt and re-encypt data when there is upserts/compactions. However, it is hard to verify the current state of encrypted data.

Currently it is not possible to identify the age of all data-keys that exist for a given store. We would like to be able to see metadata about all keys still required to decrypt data.

Describe the solution you'd like
Update the endpoint https://localhost:8080/#/reports/stores/1 to provide metadata on all keys required to decrypt data.

Describe alternatives you've considered
Checking the timestamps of .SST files to establish the age of the oldest .SST file and then consider this to be the age of the "oldest" data-key + key rotation period.

Additional context
Made this as an example, hopefully there is better terminology for old data-keys:

image

Jira issue: CRDB-15358

Epic CRDB-16419

@data-matt data-matt added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Apr 26, 2022
@mwang1026 mwang1026 removed their assignment Apr 26, 2022
@mwang1026 mwang1026 added this to Incoming in Storage via automation Apr 26, 2022
@blathers-crl blathers-crl bot added the T-storage Storage Team label Apr 26, 2022
@mwang1026 mwang1026 moved this from Incoming to To Do (investigations) in Storage May 2, 2022
@data-matt
Copy link
Author

Nice to have, at @mwang1026 looks like we can put this on the deep backlog.

@jlinder jlinder added sync-me and removed sync-me labels May 20, 2022
@nicktrav nicktrav moved this from To Do (investigations) to Backlog in Storage Jun 6, 2022
@nicktrav nicktrav moved this from Backlog to Observability in Storage Mar 21, 2023
Copy link

github-actions bot commented Jan 3, 2024

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
10 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!

@data-matt data-matt added X-nostale Marks an issue/pr that should be ignored by the stale bot and removed no-issue-activity labels Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-storage Storage Team X-nostale Marks an issue/pr that should be ignored by the stale bot
Projects
Status: Observability
Storage
  
Observability
Development

No branches or pull requests

3 participants