Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Doc encryption with symmetric key #2731

Open
wants to merge 35 commits into
base: develop
Choose a base branch
from

Conversation

islamaliev
Copy link
Contributor

Relevant issue(s)

Resolves #2711

Description

This change introduces doc encryption. Upon creation of a document the user can pass an AES-GCM key for encryption.
This will lead to all doc fields (deltas) being stored in the DAG encrypted.
The storage in the local datastore happens as plain text (devs can enable encryption-at-rest to have it encrypted as well).

@islamaliev islamaliev self-assigned this Jun 17, 2024
@islamaliev islamaliev added security Related to security area/datastore Related to the datastore / storage engine system area/collections Related to the collections system labels Jun 17, 2024
@islamaliev islamaliev added this to the DefraDB v0.12 milestone Jun 17, 2024
cli/collection_create.go Outdated Show resolved Hide resolved
)

type multistore struct {
root DSReaderWriter
data DSReaderWriter
enc DSReaderWriter
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: What do you think of naming it crypto instead? It's short enough and easier to figure out what it means.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it sounds better, but less relevant and 3 bytes longer.
I not happy myself with "enc" as it might not be immediatly clear, but tend to like it slightly more than "crypto". If other also see "crypto" being a better much (or anything else), no problem.

internal/db/db.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think you're going in the right direction. I have 2 main critiques at this stage. The first, as mentioned in an other comment, is the passing of a key on document create. I think we should avoid doing this and let Defra create a new key when needed. The second is the use of the context for the encryption key and store. It feels like it's making it more complex than it needs to be. We can discuss it some more in the standup or in a separate call if you want.

@islamaliev islamaliev force-pushed the feat/simple-doc-encryption branch 2 times, most recently from 49fa643 to 045d85a Compare June 24, 2024 10:27
Copy link

codecov bot commented Jun 24, 2024

Codecov Report

Attention: Patch coverage is 80.10610% with 75 lines in your changes missing coverage. Please review.

Project coverage is 78.78%. Comparing base (8437805) to head (415b025).
Report is 8 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2731      +/-   ##
===========================================
- Coverage    78.85%   78.78%   -0.06%     
===========================================
  Files          315      318       +3     
  Lines        23835    24097     +262     
===========================================
+ Hits         18793    18984     +191     
- Misses        3670     3715      +45     
- Partials      1372     1398      +26     
Flag Coverage Δ
all-tests 78.78% <80.11%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
cli/collection_create.go 66.13% <100.00%> (+8.13%) ⬆️
client/document.go 71.92% <100.00%> (+0.12%) ⬆️
client/request/mutation.go 100.00% <ø> (ø)
datastore/multi.go 100.00% <100.00%> (ø)
http/client.go 56.33% <100.00%> (+0.54%) ⬆️
http/client_collection.go 43.46% <100.00%> (+1.63%) ⬆️
internal/core/block/block.go 90.65% <100.00%> (-0.57%) ⬇️
internal/db/collection.go 71.60% <100.00%> (ø)
internal/db/context.go 100.00% <100.00%> (ø)
internal/db/fetcher/fetcher.go 79.18% <100.00%> (ø)
... and 24 more

... and 13 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8437805...415b025. Read the comment docs.

@islamaliev islamaliev marked this pull request as ready for review June 25, 2024 13:34
@islamaliev islamaliev force-pushed the feat/simple-doc-encryption branch 2 times, most recently from d7d8ec5 to 316f55c Compare June 26, 2024 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/collections Related to the collections system area/datastore Related to the datastore / storage engine system security Related to security
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Doc. Encryption: Enable doc encryption with symmetric keys
2 participants