Add optional chunk encryption to stores#371
Open
folbricht wants to merge 4 commits into
Open
Conversation
This was referenced Jul 8, 2026
Closed
Chunks can now be encrypted at rest on a per-store basis using XChaCha20-Poly1305 (default) or AES-256-GCM. Encryption is implemented as a converter layer, applied after compression when writing to a store and reversed when reading. Encrypted chunks carry a file extension with the algorithm name and a key ID (leading 4 bytes of SHA256(key)) so that chunks with different settings or keys can share a store. The 256-bit encryption key is provided hex-encoded, either in the store-options of the config file (encryption-key), via the DESYNC_ENCRYPTION_KEY environment variable, or with the chunk-server --encryption-key option. Requiring the raw key rather than deriving one from a password avoids weak, brute-forceable passwords entirely; a suitable key can be generated with `openssl rand -hex 32`. This revives and reworks the changes from #182 on top of the current codebase.
- Move the Compressor converter from compress.go into converter.go so the datadog build variant compiles again; compress.go is only built without the datadog tag. - Reject store options that set encryption-key or encryption-algorithm without enabling encryption instead of silently writing plaintext. - Require an explicit flag to enable encryption in chunk-server. The DESYNC_ENCRYPTION_KEY environment variable alone no longer switches the wire format, it is only consulted once encryption is enabled with --encryption, --encryption-key or --encryption-algorithm. Enabling encryption without a key is now an error rather than silently serving plaintext. - Fail when encryption is configured on casync protocol (ssh://) stores which don't support it, rather than silently ignoring it. - Reject chunk requests whose name carries an unexpected extension with a clear error again. The suffix check alone was a no-op for servers without compression and encryption, and such requests only failed deep in the chunk ID parser.
Encryption provides confidentiality while integrity relies on the regular chunk content validation, since the AEAD ciphertext is authenticated under the key but not bound to the chunk name. Spell that out in the encryption docs, including the consequence that skip-verify should only be used where a downstream reader still validates chunks.
- StorageConverters returns the named Converters type instead of []converter, giving callers access to its methods without a cast. - Derive the chunk file extension from the converters once at store construction rather than rebuilding the string on every chunk operation. All stores and the HTTP handler now follow the pattern the SFTP store already used, and SFTPStore.Prune uses the pooled base's extension instead of deriving it a second way. - Collapse the two identical AEAD converter implementations into one aeadConverter parameterized by algorithm, keyed comparison now includes the algorithm name. Adding an algorithm only requires a constructor and a case in StorageConverters.
folbricht
force-pushed
the
chunk-encryption
branch
from
20ad549 to
f4c890e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Revives #182, rebased onto the current codebase and reworked based on the discussion in that PR.
Chunks can be encrypted at rest on a per-store basis using XChaCha20-Poly1305 (default) or AES-256-GCM. Encryption is implemented as a converter layer on top of the framework from #181: it is applied after compression when writing to a store and reversed when reading. Encrypted chunks are stored with a file extension containing the algorithm and a key ID (the leading 4 bytes of SHA256 of the key), e.g.
<hash>.cacnk.xchacha20-poly1305-e1e2b9f0, so chunks with different settings or keys can coexist in the same store.pruneandverifyonly consider chunks matching the store's configured extension.The main change compared to #182: instead of deriving the key from a password with SHA256, the user provides the raw 256-bit key hex-encoded (e.g. generated with
openssl rand -hex 32). As discussed in the original PR, password-derived keys are only as strong as the password and can be brute-forced cheaply, especially since the key ID in the chunk file name acts as a fast verification oracle. Requiring the full-entropy key eliminates that class of problem, and makes the key ID in the extension harmless.Configuration:
encryption,encryption-key, andencryption-algorithminstore-optionsDESYNC_ENCRYPTION_KEYenvironment variable as fallback for the key, to keep it out of config fileschunk-server --encryption/--encryption-keyto serve/accept encrypted chunks, independent of how the upstream store is configured. The env variable supplies the key but never enables encryption on its own, and enabling encryption without a key is an errorAlso included from the rebase: the converter interface now determines chunk file extensions (
storageExtension()), replacing theCompressedChunkExt/UncompressedChunkExtconstants, and the now-unusedhasCompression()helper was removed.Note: encryption protects chunk contents only. Chunk file names remain content hashes of the plain data, so an observer who already knows a plaintext can confirm its presence in a store. Integrity comes from the regular chunk content validation rather than from the AEAD, which authenticates ciphertext under the key but does not bind it to the chunk name. Both properties are documented in the README, including the guidance to only use skip-verify where a downstream reader still validates.