Skip to content

v0.7.0 — Encryption at rest (SSE-S3)

Choose a tag to compare

@incognick incognick released this 16 Jun 19:43
· 45 commits to main since this release
Immutable release. Only release title and notes can be modified.

The seventh Hamster release: encryption at rest — server-side encryption (SSE-S3) over the erasure-coded cluster, envelope-encrypted so storage and repair never touch your keys. Object data is encrypted before it is sharded; the per-object key is wrapped by a cluster master key that lives only in memory and never reaches disk or the network.

Dev preview, and read the limits below. Encryption at rest works and is verified end to end, but the v0.x limits hold: it is a cluster feature (the single-node serve preview does not encrypt), writes commit only on the Raft leader (non-leaders answer 503 SlowDown), multipart and server-side copy are still not on the cluster path, and on-disk/on-wire formats may change between v0 releases. Hamster is not assessed or certified for any regulation (see "What this is not"). Please don't trust real regulated data to it yet.

What's in v0.7

  • Envelope encryption, encrypt-then-EC. Every object gets a fresh random data key (AES-256-GCM); the object is encrypted before it is erasure-coded, so the shards on every node are ciphertext. Repair, scrub, and rebalance verify and rebuild those shards while knowing nothing — they never need a key. The data key is then wrapped by the cluster's master key and stored, wrapped, alongside the object's metadata.
  • A cluster master key (KEK) you control, never persisted. Each node loads the key at startup from cluster run -master-key-file <path> — held in memory only, never written to disk by Hamster, never sent over the wire (not even when a node joins). Mount it off the data disk (a Kubernetes Secret volume is the intended shape) so a stolen data disk yields only ciphertext. The file holds 32 bytes, raw or hex/base64.
  • Turn it on, cluster-wide, when you're ready. cluster encrypt enables encryption for new writes (leader-only, refused if the leader holds no key). It is enable-only — once a cluster encrypts it cannot be switched back to plaintext in place. Existing objects written before you enabled it stay readable and unencrypted: a cluster holds a mix, and each object's own record says which it is.
  • The SSE-S3 surface. When the cluster encrypts, x-amz-server-side-encryption: AES256 is reported back on PUT, GET, and HEAD. A request that asks for AES256 when the server cannot honor it is refused (501) rather than silently storing plaintext; SSE-KMS and SSE-C are refused honestly.
  • cluster status shows the posture, so you can always see whether a cluster encrypts.

How it's verified

  • Under the deterministic simulation harness (the durability bar every data-path change must clear): the encrypted write path under crash schedules, a coordinator crash mid-encrypted-PUT that commits nothing, repair healing an encrypted object's lost shards from surviving ciphertext without any key, encrypted-write determinism (same seed → same key and bytes), and a read refused loudly when the key is unavailable — never plaintext served as ciphertext or the reverse.
  • The chunk transform is golden-pinned, with every single-byte tamper and the wrong key rejected by the GCM tag alongside the existing CRC.
  • A real-process end-to-end test founds a three-node cluster over loopback mTLS, each node given the key with -master-key-file, turns on encryption, writes through the S3 API, reads back with the SSE header over real HTTP, and proves a node that restarts reloads its key and still decrypts.
  • The aws CLI, rclone, restic, s3cmd, the race detector, and the simulation harness all keep passing.

What this is not

Be precise about the boundary; encryption is a mechanism, not a certification.

  • It is a cluster feature. The single-node serve preview does not encrypt and refuses SSE requests rather than pretend. Encryption at rest is where the compliance story lives, and that is the cluster.
  • It is enable-only, with no in-place disable. To stop encrypting, migrate the data to a fresh cluster (the same generic S3 copy that migrates unencrypted data into an encrypted cluster). There is deliberately no "turn it off" — quietly weakening the at-rest posture is the wrong direction for the workloads Hamster targets.
  • Key custody and rotation are the operator's responsibility, and rotation is next. Hamster never persists or transmits the master key, but where that key lives is yours to manage (mount it off the data disk). Master-key rewrap and CA rotation are v0.8, not this release.
  • SSE-KMS and SSE-C are not supported. Hamster manages keys cluster-wide (SSE-S3); the per-object-key machinery leaves room for SSE-C later.
  • Not assessed or certified. Hamster encrypts object data at rest, but it has not been assessed for any regulation by anyone, and it is v0, not production-ready, with formats that may still change. Encrypting at rest is necessary for many regimes but nowhere near sufficient on its own.

Binaries below are static (CGO_ENABLED=0), version-stamped (hamster version), with SHA-256 checksums in SHA256SUMS. Next up, v0.8: key and CA rotation — rewrapping every object's key under a new master key without touching object bytes, and the cluster CA's custody and rotation as the keys problem they are.