v1.4.0 — zstd-with-shared-dict wire compression (34.8% reduction, p99 0.68ms)

[1.4.0] — 2026-05-16

Added — zstd wire compression with shared dictionary

• New optional comp field on v1.2 sealed-body envelopes. When set to
  "zstd-dict-v1", the body plaintext was zstd-compressed (level 3) using the
  shipped shared dictionary BEFORE AES-256-GCM encryption. Decoder reverses
  the order: verify sig → unwrap body key → AES open → zstd decompress with
  dict.
• lib/compression.js — compress/decompress helpers using Node 22+
  built-in zlib zstd (no native or wasm deps). Loads
  lib/dict/piperchat-zstd-v1.dict once on first use, caches.
• lib/dict/piperchat-zstd-v1.dict — 64KB shared zstd dictionary,
  trained offline on 211 representative piperchat-shaped samples (11
  real v1.2 messages + 200 DOTpost observations). Committed as binary.
• tools/dict-trainer/{split-corpus.py,train.sh} — re-train the dict
  from a fresh corpus when ready (e.g. after collecting more traffic).
• comp field is signed in ed25519 canonical bytes when present
  (v12CanonicalBytes extended). v1.2/v1.3 envelopes without comp
  produce byte-identical canonical bytes as before — no breaking change.
• Server stores comp in a new messages.comp SQLite column and
  echoes it back in toPublic / SSE / GET responses.
• isV12Envelope now accepts version ∈ {1.2, 1.3, 1.4} — they share
  the encrypted-envelope wire shape.

Performance (measured on the included test suite)

Message size class	Sample count	v1.4 reduction vs v1.2
Short (<30B raw)	3	-35.5% (bloat — zstd frame overhead)
Meaningful (≥100B raw)	5	34.8%
Mixed corpus (60 samples)	—	median 1.62× ratio

Round-trip latency (compress + decompress) p99 = 0.68 ms on Apple Silicon.

Backwards compatibility

Fully backwards-compatible. No flag day. v1.0–v1.3 clients keep working unchanged.

Sender	Receiver	Result
v1.0–1.3	any	unchanged — no compression negotiated
v1.4	v1.0–1.3	sender omits comp; receiver handles as v1.2/v1.3
v1.4	v1.4	sender sets comp = "zstd-dict-v1"; both peers compress

Sender chooses compression based on recipient's client_caps advertised in
their identity DOT, or falls back to plaintext if capability is unknown.

Threat model

• Confidentiality: unchanged (AES-256-GCM, per-message random nonce)
• Authenticity: unchanged + comp is signed (cannot strip / downgrade)
• Plaintext-length leak: pre-existing in v1.2 (cipher_body size ≈ plaintext +
  28B); v1.4 actually reduces length variance. Padded-bucket framing
  scheduled for v1.5.
• No CRIME-style adaptive attack: bodies aren't shared across trust
  boundaries; dict is static; attacker can't inject chosen plaintext.

Tests

• test/compression.test.js — 9 unit tests (round-trip, latency, dispatch,
  cross-version isolation, dict cache)
• tests/v1.4-roundtrip.js — 6 end-to-end tests (sign+encrypt+decrypt+
  decompress, v1.2 backwards-compat, comp tampering rejection, wire-size
  reduction, v1.4-sender-to-v1.2-receiver degraded path)
• HTTP smoke: POST a v1.4 envelope to a live server, GET it back, verify
  version=1.4 and comp=zstd-dict-v1 round-trip through the SQLite
  storage layer.
• Existing tests/v1.2-roundtrip.js continues to pass (19/19) — no regression.

Engine bump

engines.node now requires Node 22+ (was 20+). Required for built-in
zlib.zstdCompressSync / zstdDecompressSync with dictionary option.

Files

• New: lib/compression.js, lib/dict/piperchat-zstd-v1.dict,
  tools/dict-trainer/{split-corpus.py,train.sh},
  test/compression.test.js, tests/v1.4-roundtrip.js
• Modified: lib/crypto.js (v12CanonicalBytes + isV12Envelope extended),
  lib/db.js (comp column), client.js (sendEncrypted +
  sendViaMailbox + readEnvelope), server.js (envForVerify includes
  comp + insert + toPublic), docs/PROTOCOL.md (v1.4 section),
  package.json (1.3.0 → 1.4.0, engines 20 → 22)

---