ADR-037 Phase 3d: docker-compose e2e encrypted lifecycle + required CI gate

[cuttlefisch]

Part of #131. Depends on 3a-3c. Closes the confidence loop: validate the whole encrypted CRDT sharing lifecycle in remote CI.

Scope

• Key-mode compose: clients get Ed25519 identities; daemon seeded with their pubkeys (encryption requires key auth).
• Encrypted scenario .scm: share → (kb-set-encryption e2e) → join → approve → ≥2 members edit → wait-for-content converge (content-barrier polling, no fixed sleeps).
• Security-negative oracle: mount the daemon's data dir read-only; grep its store/WAL and logs for a per-run plaintext canary ⇒ ABSENT. Positive control: the same canary PRESENT in each member's buffer (silent content-drop fails loudly). Assert the kbc: op-log carries the Encryption::E2e flag + a wrapped_key op.
• Replace the stale Makefile ci-docker-e2e SKIP stub; make collab / docker e2e a required check.

Acceptance tests

• The encrypted scenario converges across ≥2 members; daemon store/WAL/logs contain only ciphertext (canary absent), members' buffers contain it (present).
• A deliberately-injected regression (push plaintext instead of sealing) turns the gate RED.
• Existing unencrypted convergence scenarios still pass (regression).

[cuttlefisch]

3d build plan (harness reverse-engineered on feat/e2e-docker-gate)

Branch feat/e2e-docker-gate is cut off 3c (#164) and carries the full 3b+3c+fence stack. The existing
collab e2e harness is now fully mapped — recording the concrete build so the next push executes cleanly.

Existing harness shape (to mirror, not disturb):

• docker-compose.collab-test.yml — daemon + mae --test <scenario>.scm clients writing to named volumes +
  an alpine verifier running tests/collab-e2e/verify.sh; verdict = verifier exit code.
• CI job collab / docker e2e runs make docker-collab-test (15-min timeout). Currently keyless / no auth.
• Two-editor choreography template: scripts/collab-membership-e2e.sh (alice shares → bob joins=pending →
  alice kb-approve <fp> editor → bob joins=member), coordinated by file barriers in a shared /sync dir
  (write-file / wait-for-file in tests/collab-e2e/lib/test-helpers.scm).
• Key-mode template: scripts/collab-mtls-e2e.sh — daemon [collab.auth] mode=\"key\"; editor identity via
  mae --collab-identity, authorized via mae-daemon authorize <pubkey>; editor init.scm sets
  collab-auth-mode=key + collab-host-key-policy=accept-new.
• Enable primitive confirmed: (kb-set-encryption \"<kb>\" \"e2e\") (scheme runtime, owner-only, one-way).

To build:

1. scripts/collab-encrypted-e2e.sh (single-host FIRST, locally validatable like the mtls script): keyed
   daemon + 2 keyed+authorized editors. alice: share → (kb-set-encryption … e2e) → approve bob → edit a
   KB node with a per-run CANARY → converge. bob: join → approved → join → read node → assert CANARY present.
2. Canary security oracle (the whole point): after editors exit, grep the daemon's data_dir store + WAL
   and daemon.log for the CANARY ⇒ ABSENT; grep alice/bob node buffers ⇒ PRESENT (positive
   control catches a silent content-drop). Assert the kbc: op-log carries Encryption::E2e + a wrapped_key
   op (via (kb-sharing-status) / eval-scheme).
3. Inject-regression test: a path that pushes plaintext instead of sealing must turn the oracle RED
   (proves the gate has teeth).
4. Dockerize: docker-compose.e2e-encrypted.yml (key-mode topology, daemon data dir mounted RO into the
   verifier for the store/WAL grep) + a verify-encrypted.sh + make docker-e2e-encrypted + a CI job.
5. Replace the SKIP stub at Makefile:283 (ci-docker-e2e). Do NOT make the check required yet —
   per decision, the impl-pass security review runs first, then the required-check flip is a follow-up.

OPEN QUESTION blocking step 1 (must validate, do not fabricate): the existing scenarios sync plain file
buffers (test_share.scm shares notes.txt). Encryption seals KB-node updates (KbNodeUpdate, gated on a
content_keys entry). Need to confirm how a scenario opens+edits a KB node so its edit emits a sealed
KbNodeUpdate and the joiner subscribes+decrypts it — trace kb-create / kb-edit-source
(crates/core/src/editor/dispatch/kb.rs) + node_to_kb registration in collab_bridge. This is the crux of a
correct encrypted scenario and must be exercised locally before the docker wrapping.

[cuttlefisch]

3d PAUSED — blocked on KB-node-edit-sync bugs (#165, #166)

Local validation of the encrypted-lifecycle oracle (single-host, scripts/collab-encrypted-e2e.sh on branch
feat/e2e-docker-gate) surfaced that a naive canary oracle is vacuous: the canary node edit never reaches
the daemon store, so "absent ⇒ sealed" doesn't hold. The harness DESIGN is sound — a negative control
(MAE_E2E_NEGATIVE=1, no encryption ⇒ canary MUST leak) correctly flagged the vacuousness. The content path is
the problem:

• KB collab: editing a node created in a named instance after share doesn't sync (owner=None, gate_hit=false) #165 — a node created in a named instance after share doesn't sync (owner=None, gate doesn't fire).
• KB collab: owner editing an IMPORTED+shared node is rejected by the epoch fence (imported client_id lineage) #166 — editing an imported+shared node is rejected by the epoch fence (imported client 1 lineage).

The op-set seal is proven (build_kb_node_update_request_seals_on_encrypted_kb). Resume 3d once a content
path that actually syncs + is fence-clean exists (fix #165/#166). The :kb-update scriptable content primitive

• the harness skeleton (marked WIP/BLOCKED, not wired to CI) are committed on feat/e2e-docker-gate.

Per decision: pausing 3d and running the impl-pass adversarial security review over the merged 3b/3c crypto next.

[cuttlefisch]

3d UNBLOCKED + the security oracle is PROVEN — joiner-converge assertion is the last piece

With #166/#177 merged, the encrypted-lifecycle harness (scripts/collab-encrypted-e2e.sh, branch
wip/3d-encrypted-harness) now validates the security property end-to-end and non-vacuously:

• daemon store/WAL/logs carry only ciphertext (canary ABSENT on e2e);
• the owner holds the plaintext (authored);
• MAE_E2E_NEGATIVE=1 (no encryption) → canary LEAKS into the daemon store (inject-regression control — teeth);
• the daemon APPLIED the sealed edit (non-vacuous).

Open — the joiner "decrypted it" assertion. Joined KBs are in-memory instances on the joiner, so the
decrypted content isn't on disk, and :help <id> does not render a joined-instance node (the *Help* buffer
comes back empty). Two things to resolve next:

1. the right read primitive for an in-memory instance node in a scenario (candidates: kb-search, a node
   buffer, introspect) — to read bob's materialized content;
2. confirm the joiner actually RECEIVES the post-join node content — an isolation run hung on barriers,
   which hints at a possible joiner content-sync gap (bob may not get alice's node edits on/after join). This
   needs a focused trace (daemon broadcast → bob subscription → apply).

Neither is a security/encryption gap — the daemon-blind ciphertext property + the seal are proven. Resume the
harness from the joiner read/sync, then dockerize + wire the (initially non-required) CI gate.