Skip to content

[DO NOT MERGE] End-to-end-encrypted session history#268

Merged
posix4e merged 1 commit into
mainfrom
feature/e2e-history
May 30, 2026
Merged

[DO NOT MERGE] End-to-end-encrypted session history#268
posix4e merged 1 commit into
mainfrom
feature/e2e-history

Conversation

@posix4e
Copy link
Copy Markdown
Member

@posix4e posix4e commented May 29, 2026

⚠️ Draft — do not merge yet

replay now returns ciphertext records instead of plaintext. Any client expecting the old bytes_b64 shape (including the current web shell) breaks. Hold until the client decryptor (dd-client history open_record, Phase 4) and the web shell are updated.

What

dd-sessiond encrypted transcripts with a server-held key, so the enclave could read all scrollback — encrypted at rest, not end-to-end. This seals each record to the paired device X25519 pubkeys: random content key per record, wrapped to each device via ephemeral X25519 + HKDF-SHA256 + ChaCha20Poly1305. The enclave appends but holds no device private key, so it cannot read history back; the device decrypts client-side.

  • sessiond: seals to a live recipient set (no server-key fallback — drops history + flags meta when no device is paired); ReplayResponse is now {id, version, records}.
  • agent: pushes the non-revoked device set to sessiond at startup + on every enroll/revoke; devices::Store::live_pubkeys().
  • adds hkdf.

🤖 Generated with Claude Code

@github-actions
Copy link
Copy Markdown

DD preview ready

URL: https://pr-268.devopsdefender.com

Browser login: visit https://pr-268.devopsdefender.com — DD redirects you to
the GitHub App auth broker. A DD session cookie scoped
to .devopsdefender.com lets the preview, fleet, and
shell hosts share the same login.

Machine-to-machine: GitHub Actions workflows in the
DD_OWNER org pass their per-job OIDC JWT as
Authorization: Bearer … (audience dd-agent).

Register endpoint for a local agent: https://pr-268.devopsdefender.com/register
(authenticated by ITA attestation).

posix4e added a commit to devopsdefender/dd-client that referenced this pull request May 29, 2026
dd-sessiond seals each transcript record to paired device pubkeys; replay
returns ciphertext the enclave can't read. Add dd-client-session::history with
open_record (recover the content key from our recipient stanza via X25519 +
HKDF-SHA256, decrypt the record) and decrypt_replay (reconstruct the terminal
byte stream, also handling the legacy plaintext bytes_b64 shape during rollout).
Wire `dd-client replay` to decrypt with the device key and write the transcript
to stdout.

Tests seal exactly as dd/src/sessiond.rs::seal_record does, so they verify
cross-repo wire compatibility (recipient opens, non-recipient gets None,
multi-recipient, pty-stream reconstruction, legacy fallback).

Pairs with devopsdefender/dd#268.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@posix4e
Copy link
Copy Markdown
Member Author

posix4e commented May 29, 2026

Update: the native client decryptor has landed in devopsdefender/dd-client#4 (dd-client-session::history::open_record / decrypt_replay, verified wire-compatible with sessiond::seal_record). So dd-client replay can read this format.

Remaining blocker before merge: the in-browser dd-shell web UI still expects the plaintext bytes_b64 replay shape and has no device key, so it can't decrypt E2E history. Merging this deploys ciphertext replay and breaks that view. Decision needed: drop/replace the server-rendered web replay, or keep it on a non-E2E path. Holding as draft until that's resolved.

@posix4e
Copy link
Copy Markdown
Member Author

posix4e commented May 29, 2026

Blocker resolved — this is safe to merge.

The earlier concern (browser dd-shell expecting plaintext replay) was based on a stale branch. On main, src/shell.rs is a static notice page ("Use dd-client for interactive sessions; this endpoint no longer exposes a browser terminal or session control API") — it does not call replay. The only consumer of /api/sessions/{id}/replay is the Noise gateway → sessiond → the native client, which now decrypts the sealed records (devopsdefender/dd-client#4, Phase 4).

So no web surface breaks. Remaining caveat is only forward-compat: a pre-#4 native client hitting a post-merge agent would get ciphertext it can't parse — but there's no released native client yet, and decrypt_replay already accepts the legacy bytes_b64 shape for the reverse case. Marking ready for review.

@posix4e posix4e marked this pull request as ready for review May 29, 2026 20:53
posix4e added a commit to devopsdefender/dd-client that referenced this pull request May 30, 2026
…testation (#4)

* Redesign client as a structured-chat-document renderer; verify-only attestation

Reframes the native client from a raw byte pump into a renderer for a structured
chat document (markdown blocks + menus), with the raw terminal as an escape
hatch. Chat-style coding TUIs (Claude Code, Codex, …) map naturally onto
markdown + menus; the client derives that model and renders it, flippable
between read-only Watch, live Interact, and full Raw passthrough.

New crate dd-client-session (the engine, shared by every frontend):
- block model + streaming BlockEvent log with delta replay
- universal "floor" deriver: incremental ANSI strip → Markdown/Code/Diff blocks
- vt100 screen + menu detection (reverse-video / numbered / yes-no heuristics)
- ViewMode FSM (Watch=Clean / Interact,Raw=Controlled) + Ctrl-] chord parser +
  keystroke synthesis
- ClaudeCodeAdapter: lossless blocks from --output-format stream-json
- transport pump split out of core (NoiseConnection::split)

CLI: session_ui flips Watch ⇄ Interact ⇄ Raw over one live attachment (ratatui
structured view + real-tty Raw), --adapter floor|claude.

Attestation: the client now VERIFIES the agent's ITA token (served as
noise.ita_token) against Intel's public JWKS — no API key, no Intel account —
instead of minting its own. Pairs with the dd agent change. Falls back to a
clear error or --insecure-skip-quote-verify when the token is absent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add client-side E2E history decryptor (Phase 4)

dd-sessiond seals each transcript record to paired device pubkeys; replay
returns ciphertext the enclave can't read. Add dd-client-session::history with
open_record (recover the content key from our recipient stanza via X25519 +
HKDF-SHA256, decrypt the record) and decrypt_replay (reconstruct the terminal
byte stream, also handling the legacy plaintext bytes_b64 shape during rollout).
Wire `dd-client replay` to decrypt with the device key and write the transcript
to stdout.

Tests seal exactly as dd/src/sessiond.rs::seal_record does, so they verify
cross-repo wire compatibility (recipient opens, non-recipient gets None,
multi-recipient, pty-stream reconstruction, legacy fallback).

Pairs with devopsdefender/dd#268.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* UniFFI bindings + SwiftUI companion (Phase 5)

Replace the hand-rolled C FFI with UniFFI: one Rust surface generates Swift (and
Kotlin) bindings. Exposes keygen, a SessionHandle object (attach over Noise,
blocks() snapshot, subscribe(BlockObserver), send_text, set_mode, close) and the
block/mode DTOs. All interpretation stays in dd-client-session; the foreign side
only renders blocks and reacts to a change callback. Everything is sync across
the FFI — a shared multi-thread tokio runtime does the async work — so Swift
never bridges Rust futures. Adds a uniffi-bindgen bin for generation.

iOS app (apps/ios): SessionModel mirrors the block snapshot into SwiftUI on
change; ContentView renders the structured document — markdown as markdown,
menus as tappable buttons, code/diff monospaced — with a Watch/Interact/Raw
picker and an input bar. README documents the bindgen + xcframework steps.

The Rust FFI crate is compile/clippy/test-verified on Linux; binding generation
and the Xcode build require the Apple toolchain (not run here).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Client: pin agent measurement (MRTD/TCB) in verify path

Extend QuoteVerification::IntelTrustAuthority with expected_mrtds + expected_tcb.
After verifying the ITA token + report_data binding, verify_measurement checks
the MRTD is in the allowlist (and TCB matches) — so the client confirms not just
"a genuine TDX enclave" but "running the code we pinned". Unpinned = warn (don't
fail). CLI: --expected-mrtd (repeatable/comma-sep, DD_EXPECTED_MRTD) + --expected-tcb.
The pin must come from a source independent of the agent. Pairs with
devopsdefender/dd#<measurement-pinning>.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Surface server error detail on session create

The Noise gateway wraps upstream failures as {error, detail}; the client printed
only "shell_failed" and dropped the detail that says why (e.g. "unknown recipe:
codex"). Include the detail in the bailed error message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Render the vt100 grid for alt-screen apps in Watch/Interact

Full-screen TUIs (Codex, vim, …) paint a grid with absolute cursor moves; the
line-oriented floor mangles them (words run together, spinner litter). The engine
already keeps a faithful vt100 ScreenSnapshot with an .alternate flag — render
that grid (preserving column spacing, reverse-video) in the structured views when
the app is on the alternate screen, instead of the floor blocks. Plain scrolling
output still uses the structured block view. Title shows screen vs blocks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Read report_data from tdx_report_data claim

Intel TDX attestation tokens carry the quote's report_data in the tdx_report_data
claim; attester_held_data is only present when held-data is submitted at mint
time (which the agent does not do). The client's report_data binding check was
reading only attester_held_data, so keyless verification failed with "ITA token
missing report_data" — caught the first time the binding check ran against a real
token (prior runs used --insecure-skip). Fall back to tdx_report_data.

Verified end-to-end: keyless verify against the prod agent (no Intel account).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dd-sessiond previously encrypted persisted transcripts with a server-held key,
so the enclave could read all scrollback back — "encrypted at rest" but not
end-to-end. Re-key the transcript store so each record is sealed to the paired
device X25519 pubkeys (multi-recipient: random content key per record, wrapped
to each device via an ephemeral X25519 + HKDF-SHA256 + ChaCha20Poly1305). The
enclave can append but holds no device private key, so it cannot read history
back; replay returns the sealed records and the device decrypts client-side.

- sessiond: TranscriptStore seals to a live recipient set (skips persistence,
  flags meta, when no device is paired — never falls back to a server key);
  replay returns ciphertext records; ReplayResponse is now {id, version, records}.
- agent: pushes the non-revoked device pubkey set to sessiond at startup and on
  every enroll/revoke; devices::Store gains live_pubkeys().
- adds hkdf dependency.

BREAKING: replay now returns ciphertext. Do not deploy until the client
decryptor (dd-client history open_record) and the web shell are updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@posix4e posix4e force-pushed the feature/e2e-history branch from 28ed154 to a39ab2d Compare May 30, 2026 13:46
@posix4e posix4e merged commit 2fcab1f into main May 30, 2026
1 check passed
@posix4e posix4e deleted the feature/e2e-history branch May 30, 2026 13:46
posix4e added a commit that referenced this pull request May 30, 2026
Revert to May-10 working state (back out #267, #269, #268)
posix4e added a commit that referenced this pull request May 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant