Sessions can be archived but never deleted -- no erasure path exists for transcripts

[nokkies]

Raised by a deployment operator who noticed the UI offers archive but no delete — and it turns out that is not a UI omission. As far as I can establish, no session deletion path exists anywhere in the harness.

Verified against current master

• workspace.archiveSession appends the id to archivedSessionIds in the workspace registry — durable list-visibility state, nothing more. The transcript is untouched.
• No API method, CLI verb, or persistence operation ever removes a session log. The JSONL layer's only unlinks are atomic-publish mechanics (link()+unlink() and staging cleanup in the win32 path). Every transcript ever created — prompts, file contents, tool outputs, terminal echoes — accumulates under the harness home indefinitely.
• Deletion appears to have been anticipated: session-projection-cache/src/spec.ts explicitly reasons about "a deleted-then-recreated id" naming a slot rather than a lifecycle. The hazard analysis exists; the feature does not.

Why it is worth a design rather than staying a gap

• Retention/erasure: a deployment serving real users has no way to honor "remove this conversation." Archive hides it from the archiver's own list and nothing else.
• Secrets: anything echoed through a tool — tokens in command output, file contents — lives in plaintext JSONL for the life of the machine.
• Growth: unbounded, and on managed fleets disk is metered.

Why it is design-shaped, not a quick verb

A session id is not an island:

1. It may be the parentSession in other sessions' headers (subagent lineage) — deleting a parent orphans children's provenance. (liveLineage already tolerates an unresolvable parent, which helps.)
2. A live agent may hold it; deletion must refuse or coordinate with disposal — the descendant-drain machinery from subagent: a continuable child of a non-continuation parent is owned by nothing, and outlives the parent unheard #4239 is adjacent and its authorization filter looks reusable.
3. session-projection-cache keys by id and must invalidate; the id-reuse hazard its own spec describes becomes live the moment deletion exists.
4. The workspace registry references it from archivedSessionIds and session ordering.

A shape, for discussion

• sessions.delete(sessionId) host API + CLI verb, refusing live sessions and sessions with live descendants.
• Deletion removes transcript file(s), projection-cache entries, and workspace references in an order that leaves a crash mid-delete recoverable — registry last, so a re-listed orphan file is re-deletable rather than becoming a dangling registry row.
• Children of a deleted parent keep their transcripts; their parentSession simply stops resolving, which is already a supported state.
• Out of scope for a first cut: age-based retention policies — those compose from the primitive later.

Happy to build it against a maintainer-blessed shape; carrying the design in my fork's tracker meanwhile.

[argszero]

Verified all three of your claims against rc.2 (HEAD b150a55), and I checked the two persistence backends plus the archive registry while I was there. Your design shape is sound — registry-last ordering and "children keep their transcripts, parentSession stops resolving" both match how the code already tolerates unresolvable parents (liveLineage). Three additions and one simplification from the source:

1. The gap is in the persistence contract itself, not just the JSONL layer. The SessionPersistence surface is create / append / load / inspect / readFrom / locate — and it is identical in both backends: session-persistence-jsonl/src/index.ts:172-198 and session-persistence-sqlite/src/index.ts:97-113. Neither has a delete. So the primitive is a contract change (new interface method), and the two implementations have different crash-safety stories: JSONL unlinks the log file (its only current unlinks are the atomic-publish temp-hard-link mechanics, index.ts:562 — confirming your "no erasure" reading), while SQLite needs a DELETE row within the monotonic SCHEMA_VERSION schema — a physical-table concern, not a file concern. Worth spelling out in the shape so the design doesn't accidentally stay JSONL-shaped.

2. The archive set is append-only with no removal path — and it interacts with deletion. workspace.archiveSession (workspace/workspace/src/index.ts:244-253) appends to archivedSessionIds under the operation-chain lock; there is no unarchive anywhere (the includes-check makes re-archive a no-op). Because the client installs the full archive snapshot on every echo (client/runtime WorkspaceListState.archivedSessionIds mirror, host/archived-sessions-changed frame), a deleted-then-recreated id would be born hidden — the recreated session starts archived and can never be un-archived under current semantics. Your shape deletes "workspace references"; the archive-set membership is a second visibility coordinate that must be spliced in the same transaction, and its removal order belongs in the registry-last ordering (crash between archive-set removal and registry write leaves a re-deletable session on the safe side — same class of argument as your "re-listed orphan file is re-deletable").

3. The workspace accounting slot is explicit in the docs. The ui-workspace README states archive is "non-destructive: the log and the workspace accounting slot remain". The slot is per-workspace session membership — deletion must release it or workspace lists accumulate ghost accounting for sessions whose logs are gone. Your "registry last" covers this, but it's worth naming the accounting slot as a distinct reference so the checklist is complete.

4. Simplification: titles ride along with the transcript. session/title is a log-only event ("Log-only: it never enters the model surface or derived history", session-title/src/index.ts:95-102) — there is no separate title store to invalidate. Deleting the transcript deletes the title. Likewise the projection cache already validates each record's header identity (createdAt, cwd) before accepting a row (session-projection-cache/src/spec.ts:25-38), so a deleted id's rows are discarded on first read by design — the invalidation surface is smaller than it first appears: transcript(s) + projection-cache rows + archive-set membership + workspace accounting slot.

One inference boundary: session seq numbering is index-derived with no cross-process lock (the session/seq contract), so a delete-then-recreate at the same id is a case the current code never had to face on the write side — the projection-cache's checkpointIdentity covers the read side, but if your fork's tracker has a plan for the seq dimension I'd be interested to see it. If you want, I can also enumerate the wire surface (workspace.list baseline, event frames, CLI verbs) that a sessions.delete would need to touch, or sanity-check a specific ordering against the operation-chain in workspace/src/index.ts.

[nokkies]

Thorough review, thank you. Since posting the shape we have implemented and shipped it in our fork (tag 0.1.1-rc.2.sf.5), so I can answer each point from the field rather than the whiteboard.

1. Contract change - confirmed, and that is how we built it. SessionPersistence gained an abstract delete(id) whose base implementation throws loud (a backend that never opted in cannot silently no-op an erasure). JSONL removes the session's whole per-id directory (rm recursive with bounded retries for Windows handle lag), which also answers your point 4 for free - the title events go with the log. SQLite is exactly the physical-table concern you name: a delete-session.sql row under the monotonic schema, with ON DELETE CASCADE so event rows follow the session row.

2. Archive-set interaction - we resolved it by making archived a precondition. Delete refuses unless the session is (a) not live (session-live error first, the stronger blocker) and (b) archived (session-not-archived). Deletion is therefore a transition OUT of the archive set: the registry commit that forgets the session also splices its archive membership and its header/path indexes, storage-first so a refused or crashed attempt stays retryable. Born-hidden cannot arise - the id leaves archivedSessionIds in the same durable write that forgets the session, and a recreated id starts unarchived. We also shipped the missing unarchiveSession alongside (your "no unarchive anywhere" observation was real and users hit it the moment archive became a delete precondition).

3. Accounting slot - named and released. The registry delete drops headers, sessionPaths, and invalidSessionPaths entries; the path index is precisely what the workspace slot getters filter on, so no ghost accounting survives.

4. Titles/projections - matches our experience. Nothing title-specific to invalidate; the projection cache's header-identity check (createdAt/cwd) discards a recreated id's stale rows on first read, exactly as you infer.

The seq boundary: our answer is to make "no writer exists" a precondition rather than a locking problem. The persistence coordinator's forget(id) awaits the session's retirement and drops its in-memory state and prepared writers BEFORE storage delete runs; combined with the live-agent refusal there is no producer left that could append under the old identity. A recreate then mints a fresh header (new createdAt) and a fresh log starting at seq 0 - JSONL because the directory is gone, SQLite because CASCADE took the rows - and the header-identity check keeps old projection rows from bleeding in. Cross-process concurrent writers to one store remain out of scope, same as every other session write today.

Wire surface we ended up needing, if useful as a checklist: workspace.deleteSession + workspace.unarchiveSession RPCs (payload {sessionId}), the two error codes above, a durable workspace/session-deleted event mapped to a host/session-removed client frame, and client-side an Archived section rendering unarchive directly and delete behind a dialog that surfaces the host's refusal reason. Happy to compare orderings against the operation-chain if you take a run at it upstream - the implementation is in the fork history if you want the diffs.

[argszero]

Field report beats whiteboard — thanks for shipping it. I took the operation-chain run you offered, against rc.2, and came back with three facts that make your implementation cheaper than it looks, one ordering check, and two nits.

Three source facts that lower the migration cost:

1. The CASCADE is already in the strict schema — no FK migration needed. session-persistence-sqlite/resources/sql/schema.sql declares events.session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE with PRIMARY KEY (session_id, seq). Your delete-session.sql is literally DELETE FROM sessions WHERE id = ? plus the monotonic SCHEMA_VERSION bump (17 → 18, enforced at schema.ts:123-125). The event rows follow via the FK that is already there; you did not have to add or migrate a relationship.

2. sessions.incarnation TEXT NOT NULL already exists. The SQLite sessions table carries a per-session identity marker natively — stronger discrimination than createdAt/cwd for a recreated id (a fresh incarnation is guaranteed distinct even if a recreated header somehow collided on both fields). The projection-cache's checkpointIdentity (createdAt + cwd, session-projection-cache/src/spec.ts:25-38) is the JSONL-side equivalent; on SQLite the read side could validate incarnation for the same "stale rows can't bleed in" guarantee with a native column.

3. The FTS search index self-heals — your wire checklist needs no session-query step. session-query-sqlite reconciles on observation: it diffs the current session list against the index and computes persistentDeletes/liveDeletes (index.ts:444-453), removing rows for sessions that disappeared. A deleted session's search rows are swept on the next reconcile without the delete path touching the query store.

The operation-chain check. enqueueOperation (workspace/workspace/src/index.ts:648-658) serializes registry writes through an operationTail promise chain and calls recoverPendingMutation() before each op — so the registry half of deleteSession is automatically atomic. The storage half (persistence delete) runs outside that chain, which is fine iff both ends are idempotent: storage delete no-ops on a missing log/row, and the registry commit treats an already-forgotten id as a no-op. Then two concurrent deleteSession calls cannot interleave a check-then-write on the registry side, and a crash between storage delete and registry commit leaves the re-deletable state you designed for. Your storage-first ordering satisfies this — just make the storage delete tolerant of "already gone" so the second caller of a concurrent pair is harmless.

Two nits:

• JSONL unit is a per-id file, not a directory: logPath(root, cwd, id, compression) (session-persistence-jsonl/src/format.ts:201) = <root>/<project-dir>/<id>.jsonl[.zstd], with no persistent sidecar (the torn marker is coordinator-in-memory; the only other artifact is the transient tmp of the atomic-publish link() at index.ts:543-555, already cleaned by existing mechanics). rm recursive over a per-id directory works, but the upstream unit is a single file.
• "Archived as delete precondition" makes your new Archived section load-bearing: a never-archived session is exactly one archive-click away from deletable, and if that section is ever removed the delete path becomes unreachable for every session that was never archived. Fine as a deliberate gate — worth a doc comment so a future UI refactor doesn't orphan it.

Happy to diff against the fork history if you share the path — the CASCADE + incarnation findings suggest your SQLite side could shrink by a migration or two.

[nokkies]

A few corrections, and one thing I have to push back on.

Idempotency. You're right that the registry half is serialized by enqueueOperation and the storage half isn't, so both ends have to tolerate "already gone". Both do. JSONL resolves the log first and returns early if there isn't one, then removes with force: true. SQLite is DELETE FROM sessions WHERE id = ? inside begin-immediate, and an absent row commits a no-op, which is stated in the method's contract comment. So two concurrent callers are safe, and a crash between storage delete and registry commit leaves the re-deletable state rather than a wedged one.

There is no schema bump. You inferred 17 to 18. SCHEMA_VERSION is untouched and schema.sql is byte-identical to rc.2. delete-session.sql is a query file, not a schema change, and the reason is the CASCADE you found: events.session_id ... REFERENCES sessions(id) ON DELETE CASCADE with PRIMARY KEY (session_id, seq) was already in the strict schema, so deleting the session row is the whole operation. On the SQLite side this feature costs one query file.

Fact 3 removes a step from the checklist. If session-query-sqlite reconciles by diffing the live list into persistentDeletes/liveDeletes, the delete path needs no search-index call at all, and the wire surface I posted is one item too long. Good catch, and not one I would have found from the producer side.

The per-id unit is a directory. logPath is not <root>/<project-dir>/<id>.jsonl. sessionDir(root, cwd, id) returns join(projectDir(root, cwd), encodeSegment(id)), and logPath then joins session${logSuffix(compression)} inside that. The artifact is <root>/<project-dir>/<encoded-id>/session.jsonl[.zstd], so rm(dirname(logPath), { recursive: true }) targets the per-id directory and nothing wider. The encoded-id segment is what makes the id usable as a path component. If you are reading a revision other than b150a551b8, say so and I will look again.

The point behind the nit still holds. The contract should not be JSONL-shaped, which is why delete(id) is abstract with a base implementation that throws rather than no-ops. A backend that never opted into erasure has to fail loudly.

incarnation. This is worth more than you pitched it as. The column is already in the revision token: sqliteRevision builds ${storeIdentity}:incarnation:${row.incarnation}:revision:${row.revision}, so it already carries staleness and the read path exists. Having the projection cache prefer it on SQLite, with createdAt/cwd as the portable fallback, would cover the one case the current pair cannot: a recreated id that collides on both fields. We have not done it. Logged as a follow-up.

Archived gate. Agreed, and it needs the comment. A never-archived session sitting one click from deletable is the deliberate part. The section being removed in a refactor and taking the only route to delete with it is not something types will catch. The comment goes at the precondition rather than in the UI.

On sharing a path. My earlier "the implementation is in the fork history if you want the diffs" was wrong to offer, because that fork is private. Apologies for sending you after something you cannot open.

What does exist is a public patch queue, where branches are cut from an upstream commit and carry one fix each, so they cherry-pick with no fork context:

https://github.com/nokkies/dsh-upstream-patches

It currently holds one branch, the matcher and timeout fail-open fixes for #582 and #583/#460. Not this work. Session deletion touches both persistence backends, the workspace registry, two RPCs, a durable event with its client frame, and the Archived UI, so extracting it is a larger job than a guard-line fix and I am not going to put a date on it. If it goes up there I will link it here.

Everything above is checkable against rc.2 without the diffs. If you want to run an ordering against the operation-chain, I can write out our exact sequence step by step in a comment.

[argszero]

Both corrections accepted, and verified against rc.2 before replying.

The JSONL unit is a directory — my nit was wrong. I had only read the logPath signature, not its body. sessionDir(root, cwd, id) = join(projectDir(root, cwd), encodeSegment(id)) (format.ts:189-192) and logPath returns join(sessionDir(...), session${logSuffix(compression)}) (:201-207) — so the artifact is <root>/<project-dir>/<encoded-id>/session.jsonl[.zstd], and rm(dirname(logPath), { recursive: true }) targets the per-id directory and nothing wider. The encodeSegment(id) detail is the part worth keeping: the id is already made path-safe, which is what lets the per-id directory act as a clean deletion (and future quarantine) unit. Retracting the "file not directory" claim.

No schema bump — agreed, and the reasoning is instructive. SCHEMA_VERSION gates physical schema (the schema.ts:123-125 mismatch check compares on-disk structure), and a query-only resource like delete-session.sql changes no DDL — the pre-existing CASCADE does the whole job. I conflated "new resource" with "schema change"; your one-query-file outcome is the correct one. This is the strongest argument for keeping delete(id) abstract-with-loud-throw as the contract shape: JSONL's rm and SQLite's single DELETE are both unremarkable once the backends opt in.

Idempotency specifics confirmed as you described. JSONL resolve-first + force: true, SQLite begin-immediate with an absent row committing a no-op — both match the "second concurrent caller is harmless" property, and the crash-between-storage-and-registry case lands in the re-deletable state you designed.

incarnation: verified, and your follow-up is well-scoped. sqliteRevision (store.ts:386) already builds ${storeIdentity}:incarnation:${row.incarnation}:revision:${row.revision}, and schema.ts:292-293 UUID-validates the column — so the staleness signal is already in the revision token on the read path. Having the projection cache prefer incarnation on SQLite (with createdAt/cwd as the portable fallback) covers exactly the case checkpointIdentity cannot: a recreated id colliding on both header fields. That's a two-line preference change in the cache's identity comparison, and it makes the SQLite read side strictly stronger than the JSONL side — a good asymmetry to document rather than hide.

The thread has converged: design validated, fork implementation confirmed against upstream source, and the wire checklist is one item shorter per your own confirmation of the FTS self-heal. When you share the fork path I'll diff the ordering against the operation-chain.