Replies: 1 comment
|
Provenance, which I should have stated up front. This report was drafted by an AI coding agent running on my machine, during the session in which we diagnosed and fixed the problem locally. The source citations come from reading the installed 0.1.5-rc.2 sources, and the measurements from running the checks described; the Limitations section states what was and was not measured. I reviewed it and stand behind the technical content, but please weigh it knowing how it was produced. The repair tool described is local and has been reviewed by nobody other than the agent and me. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Packages:
@deepseek-ai/dsh-session-persistence-jsonl(projectKey,assertStoredIdentity),@deepseek-ai/dsh-app-bootPlatform: all (the invariant is platform-independent)
Verified on:
0.1.5-rc.2, Windows 11 (10.0.26100)Summary
Session logs are stored under a folder whose name is a pure function of the session header's own
cwd:projectKey(dsh-session-persistence-jsonl/lib/index.js:874-892) maps separators (/,\,:) to-, keeps[A-Za-z0-9._-]literal, escapes everything else as~XXXX, and wraps the result in--…--. So a session recorded inD:\工作文件lives in--D-~5DE5~4F5C~6587~4EF6--.Renaming that folder — a legitimate, unremarkable user action — breaks the pairing for every
session ever recorded under it, and the store rejects those logs:
The expected path is recomputed from the header's
cwd, so once the folder moves, the physicalpath and the header disagree and the log is reported as corrupt. There is no documented way to
rewrite a stored header, so the only repair available to us was to rewrite the compressed frame
directly.
The obvious repair is the wrong one
This is worth calling out because it is what a competent operator tries first, and it makes things
worse rather than better.
If you rename the bucket folder to match the new workspace path (
--D-workspace--) but leave theheaders naming the old path, the two now disagree — whereas before the rename they agreed, and the
sessions merely sat in a bucket whose path no longer existed. Renaming the folder alone converts a
self-consistent-but-stale state into exactly the state the assertion above rejects.
The folder rename and the header rewrite have to happen together. Nothing in the product says so.
Steps to reproduce
Expected: history is still available (perhaps with a re-key).
Actual: the stored logs no longer satisfy
assertStoredIdentity; the header'scwdnames a path thatno longer exists.
Observed impact
On one machine, 25 sessions were affected at once. The workspace rename also produced a second,
louder failure (a
link:dependency junction left dangling, so the harness refused to start), andwhile diagnosing the two we had to repair all 25 logs by hand. The repair required:
header.cwd,That is a reasonable amount of work for a user to be forced into by renaming a directory.
Why a byte-safe repair is possible
session.v<N>.jsonl.zstdon this version is a two-frame zstd stream: frame 1 is exactly theheader line, frame 2 holds every event row. That means a repair can recompress frame 1 and copy the
rest verbatim, and can prove afterwards that frame 2 is unchanged. Any repair API or documented
procedure should use that property rather than re-encoding the whole stream.
Suggested improvements (any one of these would have prevented the incident)
dsh session repair --cwd <new-path>that rewrites the headers of every log whose folder nolonger matches, using the frame-level technique above. The product knows the invariant; it is
better placed to repair it than a user.
cwdno longer exists but the log is otherwiseintact, migrate it (move the folder and rewrite the header) instead of reporting it corrupt.
sameFilealready exists in the same function, so path-drift is evidently an anticipatedcondition.
("the folder name is derived from the header's cwd, and the two are asserted to agree; do not
rename the bucket on its own") should be stated where a user can find it — because the natural
fix is the destructive one.
Limitations of this report
projectKeyare cited from the installed0.1.5-rc.2sources; the mismatch statewas reproduced and measured directly (25 logs), but I did not separately measure the behaviour of
session enumeration as opposed to session reading — the rejection is proven for the read path
shown above.
Addendum: what a fix for this must verify (a note on verification)
This is offered because the failure mode above is unusually easy to reproduce in the repair tool
itself, and a repair that is not verified against the right predicate will look successful while
making the state worse.
During this incident the rename tool did exactly that. It moved the bucket folder to the new key,
counted the entries in it, saw 25 of 25, and reported the session history as preserved. It had in
fact created the very disagreement
assertStoredIdentityrejects — the folder rename is only half ofthe repair, and the half it performed is the destructive half. Nothing in the tool's output was
false about its own actions; the false part was the conclusion.
The generalisable rule:
Two consequences that apply directly to this component:
Verify content, not containers. Counting entries in a bucket folder proves the folder exists
and holds something. It says nothing about whether those entries satisfy the invariant. The check
that would have caught this is cheap and exact:
— the same predicate
assertStoredIdentityapplies at read time. Any repair tool (or any built-inmigration, if one is added) should assert this for every affected log, and refuse to report success
otherwise.
A verifier is code, and needs the same scrutiny as the thing it verifies. The first version of
our verification script died on malformed JSON instead of reporting a failure, and separately
reported a sandbox permission error as a real defect — both of which would have turned a broken
repair into a green light. A verification step that has not itself been exercised against a
deliberately broken fixture is not evidence.
For this specific issue, a fix should therefore ship with at least one test that starts from a
mismatched fixture (folder re-keyed, header still stale) and asserts that the repair either
restores agreement or fails loudly — rather than a test that only covers the already-consistent case.
All reactions