Skip to content

Report a non-committed shared-structure save as not-saved - #1154

Merged
kriszyp merged 2 commits into
mainfrom
kris/savestructures-commit-signal
Jun 9, 2026
Merged

Report a non-committed shared-structure save as not-saved#1154
kriszyp merged 2 commits into
mainfrom
kris/savestructures-commit-signal

Conversation

@kriszyp

@kriszyp kriszyp commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

RecordEncoder.saveStructures (RocksDB path) wraps rootStore.transactionSync, which returns undefined when the structure-save txn was aborted (ERR_ALREADY_ABORTED is swallowed) — and the success path also fell through to undefined. msgpackr only re-packs on a === false return, so a swallowed abort looked like success: the already-encoded record (referencing the new structure id) was written even though the structure was never persisted → later Record id is not defined for N on decode (the dominant CDI rt-dev error, ~1991/10min, on local reads).

This returns true only on a committed save and false otherwise (CAS conflict or aborted txn).

Why this is the real fix for the desync

A concurrent-writer harness showed msgpackr's CAS coordination is sound; the bug is in the RocksDB save reporting. The existing retryOnBusy: true is necessary-but-insufficient (it caps at 3 busy retries, then throws; and the abort path returns undefined, never re-packing).

Pairing (must land together)

This is half the fix and depends on kriszyp/msgpackr#186: returning false alone doesn't recover, because msgpackr's re-pack reused the cached transition and re-emitted the same unpersisted-structure record. #186 marks structures uninitialized on save-failure so the re-pack reloads durable structures, rebuilds the trie, and re-mints + re-saves. Blocked on msgpackr 1.11.14 publish + dep bump.

Verified end-to-end (repro ~/dev/scripts/savestructures-abort-repro.cjs): with both fixes, a declined/aborted save → re-pack → reload + re-mint + re-save → no dangling record.

Note

Independent of the typed-structs default (#1152) — this is the classic shared-structure path, so it's needed regardless. Should cherry-pick to v5.0 (the deployed line where the error occurs).

🤖 Generated by Claude (Opus 4.7).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates RecordEncoder.ts to explicitly return false instead of undefined when a RocksDB transaction in saveStructures fails or aborts, preventing downstream decoding issues in msgpackr. The reviewer pointed out a critical issue where this.structureUpdate is updated inside the transaction callback before the transaction is guaranteed to commit, which could lead to desyncs on aborted transactions. They provided a code suggestion to only update this state after a successful commit.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread resources/RecordEncoder.ts Outdated
Comment on lines +236 to +241
this.structureUpdate = structures;
return true;
},
{ retryOnBusy: true }
);
return committed === true ? true : false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If transactionSync aborts or fails to commit after the callback returns (e.g., due to a conflict or a busy timeout during the commit phase), the callback has already executed and set this.structureUpdate = structures. This means this.structureUpdate will remain set even though the transaction was never committed.

This can lead to downstream issues, such as the audit log incorrectly writing the HAS_STRUCTURE_UPDATE flag (lines 670-673) for a failed structure update, causing desyncs during replication or replay.

To prevent this side-effect on aborted transactions, this.structureUpdate should only be updated after transactionSync successfully commits (i.e., when committed === true). This also simplifies the redundant ternary operator.

Suggested change
this.structureUpdate = structures;
return true;
},
{ retryOnBusy: true }
);
return committed === true ? true : false;
return true;
},
{ retryOnBusy: true }
);
if (committed === true) {
this.structureUpdate = structures;
return true;
}
return false;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in c7e2038. Moved this.structureUpdate = structures out of the transactionSync callback so it's only assigned when committed === true. On an aborted/declined txn it now stays unset, so no spurious HAS_STRUCTURE_UPDATE can be flagged for an unpersisted structure. Also dropped the redundant ternary. — Claude

@claude

claude Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

kriszyp and others added 2 commits June 8, 2026 22:04
RecordEncoder.saveStructures wrapped rootStore.transactionSync, which returns
`undefined` when the structure-save txn was aborted (ERR_ALREADY_ABORTED is
swallowed) — and the success path also fell through to `undefined`. msgpackr only
re-packs on a `false` return, so a swallowed abort looked like success and the
already-encoded record (referencing the new structure id) was written even though
the structure was never persisted → later "Record id is not defined for N".

Return `true` only on a committed save and `false` otherwise (CAS conflict or
aborted txn). Paired with the msgpackr fix that marks structures uninitialized on
save-failure, returning `false` makes the re-pack reload the durable structures
and re-mint + re-save, so records never reference an unpersisted structure.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move `this.structureUpdate = structures` out of the transactionSync
callback so it is only set on a confirmed commit. Setting it inside the
callback left it dangling on an aborted txn, which could flag a spurious
HAS_STRUCTURE_UPDATE in the audit log for a structure that was never
persisted. Addresses review feedback.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/savestructures-commit-signal branch from c7e2038 to 14cf21e Compare June 9, 2026 04:07
@kriszyp kriszyp mentioned this pull request Jun 9, 2026
2 tasks
@kriszyp
kriszyp merged commit a80d486 into main Jun 9, 2026
36 of 37 checks passed
@kriszyp
kriszyp deleted the kris/savestructures-commit-signal branch June 9, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant