This repository was archived by the owner on Aug 3, 2026. It is now read-only.
fix: torn stores refuse cleanly instead of dying by signal - #180
Merged
Conversation
Pairs with pathscale/DataBucket#66 (validated rkyv access on disk reads), pulled from its branch via a crates-io patch until 0.4.1 is published. The consumer-side cost of validation is six where-clause extensions in the space-index layer: CheckBytes on the archived key types, satisfied automatically by every derived Archive type. The torn-shutdown repro is now split against two bars. The active test, test_torn_store_fails_clean_never_by_signal, holds the invariant the fix delivers: a store torn by five mid-write kills never takes a process down with a signal — every load either succeeds or refuses with an error naming corruption, proven by round-tripping the kills and scanning in-process through an unwind boundary. The full bar, test_store_survives_torn_shutdowns, stays ignored: a dangling index link into a zeroed data region still reads as a phantom row of empty fields that validates perfectly, and only crash-consistent writes (WAL, shadow paging, page checksums) can meet it. Run against agencyzero's real poisoned store, the stack now reports InvalidSubtreePointer as a clean error where it previously died of SIGBUS: the corruption cascade (each SIGBUS a mid-write death planting the next tear) is broken.
pathscale
force-pushed
the
fix/torn-stores-refuse-cleanly
branch
from
August 1, 2026 13:06
fbd4f01 to
37381f3
Compare
added 2 commits
August 1, 2026 20:18
Persistence is best-effort by contract: consumers drain every catchable exit, the accepted loss window is the instant between in-memory and on-disk, and a SIGKILL mid-write may cost data with an index rebuild or snapshot restore as the recovery. The ignored test now says exactly that, so nobody reads it as a WAL work order.
The published 0.4.1 carries the validated page reads this branch was pulling from the PR branch; the crates-io patch goes away and the pin moves forward. All 439 tests pass in both validate-reads states.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pairs with pathscale/DataBucket#66 (validated rkyv access on every disk-read path), pulled from its branch via a crates-io patch until data_bucket 0.4.1 is published — swap the patch for a version bump then.
Consumer-side cost of validation: six where-clause extensions in the space-index layer (CheckBytes on archived key types, satisfied automatically by every derived Archive type). Full suite passes: 439 tests.
The torn-shutdown repro from #179 is now split against two bars:
test_torn_store_fails_clean_never_by_signal(active, passes with the fix): a store torn by five mid-write kills never takes a process down with a signal — every load either succeeds or refuses with an error naming corruption. Verified by round-tripping kills through writer children (a child that dies on its own must die with an exit code, not a signal) and scanning the final store in-process through an unwind boundary.test_store_survives_torn_shutdowns(still ignored, the full bar): no phantom rows. A dangling index link into a zeroed data region reads as a row of empty fields that validates perfectly — only crash-consistent writes (WAL / shadow paging / page checksums) can meet this bar.Proof on real corruption: run against agencyzero's poisoned store from this morning, the stack now reports
InvalidSubtreePointer { address: 5202627079, size: 7679975808 }as a clean exit-101 error where it previously died of SIGBUS. That breaks the production corruption cascade — each SIGBUS was itself a mid-write death planting the next tear.