Conversation
Mic92
enabled auto-merge (rebase)
July 15, 2026 11:30
AddToStoreNar needs a path's references valid first, but that was only enforced within one build: the hub topologically ordered a build's own missing paths, while the worker deduped shared paths across concurrent builds (owner/awaiter). Two builds could each own one half of a reference edge and, depending on staging-permit order, the referrer was imported before its reference, failing the build with "path ... is not valid". Move the dedupe to the hub: the worker reports every invalid path, and the hub skips paths already streamed to this worker session. Staging phases are serialized on the per-session permit and the worker imports NARs in stream order, so a skipped path is committed before a later build's inputs arrive; the worker re-checks skipped paths when its staging completes. Paths of a failed build are dropped from the session set so a later build streams them again. Removes the worker-side SessionImports machinery; docs/staging.als models the invariant (Alloy 6).
When an output or tmp-dir unpack task fails, the next chunk send only sees a closed channel and reported a generic "unpacker died", hiding the actual cause (e.g. no space left on device). Reap the task and propagate its error, with the store path for output unpacks.
Pull in latest harmonia crates so tribuchet builds against current store protocol and NAR handling fixes.
writeText-style derivations do `mv "$textPath" $out`; on a rootless Linux worker the .attr file is unpacked by the worker uid, which is unmapped in the build's user namespace, so the kernel refuses the unlink (may_delete: HAS_UNMAPPED_ID -> EPERM). Currently fails; the fix needs the tmp dir to be unpacked with mapped ownership.
The worker unpacks the tmp dir archive as its own uid, which is unmapped in the build's user namespace, so the kernel refuses to unlink those files there (may_delete: HAS_UNMAPPED_ID -> EPERM). writeText's `mv "$textPath" $out` hits this on the passAsFile .attr-* files. Attach the top tmp dir as a fourth fd to the Allocate call and have sandboxd chown the tree to the leased base uid before replying. Only worker-owned inodes are re-owned, so the worker cannot repurpose this to take over files it can merely open. Disk cleanup now covers the whole leased-owned top/ tree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AddToStoreNar needs a path's references valid first, but that was only
enforced within one build: the hub topologically ordered a build's own
missing paths, while the worker deduped shared paths across concurrent
builds (owner/awaiter). Two builds could each own one half of a
reference edge and, depending on staging-permit order, the referrer was
imported before its reference, failing the build with
"path ... is not valid".
Move the dedupe to the hub: the worker reports every invalid path, and
the hub skips paths already streamed to this worker session. Staging
phases are serialized on the per-session permit and the worker imports
NARs in stream order, so a skipped path is committed before a later
build's inputs arrive; the worker re-checks skipped paths when its
staging completes. Paths of a failed build are dropped from the session
set so a later build streams them again.
Removes the worker-side SessionImports machinery; docs/staging.als
models the invariant (Alloy 6).