sendfile(2) binding (foundation, no caller yet) - #102
Conversation
The foundation for serving static files without copying them through userspace. No caller yet — the event loop can still only send a flat buffer it already holds, and giving it an fd-backed response body is the larger change this unblocks. The two platforms disagree about nearly everything except the name, and this module exists to make them agree about what a caller wants: "send up to `count` bytes from `in_fd` at `offset` to `out_fd`; tell me how many went and whether to come back." Linux ssize_t sendfile(out_fd, in_fd, off_t *offset, size_t count) Darwin int sendfile(fd, s, off_t offset, off_t *len, hdtr, flags) Two traps are worth naming, because both are silent: - The first two arguments are in the OPPOSITE order. Darwin takes the file then the socket; Linux takes the socket then the file. A swap compiles cleanly on both and fails only at runtime, which is why the tests actually move bytes through a socketpair and compare them rather than just checking a return code. - Darwin reports the count through its in/out `len` parameter on EAGAIN as well as on success — a short write is `-1` WITH a positive count. A caller that reads -1 as "nothing happened" would resend those bytes and corrupt the response. `SendFileResult` carries `sent` alongside `again` so that cannot be expressed. No `hdtr` on the Darwin side: Linux has no equivalent, so a portable caller has to send its headers separately anyway, and using the header vector on one platform only would mean two orderings to reason about for no gain. The in/out parameter is a stack local addressed with `Pointer(to=)` rather than a one-element heap allocation — the callee writes it before returning, so it cannot outlive the frame, and it keeps this file off the warning ratchet (`alloc` without a `Layout` is deprecated with no replacement on this toolchain). Ratchet holds at 68. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`smoke-asgi-fanout` opens six streams against two workers and requires them to land on both. Nothing makes that happen: with a shared listener the accept race can legitimately put all six on one worker, and then there is no cross-worker delivery left to verify. Measured on an idle laptop, interleaving the two binaries over 20+ runs each: main fails this roughly 5% of the time and so does the branch it was found on. It is pre-existing and unrelated to whatever PR draws the short straw — it just reads as a mystery failure there. The fix separates the two outcomes, because they are not the same kind of event. A stream that misses the broadcast is the real failure this probe exists to catch, and still exits immediately — retrying it would turn a genuine delivery bug into an intermittent one. Streams that failed to spread are a failed *setup*: the attempt is discarded and the race is run again, and only a run that cannot achieve spread in `FANOUT_ATTEMPTS` (default 4) tries fails, because at that point it is no longer luck. Verified load-bearing: asking for three workers from a two-worker server retries and then fails, and the retry notice names what happened rather than passing quietly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
CI note. The first smoke run here failed — on Note That makes this PR fairly strong evidence the flake is pre-existing, and it is now the second PR it has cost a red build. I've extracted the fix to #103, off |
Clean automerge; the cherry-picked fan-out probe fix is byte-identical to the one that landed via #103, so it resolves to a no-op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Groundwork for the zero-copy static-file step
docs/ROADMAP.mdrecords. No caller yet — the event loop can still only send a flat buffer it already holds, and giving it an fd-backed response body is the larger change this unblocks. Opening it separately because it is self-contained, independently testable, and makes the eventual feature PR smaller.Why a module and not two
external_calls at the use siteThe platforms disagree about nearly everything except the name:
Two traps, both silent:
-1with a positivelen. A caller reading-1as "nothing happened" would resend those bytes and corrupt the response.SendFileResultcarriessentalongsideagainso that state cannot be expressed.No
hdtron the Darwin side: Linux has no equivalent, so a portable caller must send headers separately anyway, and using the header vector on one platform only would mean two orderings to reason about for no gain.Notes
The in/out parameter is a stack local addressed with
Pointer(to=)rather than a one-element heap allocation — the callee writes it before returning, so it cannot outlive the frame, and it keeps this file off the warning ratchet (allocwithout aLayoutis deprecated with no replacement on this toolchain). Ratchet holds at 68.test_sendfile.mojocompiles and exercises the module on everytest-httprun, which is deliberately the guard this repo already uses forffi_exports.mojo— the fix for the timepackages/m0-core/ffi/rotted because nothing compiled it. 5 tests: the no-op guards, a real whole-file transfer, and an offset transfer (the Range case). The partial-send path is documented as not covered here — forcing it needs a full socket buffer and a peer that never reads, which is timing-dependent at unit scale; it belongs in the loop-level smoke that serves a file larger than every buffer.build-all+test-allclean.🤖 Generated with Claude Code