feat(fspy): let the caller size the tracking channel - #680
Open
wan9chi wants to merge 6 commits into
Open
Conversation
The shared memory a tracked run reports its file accesses through was a constant in `fspy`, four gibibytes wide. How many accesses a program makes is the runner's business rather than the tracer's, and nothing could ask for a different size, so no test could put a task in front of a channel too small for it. `Command::shm_capacity` sets it, and the runner reads `VP_RUN_INTERNAL_FSPY_SHM_CAPACITY` for the value, keeping the same four gibibytes when the variable is unset. The variable is internal: it exists so a test can shrink the channel until a task overruns it, and nothing outside this repository should set it. A builder method rather than a second argument to `Command::new`, because the benchmark measures both revisions of `fspy` with a single launcher, overlaying the head's launcher source onto the baseline checkout. A launcher calling a signature only the head has cannot build the baseline arm. Leaving `new` alone also keeps the e2e tool, the examples and fspy's own tests out of this, since none of them care what size they get. The e2e case that comes with it stats one 2 MiB path, the largest single record tracking can be asked to hold, under a 64 MiB channel. That leaves room to spare, so the run caches like any other, which is what tells us the size arrived. The interesting case, a channel with no room for the record, has to wait: today it aborts the task process, and the panic it prints carries a thread id, a toolchain path, a backtrace and a platform's own abort code, none of which snapshot the same way twice. `vtt stat_long_filename` needed one fix to run there at all. Windows reports an over-long name as `ERROR_FILENAME_EXCED_RANGE`, which arrives as `InvalidFilename` rather than the `ENAMETOOLONG` unix returns, so the command exited 1 where it means to carry on: it exists to have the access attempted and recorded, not to find a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
wan9chi
force-pushed
the
claude/fspy-shm-capacity-env
branch
from
August 17, 2026 06:23
5ca10c9 to
f7a2284
Compare
fspy benchmarklinuxmacoswindows |
Win32 spells `ERROR_FILENAME_EXCED_RANGE` without the second E, and the comment naming it is more use to a reader than the spelling checker is, so the word joins the allowed list beside the other Windows one. The `shm_capacity` field needed no musl exemption after all. It is read there, by the setter, so claiming it is dead made the expectation unfulfilled instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
wan9chi
force-pushed
the
claude/fspy-shm-capacity-env
branch
from
August 17, 2026 06:28
7bdf514 to
9eb5f30
Compare
Windows never overran the small channel the first version of this case set up, because a path record cannot get large enough there. A path reaches the tracer through a `UNICODE_STRING`, whose length field is a `u16`, so however long a name the caller asks for, no single record exceeds 64 KiB. Its 1 MiB channel had room to spare, tracking came back complete, and the run cached. Record count is the portable lever, and the slot table makes it exact: one slot per 64 bytes of the region, so a channel of a given size admits a known number of records whatever their paths look like. `vtt stat-many` makes as many accesses as asked for, under distinct names so none can fold into one record, and prints last to show the process outlived them. The case skips musl, which has no preload: those builds collect through the seccomp supervisor, on the runner's own side of the boundary, so there is no shared-memory channel there to fill. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`4 << 30` says how the number is built; `4 * 1024 * 1024 * 1024` says what it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The size arrived through a builder on `fspy::Command`, a public default constant, and a `LazyLock` in the runner that read the override and passed it down. Three places to look, for a number with exactly one consumer. It now reads the override next to the `channel` call that uses it, and falls back to the default there. `Command` goes back to what it was, and so do the e2e tool, the examples, the benchmark launcher and fspy's own tests, none of which ever wanted a say in the size. The runner no longer names the variable at all, which also settles the musl question: `fspy::ipc` is already `cfg(not(target_env = "musl"))`, so the size lives behind the same gate as the channel it sizes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two commands stat generated names to be tracked; one varied the name's length and the other how many names. They are now `stat-many <count> [name-length]`, which also puts the name in kebab case with every other subcommand. Count leads because it is the knob that travels. A long name only fills a channel on unix: on Windows a path reaches the tracer through a `UNICODE_STRING` whose length is a `u16`, so no single record there exceeds 64 KiB however long a name the caller asks for. Names now carry their index, so a run of them cannot collapse into one record, and padding fills out whatever length is asked for. The `/dev/shm` case keeps its one 1 MiB name as `stat-many 1 1048576`, and gains the trailing line that reports the process survived its accesses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The shared memory a tracked run reports its file accesses through is four gibibytes, fixed. Nothing could ask for a smaller channel, so no test could put a task in front of one too small to hold its records — and that path decides whether a run may be cached. It had no coverage at all.
What changes
VP_RUN_INTERNAL_FSPY_SHM_CAPACITYoverrides the size, read next to thechannelcall that uses it and falling back to the same four gibibytes when unset. Nothing about a normal run moves. The variable is internal: it exists so a test can shrink the channel until a task overruns it, and nothing outside this repository should set it.The read sits at the point of use rather than travelling there. A size threaded through
Commandwould put a builder method, a public default constant and a lookup in the runner between the variable and its one consumer, and would drag every other caller — the benchmark launcher, the e2e tool, the examples, fspy's own tests — into a decision none of them want to make. It also keeps the whole thing behind onecfg:fspy::ipcis alreadycfg(not(target_env = "musl")), so the size lives behind the same gate as the channel it sizes.The test
vtt stat-many <count> [name-length]stats generated names to be tracked, under distinct names so none can fold into a single record, and prints its last line afterwards to show the process outlived them. It absorbsstat_long_filename, which did the same thing along the other axis; the/dev/shmcase that used it now saysstat-many 1 1048576. The e2e case makes twenty thousand accesses under a 64 MiB channel, which holds every one, so the run caches like any other — that is what tells us the size arrived where it was meant to.Record count is the lever rather than record size, because size cannot be pushed far enough on every platform. A path reaches the tracer on Windows through a
UNICODE_STRING, whose length field is au16, so no single record there exceeds 64 KiB however long a name the caller asks for — an earlier version of this case tried one 2 MiB path and Windows had room to spare for it.The case skips musl, which has no preload: those builds collect through the seccomp supervisor, on the runner's own side of the boundary, so there is no shared-memory channel there to fill.
The case worth testing, a channel too small for the task, has to wait for #675. On this base a full channel aborts the task process, and the panic it prints carries a thread id, a toolchain path, a backtrace and a platform's own abort code, so there is nothing there that snapshots the same way twice. #675 makes that a skipped record and a reported reason instead, and updates this snapshot to show it.
Split out of #675.
🤖 Generated with Claude Code