Skip to content

feat(fspy): let the caller size the tracking channel - #680

Open
wan9chi wants to merge 6 commits into
mainfrom
claude/fspy-shm-capacity-env
Open

feat(fspy): let the caller size the tracking channel#680
wan9chi wants to merge 6 commits into
mainfrom
claude/fspy-shm-capacity-env

Conversation

@wan9chi

@wan9chi wan9chi commented Aug 17, 2026

Copy link
Copy Markdown
Member

Motivation

The shared memory a tracked run reports its file accesses through is four gibibytes, fixed. Nothing could ask for a smaller channel, so no test could put a task in front of one too small to hold its records — and that path decides whether a run may be cached. It had no coverage at all.

What changes

VP_RUN_INTERNAL_FSPY_SHM_CAPACITY overrides the size, read next to the channel call that uses it and falling back to the same four gibibytes when unset. Nothing about a normal run moves. The variable is internal: it exists so a test can shrink the channel until a task overruns it, and nothing outside this repository should set it.

The read sits at the point of use rather than travelling there. A size threaded through Command would put a builder method, a public default constant and a lookup in the runner between the variable and its one consumer, and would drag every other caller — the benchmark launcher, the e2e tool, the examples, fspy's own tests — into a decision none of them want to make. It also keeps the whole thing behind one cfg: fspy::ipc is already cfg(not(target_env = "musl")), so the size lives behind the same gate as the channel it sizes.

The test

vtt stat-many <count> [name-length] stats generated names to be tracked, under distinct names so none can fold into a single record, and prints its last line afterwards to show the process outlived them. It absorbs stat_long_filename, which did the same thing along the other axis; the /dev/shm case that used it now says stat-many 1 1048576. The e2e case makes twenty thousand accesses under a 64 MiB channel, which holds every one, so the run caches like any other — that is what tells us the size arrived where it was meant to.

Record count is the lever rather than record size, because size cannot be pushed far enough on every platform. A path reaches the tracer on Windows through a UNICODE_STRING, whose length field is a u16, so no single record there exceeds 64 KiB however long a name the caller asks for — an earlier version of this case tried one 2 MiB path and Windows had room to spare for it.

The case skips musl, which has no preload: those builds collect through the seccomp supervisor, on the runner's own side of the boundary, so there is no shared-memory channel there to fill.

The case worth testing, a channel too small for the task, has to wait for #675. On this base a full channel aborts the task process, and the panic it prints carries a thread id, a toolchain path, a backtrace and a platform's own abort code, so there is nothing there that snapshots the same way twice. #675 makes that a skipped record and a reported reason instead, and updates this snapshot to show it.

Split out of #675.

🤖 Generated with Claude Code

The shared memory a tracked run reports its file accesses through was a
constant in `fspy`, four gibibytes wide. How many accesses a program makes is
the runner's business rather than the tracer's, and nothing could ask for a
different size, so no test could put a task in front of a channel too small
for it.

`Command::shm_capacity` sets it, and the runner reads
`VP_RUN_INTERNAL_FSPY_SHM_CAPACITY` for the value, keeping the same four
gibibytes when the variable is unset. The variable is internal: it exists so
a test can shrink the channel until a task overruns it, and nothing outside
this repository should set it.

A builder method rather than a second argument to `Command::new`, because the
benchmark measures both revisions of `fspy` with a single launcher, overlaying
the head's launcher source onto the baseline checkout. A launcher calling a
signature only the head has cannot build the baseline arm. Leaving `new`
alone also keeps the e2e tool, the examples and fspy's own tests out of this,
since none of them care what size they get.

The e2e case that comes with it stats one 2 MiB path, the largest single
record tracking can be asked to hold, under a 64 MiB channel. That leaves
room to spare, so the run caches like any other, which is what tells us the
size arrived. The interesting case, a channel with no room for the record,
has to wait: today it aborts the task process, and the panic it prints
carries a thread id, a toolchain path, a backtrace and a platform's own abort
code, none of which snapshot the same way twice.

`vtt stat_long_filename` needed one fix to run there at all. Windows reports
an over-long name as `ERROR_FILENAME_EXCED_RANGE`, which arrives as
`InvalidFilename` rather than the `ENAMETOOLONG` unix returns, so the command
exited 1 where it means to carry on: it exists to have the access attempted
and recorded, not to find a file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the claude/fspy-shm-capacity-env branch from 5ca10c9 to f7a2284 Compare August 17, 2026 06:23
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +0.36%  [ -6.47% ..  +8.29%]  overhead   +53.04%
dynamic/access             change  +1.75%  [-14.43% .. +27.22%]  overhead    +6.68%
dynamic/access-relative    change  +1.13%  [-13.75% .. +24.55%]  overhead   +47.12%
static/launch              change  +0.71%  [ -8.18% .. +11.27%]  overhead  +141.32%
static/access              change  -1.73%  [-16.98% ..  +6.26%]  overhead  +824.99%
static/access-relative     change  +0.23%  [ -6.89% ..  +6.80%]  overhead +1154.04%

macos

dynamic/launch             change  -0.13%  [ -3.68% ..  +4.11%]  overhead  +228.99%
dynamic/access             change  -0.27%  [-13.40% .. +15.71%]  overhead    +4.79%
dynamic/access-relative    change  +2.03%  [-10.31% .. +29.14%]  overhead  +253.76%

windows

dynamic/launch             change  +0.13%  [ -3.24% ..  +3.46%]  overhead   +27.13%
dynamic/access             change  +0.00%  [ -1.28% ..  +1.12%]  overhead    +1.50%
dynamic/access-relative    change  +0.00%  [ -2.30% ..  +2.73%]  overhead    +2.02%

Win32 spells `ERROR_FILENAME_EXCED_RANGE` without the second E, and the
comment naming it is more use to a reader than the spelling checker is, so
the word joins the allowed list beside the other Windows one.

The `shm_capacity` field needed no musl exemption after all. It is read
there, by the setter, so claiming it is dead made the expectation unfulfilled
instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the claude/fspy-shm-capacity-env branch from 7bdf514 to 9eb5f30 Compare August 17, 2026 06:28
wan9chi and others added 4 commits August 17, 2026 14:37
Windows never overran the small channel the first version of this case set
up, because a path record cannot get large enough there. A path reaches the
tracer through a `UNICODE_STRING`, whose length field is a `u16`, so however
long a name the caller asks for, no single record exceeds 64 KiB. Its 1 MiB
channel had room to spare, tracking came back complete, and the run cached.

Record count is the portable lever, and the slot table makes it exact: one
slot per 64 bytes of the region, so a channel of a given size admits a known
number of records whatever their paths look like. `vtt stat-many` makes as
many accesses as asked for, under distinct names so none can fold into one
record, and prints last to show the process outlived them.

The case skips musl, which has no preload: those builds collect through the
seccomp supervisor, on the runner's own side of the boundary, so there is no
shared-memory channel there to fill.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`4 << 30` says how the number is built; `4 * 1024 * 1024 * 1024` says what
it is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The size arrived through a builder on `fspy::Command`, a public default
constant, and a `LazyLock` in the runner that read the override and passed
it down. Three places to look, for a number with exactly one consumer.

It now reads the override next to the `channel` call that uses it, and falls
back to the default there. `Command` goes back to what it was, and so do the
e2e tool, the examples, the benchmark launcher and fspy's own tests, none of
which ever wanted a say in the size.

The runner no longer names the variable at all, which also settles the musl
question: `fspy::ipc` is already `cfg(not(target_env = "musl"))`, so the size
lives behind the same gate as the channel it sizes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two commands stat generated names to be tracked; one varied the name's
length and the other how many names. They are now `stat-many <count>
[name-length]`, which also puts the name in kebab case with every other
subcommand.

Count leads because it is the knob that travels. A long name only fills a
channel on unix: on Windows a path reaches the tracer through a
`UNICODE_STRING` whose length is a `u16`, so no single record there exceeds
64 KiB however long a name the caller asks for.

Names now carry their index, so a run of them cannot collapse into one
record, and padding fills out whatever length is asked for. The `/dev/shm`
case keeps its one 1 MiB name as `stat-many 1 1048576`, and gains the
trailing line that reports the process survived its accesses.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant