You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
The machine half of cache-bench doctor. It reads what the kernel publishes about the box it is on, the CPU, the memory, the governor, the mitigations, the load average and whether there are counters to count with, and then measures the profile that is about to be swept against it. A profile that names more cores than the machine has, a profile that sweeps the cycles half of the matrix on a host with no PMU, a working set that would not fit in memory, or a load average that says somebody else is using the machine: each of those is refused rather than warned about, because a warning printed at the start of a job that runs for eight days is read by nobody and each of them produces numbers rather than an error.
doctor --write, which records what the machine is in host.json next to the results. A fact the machine does not publish is refused rather than written as unknown, since this file is the whole of what a published results directory says about where its numbers came from. Nothing in it names the machine, and if a hosts.toml is there its names are checked against the file before it is written.
doctor --deep, which starts each of the seven servers in turn, waits for it to answer, stops it and checks the group is gone. A binary of the wrong architecture, a server built without unix socket support, a Garnet whose runtime is not installed: all of them read as a correct config and all of them fail on the first run of a sweep instead of here.
Every file the kernel publishes is parsed by a function that takes a string, with tests over the text an ARM box, an x86 box and a container each produce. A parser that has only ever run on the machine it was written on is a parser nobody has checked.
cache-bench sweep, the loop around run. It measures engine, then thread count, then pipeline depth, then whether counters are attached, then the run number, which is the original's order and which keeps all the runs of one cell together in time, so an hour where somebody else was using the machine comes out as one cell that is visibly wrong instead of a slight tilt across every cell in the matrix. It takes the results directory once for the whole sweep rather than once per cell, and it checks that the config names a binary for every engine it is about to sweep before it starts, because a config that never named Garnet is one that fails on day three of eight having measured everything before it.
Restart by file existence, with a file that will not parse counted as absent rather than as a run. A sweep that lost power partway through a write leaves a file of the right name holding half a run, and that file measured again is a run, while that file trusted is a truncated measurement in a median that nothing downstream can tell from a real one.
results/logs/sweep.jsonl, one line per run attempted, in the order they were attempted, with the load average taken just before each one. Before rather than after, because the question it answers is whether the machine was already busy and a run is itself load. Nothing reads this file. It is there for the person who looks at one odd cell a week later and wants to know what else was happening at three in the morning on day six.
results/failures.json, which names every cell that was attempted and produced no file, with the reason verbatim, when it last failed and how many times it has been tried across every sweep of that directory. A chart cannot draw a cell that is not there, and the alternative to naming it is a zero, and a zero is a claim about an engine while an absence is not.
A failed cell no longer stops the sweep, since the other ten thousand are still worth measuring, but a sweep that did not measure everything says so on the way out rather than reporting success. An engine whose cells fail three times in a row is put down for the rest of the session and named in the failure file, because a thousand cells that each take their own time to fail is a day of the machine spent learning one thing. The next sweep tries it again, since the usual reason an engine failed is something somebody has since fixed.
An estimate of how much longer a sweep has, from the runs it just did rather than from an estimate scaled off somebody else's silicon. It averages the last twenty, because a sweep walks from one thread up to sixteen and from pipeline one up to fifty and the cells are not the same size as each other, and it says nothing at all until there are a few, because an estimate from one run is a number with no information in it and people believe printed numbers.
sweep --dry-run, which prints the cells it would measure in the order it would measure them and writes nothing, and sweep --cache, which sweeps one engine rather than seven. Between them they are how a sweep is checked before a machine is given up for a week.
tools/provision/install.sh, which turns a fresh Ubuntu box into one that can run a sweep. It installs the build dependencies, a Rust toolchain, a .NET SDK for Garnet, memtier_benchmark and the seven cache servers, all at the versions pinned in tools/provision/versions.env, into the sibling directories config.jsonc already names. Every version it installs is written into every run file this harness produces, so the pins are part of the results rather than part of the setup, and two sweeps on two machines are comparable only if they agree. Running it again is the normal case: a checkout already there is fetched rather than cloned, and a binary already built at the pinned commit is left alone.
Every crate is published to crates.io, so cargo install cache-bench is the answer to how this gets onto a machine that is about to be measured. The libraries go up with the binary because a binary cannot be published without the crates it is built from, and because the file model, the statistics and the chart engine are each worth reading on their own.
A package job in CI that builds every crate from its own packaged tarball. A crate carries only what is inside its own directory, so an embedded file that reaches up out of the crate builds here and fails for everybody who installs it, and finding that out at tag time means a version on crates.io that cannot be replaced.
The release workflow publishes the crates last, after every gate has passed and after the GitHub release exists, because a release can be deleted and a version on crates.io cannot. It skips a crate whose version is already up, so finishing a half published release is a matter of running it again, and it waits out the rate limit on crates that have never been published rather than failing, which the first release will spend half an hour doing.
Changed
The golden fixtures moved from testdata/golden into crates/cb-core/golden, and every test in the workspace reads them through cb_core::golden rather than through a path that climbs out of its own crate. The fonts moved from assets/fonts into crates/cb-chart/assets/fonts for the same reason. This is what makes the crates publishable, and it is also what makes cache-bench verify carry its own evidence into the binary rather than depending on where it was built.
The command line crate is called cache-bench rather than cb-cli, since that is the name of the binary it has always produced and the name somebody types to install it. The directory moved to match.
config.jsonc looks for Garnet under net10.0 rather than net9.0. The pinned Garnet targets .NET 10 and pins that SDK in its own global.json, and the framework is part of the path its build writes to, so this line and GARNET_FRAMEWORK in the provisioner move together. The copy of the original's own config in the tests still says net9.0, because that file is theirs and quoting it wrongly would make the test prove nothing.