A minor release: one backwards-compatible feature, and a fix for a platform that did not compile at all.
Added
-
Bench::compare()— compares any number of alternatives against each other, paired and interleaved, instead of one run entirely before another.ankerl::nanobench::Bench().epochs(52).compare( "std::mt19937", [&] { ... }, // the baseline "sfc4", [&] { ... }, "romu", [&] { ... });
It prints an ordinary nanobench table with two extra columns — the ratio to the baseline and a confidence interval for it — and a summary line underneath.
The reason to want this:
relative()compares medians from runs separated in time, which measures the machine's drift as much as the code. nanobench's own test suite has a case where two identical workloads came out 38% apart on a CI runner, each reporting anerr%of 0.5.compare()runs one epoch of each per round, so a frequency ramp or a noisy neighbour hits every alternative alike and cancels out of the ratios — and what it reports is an uncertainty about the ratio, which is the number you wanted.Rounds are ordered as a randomly chosen cyclic Latin square, so every alternative occupies every position exactly once per block; the iteration count is calibrated once and shared, so epoch overhead amortizes identically; the interval comes from a sign test on the paired log-ratios, widened to keep the whole table at 95% rather than each row separately. Use more rounds than the default 11 —
epochs(52)costs about a tenth of a second per alternative. The tutorial page has a worked example and the reasoning behind each choice.Note that interleaving means each alternative runs with the others' cache and branch predictor state. That is usually the more honest answer to "which should I ship", but it is a different measurement from running one alone — use
run()for that.
Fixed
- nanobench did not compile for any Android ABI (#167, reported in microsoft/vcpkg#53422). bionic declares
ioctl()twice — takingintand takingunsigned— so that either signedness of a request constant compiles without a cast. That makes&::ioctlan overload set, and the deduction added in v4.4.0 to tell glibc'sunsigned longapart from musl'sintcould deduce nothing from it. bionic is now handled explicitly. - The warmup loop tripped
-fsanitize=integer.
Internal
- A new
Android NDK (bionic, cross-compile)CI leg builds all four ABIs at C++11 and C++17, so bionic joins glibc and musl as a libc that CI actually compiles for. The Android break was invisible to every existing leg; it cannot happen quietly again. - The test suite grew 122 → 145 cases.
Full changelog: v4.5.0...v4.6.0