Skip to content

bench/boundary: record the post-#261 numbers, and what this box can't measure - #266

Merged
lannbot merged 1 commit into
mainfrom
refresh-baseline
Sep 4, 2026
Merged

bench/boundary: record the post-#261 numbers, and what this box can't measure#266
lannbot merged 1 commit into
mainfrom
refresh-baseline

Conversation

@lannbot

@lannbot lannbot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

#263/#264/#265 moved the boundary numbers enough that the committed baseline now misleads — the compound-element rows read ~4-5x high, and the block recording them is still headed "pre-#261 optimization" with no "after" anywhere in the file.

Two dated blocks, not one rewritten one

The 2026-08-11 block is left exactly as it was. A dated baseline is a historical record; overwriting it destroys the before/after that makes the numbers mean anything.

The new block carries one table, deliberately

compound-element lanes (ns/element; n=10000; jco lane skipped):
shape         polyengine-node-callback      polyengine-node-jspi  polyengine-deno-callback
lift-ops                         677.8                     704.2                     867.1
lower-ops                        630.4                     661.4                     694.6

3.7x–5.6x fewer ns/element than the 2026-09-03 "before" table, depending on lane. This table is trustworthy: four independent runs across the day landed in the same band every time.

The calls-per-second table is not refreshed, and the file says why. This box cannot currently reproduce it — send immediate 0 on the node-jspi lane read 780,785/s, then 1,023,625/s, then 521,044/s across three runs whose code differed only by the changes under test. A 2x spread on effectively identical code is not a baseline; committing it would hand the next person chasing a regression a set of ghosts, and the README's own framing ("compare the same lane across commits on one box") is exactly the use that would break.

What is known is recorded as a delta rather than an absolute, from interleaved before/after pairs — medians of paired differences, not of arms — reproduced across two independent passes: send-sync +27%/+32%, send +22%/+28%, recv +34%/+31%, attributable to #265. The 2026-08-11 table stays the recorded absolute baseline, explicitly labelled as understating the current tree until a quiet box allows a real re-measurement.

Stream rows are untouched for the same reason, also stated. stream-sink at 256 KiB spans 2,900–10,800 MB/s across four interleaved runs with no consistent before/after sign — and none of #263/#264/#265 touch the stream<u8> bulk-copy path, so there is nothing to expect. I checked for a regression there specifically rather than assuming: four interleaved pairs against the pre-series commit, differences in both directions, no signal.

Also

The calls-per-second and stream tables padded lane columns to 22 characters — narrower than the longest lane name — so their headers ran together (polyengine-node-jspipolyengine-deno-callback). That is why the committed baseline block is unreadable in exactly that spot. Widened to 26, matching the compound table. Every table in this PR is literal tool output.

just check passes. Documentation and one sweep formatting fix; no runtime code, no version bump.

… measure

#263/#264/#265 moved the boundary numbers enough that the committed
baseline now misleads: the compound-element rows read ~4-5x high, and the
block recording them is still headed "pre-#261 optimization" with no
"after" anywhere.

Adds a 2026-09-04 block alongside the 2026-08-11 one rather than
overwriting it — a dated baseline is a historical record, and overwriting
it destroys the before/after that makes the numbers mean anything.

The new block carries the compound-element table and nothing else, on
purpose. This box cannot currently reproduce the calls-per-second table:
`send immediate 0` on the node-jspi lane read 780,785/s, then 1,023,625/s,
then 521,044/s across three runs whose code differed only by the changes
under test. Committing that would be noise with a date on it, and the
README's own framing — compare the same lane across commits on one box —
is precisely the use it would break. What is known instead is stated as a
delta from interleaved before/after pairs (medians of paired differences,
reproduced across two passes): send-sync +27%/+32%, send +22%/+28%, recv
+34%/+31%. The 2026-08-11 table stays the recorded absolute baseline,
labelled as understating the current tree.

Stream rows are untouched for the same reason and it is stated: stream-sink
at 256 KiB spans 2,900-10,800 MB/s across four interleaved runs with no
consistent sign, and none of the three PRs touch the stream<u8> bulk-copy
path.

Also: the calls-per-second and stream tables padded lane columns to 22
characters, narrower than the longest lane name, so their headers ran
together — which is why the committed baseline block is unreadable in
exactly that spot. Widened to 26, matching the compound table.
@lannbot
lannbot enabled auto-merge September 3, 2026 23:56
@lannbot
lannbot merged commit df2b455 into main Sep 4, 2026
4 checks passed
@lannbot
lannbot deleted the refresh-baseline branch September 4, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants