Skip to content

v2.4.2

Latest

Choose a tag to compare

@github-actions github-actions released this 31 Aug 08:54
63d3452

A bug-fix release, and the bug is a number: under a ramp, achieved_rate
reported the wrong thing. It was requests / duration_s in every mode, which
for -R A:B is the midpoint of the offered range — a figure that describes no
part of the run. -R100:1000 reported ~550 whether the server held 1000 all the
way up or fell over at 200 (#69).

The rate a ramp exists to find is now what achieved_rate reports. Nothing else
about zrk's interface changes: no new flags, no fields renamed or removed.

The number a ramp was run to find

Under -R A:B, achieved_rate is now the run's tail — the throughput over
the last --interval — and bytes_per_sec covers the same window, so the two
stay proportional and still divide out to bytes-per-request. Constant load and
--closed (which cannot ramp) keep the whole-run average, where it was never
misleading. The average remains requests / duration_s, both of which are still
in the report.

rate_ratio follows, and divides by the load offered across that window —
never by target_rate_end. This matters more than it sounds. achieved_rate is
an average over a window during which the ramp keeps climbing, while
target_rate_end is the schedule at the final instant; the two differ by half a
window of slope, and dividing one by the other books a ramp that missed nothing
as short by exactly that much. On -R100:1000 -d6s with a 1s window it is the
difference between 0.9999 and 0.925 for the same run.

Both rows below come from the same run — 2.4.1's figures are recomputed from
requests / duration_s, which is what it printed — so what separates them is
the definition and not the machine.

Against a loopback server it serves comfortably, -c 8 -d 6s -R100:1000,
3300 requests over 6.000s:

achieved_rate rate_ratio
2.4.1 550.00 1.0000
2.4.2 924.90 0.9999

Both ratios are right — the old one held the average against the ramp's own
average — but only the new rate answers "what did it sustain". The difference is
sharper where the server cannot keep up. -c 64 -d 5s -R1000:200000, 247,839
requests over 5.001s:

achieved_rate rate_ratio
2.4.1 49557.89 0.4931
2.4.2 55869.03 0.3102

55869 req/s is the max sustained rate, and 0.31 is how far up the ramp it got.
The 2.4.1 pair says neither: 49558 is an average over a range that started at
1000, and 0.49 is that average over the range's own midpoint. Read alone, 0.49
suggests a server that managed about half the offered load throughout; it in
fact kept the schedule perfectly for the first stretch and then flattened.

Until now this number was only recoverable from the last --timeseries row,
which meant writing a file to read one figure out of it. That still works, and
achieved_rate_end below makes it unnecessary.

The panel was red on a ramp it should not have been

The same arithmetic was in the live dashboard, and worse there: it compared a
windowed achieved against the schedule at the instant. Under a ramp those
differ by half a window of slope by construction, so a target keeping its
schedule perfectly still showed achieved in red — not only on the final frame,
but for as long as that gap exceeded the 5% the threshold allows.

The gap is fixed while the offered rate climbs, so it bites hardest early. At
-R100:1000 with a 1s window it is 15 req/s over 30s and 110 req/s over 4s:
the 30s ramp read red from the 2s warm-up guard until the schedule passed
300 req/s about seven seconds in, and the 4s ramp — where 110 never falls under
5% of an offered rate that tops out at 1000 — read red from the guard to the
final frame.

Both sides of that comparison now come from the same window, so a kept ramp
reads offered 888 req/s achieved 888 req/s and stays uncoloured, while a
target that genuinely falls behind still goes red. The final frame also stopped
swapping in whole-run averages, which used to make the transfer figure jump on
the last repaint.

The time series had it too

--timeseries rows carried target_rate sampled at the instant the window
closed, next to an achieved_rate averaged across it. Piped into the README's
jplot achieved_rate+target_rate, a run that missed nothing drew a permanent
gap between the two lines. The rows now offer the schedule averaged across the
window, which is what docs/output.md had described all along:

t=1.0 target=  175.0 achieved=  174.9      t=4.0 target=  625.0 achieved=  625.0
t=2.0 target=  325.1 achieved=  326.0      t=5.0 target=  775.0 achieved=  775.0
t=3.0 target=  475.0 achieved=  474.1      t=6.0 target=  925.0 achieved=  924.9

Where a window actually sat

A window closes at the last progress row, which is not duration_s: the two
differ by the fleet join, and by most of an --interval when a signal cuts the
run between rows, since an interrupt raises no row of its own. Reading a ramp's
schedule at duration_s therefore judges the window further up the ramp than it
ran.

A -R100:1000 -d60s --interval 5s run interrupted 4s after its last row was
scored 4s too high — 0.86 for a ramp that missed nothing. It now reports
rate_ratio_end 0.9998, with end_window_s 5.000 against a duration_s of
24.009.

New JSON fields

Additive, all of them:

field what it is
achieved_rate_end always the tail, ramp or not, so a consumer can read one key without first working out whether -R A:B was passed
rate_ratio_end that rate over the load offered during its window
end_window_s how long the window was
target_rate_end top level: the offered rate the ramp climbed to
config.target_rate_end the -R endpoints as given — the config section carried only the start
config.interval_s the stats window, and so the basis for everything above

Under a ramp achieved_rate_end and rate_ratio_end are the same two numbers
as the top-level pair. Under constant load they are the last window rather than
the whole run, which is how a server that degraded partway through a run shows
up at all.

Embedders get end_rate, end_bytes_per_sec, end_window_s and
end_window_at_s on runner.Report; report.writeJson takes them as a
report.Run rather than a tail of positional parameters.

Upgrading

Every field emitted by 2.4.1 keeps its name and type. Two change meaning,
and only under -R A:B:

  • achieved_rate — the tail, not the whole-run average
  • bytes_per_sec — the same window, for the same reason

A harness that ramps and stores achieved_rate as a throughput figure wants the
new number; that is the fix. A harness that wants the old one should compute
requests / duration_s, which is exactly what 2.4.1 reported and both fields
are still there. Constant-rate and --closed runs are unaffected: their
achieved_rate is the whole-run average as before.

--timeseries rows keep every field. target_rate shifts by half a window of
slope under a ramp and is unchanged under constant load.

Verified

285 tests pass and zig fmt --check is clean. Eight tests are new, covering the
ramp tail, the constant-load path, an interrupted run's window anchor, a
report.Run built with no window, the time-series row's offered rate, the panel
not reddening on a kept ramp, and the window arithmetic itself.

Live runs against a loopback server: ramp, saturating ramp, constant rate,
--closed, a duration that is not a whole number of intervals, a real SIGINT
mid-window, and the dashboard under a pty. --version reports 2.4.2 from the
built binary.