Repository navigation
v0.31.0
-
Coding-agent integration (issue #15): two new commands wire perfscale into
AI coding agents.perfscale init-skills [--loop claude|agents] [-g]
installs embedded skill folders (perfscale-cliwith YAML/actions/
load-shapes/thresholds/metrics/reporters/protocols references,
perfscale-library-sdk,perfscale-migration-k6) into./.claude/skills,
./.agents/skills, or the home directory with-g; re-running overwrites.
perfscale init-agents [--loop claude|codex|copilot|opencode|vscode| vscode-legacy] [-c config] [--prompts](default loop:claude) generates
theperfscale-test-{planner,generator,healer}agent definitions per loop,
registers the@perfscale/mcpserver in the loop's MCP config
(npx -y @perfscale/mcp), and initializes the repo:specs/README.md, a
found-or-createdtests/seed.yaml, and a minimal config.--prompts
installs theperfscale-{plan,generate,heal,threshold}prompt templates —
thethresholdprompt replaces Playwright'scoveragewith a HOLD /
NOT HOLD verdict at a confirmed full-load run. Agents verify with small
1-VU runs by default, and the healer never editsthresholds:. See
docs/agents.md. -
Streaming reporters (issue #12):
report.reporterscan now stream live,
cumulative metrics during the run to pluggable sinks —std/report/serve@v1
(the historicalreport.urlpath as a reporter),std/report/prometheus-rw@v1
(Snappy + protobuf remote write; classic_bucket{le}histograms by default,
native histograms opt-in viawith.histogram: native) and
std/report/otel@v1(OTLP/gRPC with cumulative exponential histograms).
Engine-owned keys sit next touse::during_run,interval_ms,labels
andmax_series(default 10000, deterministic drop counted in
perfscale_report_dropped_series_total{reporter});max_samplesmoved out
ofwith:(the legacy key still reads). Labels merge step > reporter > run,
interpolate${{ inputs.* }}/${{ env.* }}once before the run, and steps
accept their ownlabels:;run_id/workerare engine-set and rejected in
user config. No client-side quantile series in the new exporters — backends
compute correct cross-worker quantiles from the buckets
(histogram_quantile(0.95, sum by (le) (rate(http_req_duration_bucket[1m])))).
Each reporter gets an independent bounded queue (with.max_pending), batched
delivery (with.batch_size, never exceeded — oversized snapshots split in
order), retry with exponential backoff (with.retry,Retry-After
honored, other 4xx poisoned) and a synchronous final flush bounded by
with.final_timeout(default 30s) that coalesces the backlog to the
terminal cumulative snapshot; overflow drops the OLDEST snapshot (counted in
perfscale_report_dropped_snapshots_total{reporter}), and a slow or dead
backend can never block the VUs, another reporter, or fail the run. The
delivery counters are per-run, registered as thereportmetric family and
exported to each backend asperfscale_report_*_total{reporter="…"}. Aduring_run: trueon a final-only reporter (json/junit/html/wasm) is
now a lint/run error. -
Benchmarks: a
renaissancesuite (in the default suites and the daily CI
run) executes the official Renaissance MIT v0.16.2 jar
(renaissance-mit-0.16.2.jar, sha256-pinned and cached by CI under
target/renaissance/) viascripts/bench_renaissance.sh— default
workloadfinagle-http(RENAISSANCE_BENCHMARKS), bounded to
RENAISSANCE_RUN_SECONDS(default 60) measured seconds per benchmark
through the harness's own--run-seconds. The report and results JSON
record benchmark operation throughput (ops/s, never HTTP RPS) and
per-operation time plus CPU/RSS/duration; a missing/invalid jar, a Java
failure or empty results fail the run rather than reporting fake numbers.
Console log and native JSON are preserved underrenaissance-artifacts/
and uploaded with the bench artifact;bench_compare.pycompares only
same-benchmark rows. -
Benchmarks: new opt-in
specjbbsuite (SUITES=specjbb ./scripts/bench.sh)
runs a caller-supplied commercial SPECjbb2015 kit (SPECJBB_HOME/
SPECJBB_JAR, plusSPECJBB_JAVA/SPECJBB_MODE, defaultCOMPOSITE)
via the officialjava -jar specjbb2015.jar -m <MODE>— never bundled,
never downloaded, never shortened — and reports native max-jOPS /
critical-jOPS (Java operations/s, not HTTP RPS) with mode, duration, CPU
and peak RSS in the report and results JSON. It is excluded from the
default suite list and CI; with an invalid kit path the bench fails loudly
instead of reporting a fake success. Console log and the SPEC report tree
are preserved as artifacts. -
Pub/Sub drivers can report asynchronous delivery failures through
PubSubOutcome.errors. The engine adds them to the synchronous failure flag
inpubsub_errors, so Kafka delivery errors, MQTT ack timeouts and Redis
pipeline error replies remain visible withflush: none; the counter can
exceed 1 per exchange without failing the step. -
k6-style mid-run threshold abort plus failure diagnostics: an
after:
std/thresholds@v1gate withabort_on_fail: trueis now monitored while
the VUs run (re-evaluated every 250 ms against the metrics collected so
far), and the first violatedseverity: failgate stops all load schedules
early — between steps, like Ctrl-C — then runs the usualafter:teardown,
reporters and summary (theafter:evaluation stays the single recorded
gate result). The monitor starts only after a successfulbefore:and never
records intermediate results; a normal finish or Ctrl-C prints no abort
line. Addinghttp_debug: error(valid only withabort_on_fail: true;
lint/run reject other combinations) emits one secret-masked
http_debug: {…}JSON line per failed step action and failedcheck:,
carrying the VU id, iteration, interpolated request params (payload
included) and the response output (body included) — k6 #6618-style
diagnostics with zero output for successful exchanges.
What's Changed
- fix(release): idempotent npm publish, re-runnable for an existing tag by @vitalics in #28
- feat: streaming metric reporters and JVM benchmark suites by @vitalics in #29
- feat: report asynchronous pubsub errors by @vitalics in #30
- feat: add agent and skill initialization commands by @vitalics in #32
- feat: add aborting thresholds and failure diagnostics by @vitalics in #31
- chore(release): v0.31.0 by @vitalics in #33
Full Changelog: v0.30.0...v0.31.0