What's new in v0.9.28
v0.9.28 is about three silences. An OTel Collector left at its defaults exported into the void and nothing in perf-sentinel said so, an acknowledgment kept hiding a problem long after it was fixed and nothing said so either, and a daemon that refused to start named the file it could not read without ever naming the reason. Each of those becomes something a reader can act on, and two HTML report fixes ride along.
The version skips 0.9.27, which was a chart-only release. The application line goes from 0.9.26 to 0.9.28 and the two are aligned again.
A collector at its defaults was sending into the void
The OTLP gRPC listener never declared an accepted encoding, so tonic answered every compressed export with Content is compressed with `gzip` which isn't supported, which the Collector logs under rpc error: code = Unimplemented. That status is permanent, so the Collector dropped each batch without retrying. The OTLP exporter compresses with gzip by default, which makes this the nominal path rather than an exotic one: a collector wired by the documented recipe sent nothing through.
Nothing on the perf-sentinel side reported it. The pod stayed ready, /health kept answering, and every /metrics counter stayed at zero, which reads as an idle daemon rather than a rejecting one. The failure existed only in the Collector's own logs, and the instrumentation guide made it worse by stating that no compression: none override was needed, a claim that only ever held for the HTTP endpoint.
The gRPC listener now accepts gzip and deflate. The HTTP endpoint, which has decompressed gzip since 0.5.5, gains deflate as well, so both endpoints accept the same encodings. snappy and zstd remain refused on both: a Collector only sends them when explicitly configured, and zstd would pull C bindings into a static musl binary. The guide now says which encodings are accepted, and quotes the tonic message exactly, backticks included, since that string is the only diagnostic an operator has on an affected version.
The decode size cap keeps applying to the decompressed message, so [daemon] max_payload_size still bounds a compressed payload and no window opens for a compression bomb. That property is now pinned by a test rather than by a comment, and the test asserts both halves: a message over the cap once decoded, under it on the wire. Without the second assertion the refusal could come from the frame-length check and prove nothing.
perf-sentinel capture, which receives from the same collectors, gets the same two encodings. It also gains the request timeout, the stream cap and the global concurrency limit the daemon listener already carried, because a compressed export now expands to the decode cap from a fraction of that on the wire, so the ceiling has to come from a request count rather than from traffic. Both listeners now mount their service through one shared function, so the encodings and the cap cannot drift apart on one transport.
An acknowledgment says when it has outlived its problem
.perf-sentinel-acknowledgments.toml was write-mostly. Nothing ever reported that an entry had become dead weight, so a finding accepted once stayed accepted long after it was fixed, and the list of accepted problems only ever grew.
Worse, no baseline comparison could close that loop. Acknowledging a finding removes it from a baseline produced with filtering on, so the pull request that genuinely fixes it later has nothing left to make disappear and its diff shows no resolution at all.
An active entry that suppressed nothing in a run is now reported under the stable warning kind unmatched_acknowledgment, in the CLI output, in warning_details of the JSON, and in the diff output. Two optional entry fields, service and source_endpoint, copied verbatim from the finding, let the message check the run's per-endpoint I/O counts and say which of two things happened: an endpoint that did I/O without the finding firing reads as fixed and the entry as removable, an endpoint that emitted no I/O proves nothing and the entry stays. The counts cannot tell an unexercised endpoint from a fix that removed the I/O outright, and the message says so rather than guessing. An entry without the two fields keeps both readings, since an opaque signature cannot separate them and claiming a fix would invite deleting an entry that still protects the gate.
Two boundaries keep the signal honest. It is derived only from a fresh analysis of traces: re-rendering a saved report or a daemon snapshot emits nothing, because such a report may already be ack-filtered, where "matched nothing" would mean "consumed on the previous pass", and its I/O counts describe another run entirely. And an expired entry is never reported, since it is inactive and its finding is already back in the gate.
A daemon that refuses to start says why
The daemon's startup errors name the resource that failed and carry the cause as an error source, which nothing printed. failed to load acknowledgments TOML at '<path>' therefore looked identical whether the file was missing, unreadable, or refused.
The refusal case is the one that matters. The loader declines to follow a symlink, a hardening against a link pointing at a sensitive file, and a Kubernetes ConfigMap projects every key as exactly that kind of symlink. Mounting a CI ack baseline the obvious way produced a CrashLoopBackOff whose message named no cause, on an image with no shell to investigate from. The chain is now walked and printed as caused by: lines. Behaviour is unchanged, only the diagnosis, and the deployment guide documents the subPath mount that materialises a real file.
Two HTML report fixes ride along
Both landed just after 0.9.26 was cut and ship here for the first time. The correlation cards wrap instead of overflowing their panel on a narrow viewport: the meta column beside each card cannot shrink, so below roughly a phone width it pushed the card past the panel's right edge rather than reflowing. And the diff zones are readable in dark mode, where the severity backgrounds they share with the findings list read far too bright, because a zone tints a whole column rather than a badge. Light mode needed no correction and gets none.
Upgrade impact
Accepting more encodings is additive. A collector already sending uncompressed, or carrying the compression: none workaround, keeps working untouched. The workaround can be removed to recover the bandwidth, and nothing forces it.
No finding signature changes, so no acknowledgment needs re-capturing anywhere. The two new TOML fields are optional and ignored by any binary that does not know them, so one ack file can be shared with an older daemon.
Two additive fields appear in output. DiffReport gains warning_details, absent when empty, so a diff consumer keyed on new_findings is unaffected. Reports already carried warning_details, which now may hold the new kind.
One sizing note for a memory-capped pod. max_payload_size used to cap buffers a client had genuinely uploaded, and it now caps what a much smaller upload expands to. On a container with a tight memory limit, [daemon] memory_high_water_pct is the setting to raise off its default before accepting compressed ingest.
Why this is a patch and not a minor
The pre-1.0 scheme bumps the third number for features too. Nothing is removed, renamed or given a different meaning. The two acknowledgment fields are optional, the diff field is additive and skipped when empty, and no public type changes shape. A pipeline that sends uncompressed spans and reads no warnings cannot tell 0.9.26 from 0.9.28.
Validation
Validated in the external simulation lab across two runs, both PASS, both recorded in release-gate/lab-validations.txt with their scope written into the ledger.
The first covers the compressed ingest. Every verdict carries an A/B against the published 0.9.26 image: gzipped gRPC ingests 360 spans where 0.9.26 drops 360 with the Unimplemented in the collector's log, reproduced in a real cluster with the lab's own collector repinned to the gRPC port, where the 0.9.26 span counter did not move at all under the same burst while the 0.9.28 one did and produced 20 N+1 findings. Deflate is exercised end to end on both transports, and zstd and snappy are confirmed still refused, which is what the guide now claims. The lab had never caught the original bug because it had a gRPC path and a compressed path but never both at once, and it now carries a scenario that crosses them.
The second covers the acknowledgment path, on real artefacts rather than fixtures: the full life cycle from an active entry suppressing a finding to a fixed one reporting as removable, the guard that no warning is derived from a pre-computed report checked against a daemon export and a replayed analysis, and a positive control committed beside both silence checks, since silence is free on any version that never emits the warning. Against 0.9.26 the scenario fails five legs including that control.
Two limits are stated rather than buried. The scenario scope was deliberately narrowed to 37 of 67 scenarios after auditing the dependency bumps this release carries, on the grounds that the only one crossing a semver boundary is a compile-time proc-macro parser that never reaches the binary. And the saturation ramp does not exercise the amplification corner, a small compressed frame expanding toward the 16 MiB ceiling, since it batches 50 traces per export. The ramp that was run shows no degradation attributable to this release: 0.9.26 and 0.9.28 reach the container memory limit at statistically the same point.
The full Rust suite passes (2934 tests), clippy is clean at -D warnings in the default configuration and with the daemon feature, cargo fmt --check passes, and the default and --no-default-features builds both pass.
Verifying this release
# Binary integrity via SLSA Build L3 attestation
gh attestation verify perf-sentinel-linux-amd64 \
--repo robintra/perf-sentinel
# A periodic disclosure produced by this binary
perf-sentinel verify-hash --report perf-sentinel-report.json \
--expected-identity "https://github.com/robintra/perf-sentinel/.github/workflows/release.yml@refs/tags/v0.9.28" \
--expected-issuer "https://token.actions.githubusercontent.com" \
--verify-binary ./perf-sentinel-linux-amd64gh CLI 2.49 or newer required for gh attestation verify.
Full Changelog: v0.9.26...v0.9.28