What's new in v0.10.0
v0.10.0 is about what a report leaves out. A span carrying an empty statement was read as a query with no name and became a finding advising the team to cache it, a snapshot exported by the daemon carried findings with no way to see what happened around them, and a list ranked by unitary severity buried the cheap problem repeated two hundred times under the expensive one seen twice. Each of those is now stated rather than silently dropped, and the correlation confidence that could read 150% became a number that means what its field documents.
A blank statement was read as a query with no name
Instrumentations do emit db.statement with an empty value, a redacting layer that keeps the attribute. Only the dd-trace fallback path rejected blanks: the OTel path took "" at face value, produced a SQL event with an empty target, normalized it to an empty template, and grouped every blank span on the endpoint into one redundant_sql advising the team to cache an operation with no name.
Found on a real capture where those artefacts were 10 of 205 findings and, more to the point, all three of the run's only non-info findings. The entire warning-level signal of that run was noise.
The span is now the missing_db_statement gap it always was, which is what the retention metrics and the min_usable_span_ratio gate rule exist to surface. A blank legacy db.statement also stops shadowing a populated stable db.query.text. Blank-checked rather than trimmed, so a non-blank statement keeps its exact bytes and its acknowledgment signature.
A daemon snapshot carries its span trees
The correlation window drops a trace's spans a few seconds after it completes, which is why query explain <trace_id> only answers while the trace is still live. A report exported afterwards showed an empty Explain tab for every finding, and the message blamed the embed cap, sending the reader to tune a flag that changes nothing.
The daemon now keeps the masked spans of the traces its retained findings point at, in a ring buffer sized by [daemon] max_retained_traces (default 50, 0 to keep none), and exports them under the report's new embedded_traces. report --input <snapshot>, explain --input <snapshot> and the TUI all draw from them.
Only masked fields travel, the normalized template and never the raw statement, the same shape the HTML sink has always embedded. The export is capped in bytes to half the shared client fetch limit, so a busy daemon cannot grow it past what query monitor and the TUI can read, and a tree too large for the remaining budget is skipped rather than stopping the whole set. A trace flushed twice by the window merges both flushes' spans, since each flush carries only its own and a finding from either can point at the other's.
A finding whose trace has aged out is exported as before, and the empty-tree message now names the real knob for each case: the retention buffer on a snapshot, the embed cap on a batch render, a spanless Report JSON otherwise.
Severity says how bad, not how much
Severity measures the intensity of one request, never the total cost. An info-level duplicate query detected on two hundred traces wastes more I/O than a critical seen three times, and a list ordered by unitary severity buried it.
Every triage surface can now rank by aggregate impact, the sum of a signature's avoidable operations across every detection the report holds. The dashboard gained a sort control picking the primary key, severity or impact, and its direction, with the other axis as the automatic descending tie-break, so "highest total waste first, worst severity among equals" is one click. analyze and query findings gained --sort severity|impact with the same semantics, the TUI's Inspect list cycles its order with s, and the Grafana findings table gained an Est. impact (ops) column.
analyze also folds its output by signature: the first detection prints in full with a Recurrence: line carrying the trace count and the total avoidable ops, the repeats compact to one line each carrying only their trace id. On the capture above that turns 205 blocks into 26. A detection whose severity outranks what already printed keeps its own full block, since signatures ignore severity and folding on first-seen alone would hide a critical behind a warning's stub.
Severity stays untouched everywhere it is load-bearing: the quality gate, SARIF levels and acknowledgment folding read the same values as before.
Cross-trace confidence became a number that means something
A live report showed conf 150% on a pair whose field documents a ratio in [0, 1]. Two counting bugs stacked. The correlator scored one co-occurrence per (source occurrence, target finding) couple, so a source seen twice followed by three targets counted six against a denominator of two. And the pair counter accumulated for the pair's whole life while the source total only spanned the rolling window, so any long-lived pair would eventually saturate whatever its real strength.
Each source occurrence now counts once per pair, tracked by a monotonic sequence number since every finding in a batch shares one timestamp and timestamps could not separate them. The count is window-scoped through two aligned half-window buckets, rotated on every batch rather than only when a pair records, so a pair whose traffic stopped decays instead of reading as certain on frozen counts. co_occurrence_count finally means what its doc says, and min_confidence filters mature pairs again. The sample trace id also refreshes on every matching target rather than freezing on the oldest one, which had usually expired by the time an operator jumped through it.
The report hands over what it was hiding
The HTML report carried a trace_id all along, in the embedded payload, but no view ever showed it whole: the breadcrumb truncated it to 12 characters and the CSV export omitted the field. The one thing an operator needs to go look at a trace elsewhere was the one thing the report would not give. The breadcrumb keeps a short form, since 32 hex characters would crowd out the service and endpoint beside it, and gains a Copy trace_id button plus a title attribute carrying the full value. trace_id is a CSV column.
The finding detail panel stopped hiding everything behind a missing span tree. Only the tree needs the trace, but the whole panel was skipped without one, so a snapshot showed no suggested fix, no pg_stat cross-reference, no source location and no first/last seen, on findings that carried all four. The suggestion field also reached the HTML for the first time: every other sink printed it, the dashboard never did.
The detail foot says how many traces a problem was detected in and over which window, with the date joining the time when the window straddles days. A forty-column UPDATE no longer runs off the right edge or widens the page: the template breaks after commas, a boundary a reader of SQL expects, and the tree row ellipsizes with the full operation on its tooltip.
Absent stops looking like healthy
Nothing on /metrics could tell a missing energy backend from a working one. Every energy gauge is pre-registered at zero whether or not the backend exists, and a successful scrape sets last_scrape_age_seconds back to zero, so "not configured" and "configured and perfectly healthy" were the same flat line.
perf_sentinel_energy_backend_configured{backend} says which measured-energy backends are actually configured. The daemon publishes it once at startup from the same flags /api/energy reports, and the label set is the five compile-time backend names, so cardinality stays bounded. [daemon] max_retained_traces is exposed on /api/config and listed in query monitor's Config tab, beside the sibling that was already there.
Upgrade impact
Existing acknowledgments stay valid: no signature changed. A report produced by 0.9.28 still loads, and the archived per-window JSON the daemon writes stays byte-identical, so a disclosure content_hash computed on 0.9.28 verifies unchanged here.
[daemon] max_retained_traces defaults to 50 and costs memory in proportion to max_events_per_trace. It is neutralized automatically, with a startup warning naming the cause, when api_enabled = false or max_retained_findings = 0, since only /api/export/report reads it. Set it to 0 to keep none.
One behavioral change to expect on a run that ingests OTLP: spans carrying a blank db.statement now count as an instrumentation gap rather than producing an empty-template redundant_sql. On an affected capture the finding count drops and analysis.ingest reports the gap instead.
Why this is a minor and not a patch
perf-sentinel-core is published on crates.io and several public structs gained fields, and quality_gate::evaluate takes a fourth parameter. A downstream crate pinned to "0.9" would have broken on a patch number that promises it cannot. Under SemVer's 0.x rules the minor position carries that meaning, so the bump is the honest one.
The growable public structs are now #[non_exhaustive], so the next added field is a minor bump rather than another breaking change. External crates construct them from Default::default() and set the public fields one by one.
Validation
The full Rust suite passes (2998 tests), clippy is clean at -D warnings across all targets, cargo fmt --check passes, and the default and --no-default-features builds both pass. The browser suite covering the HTML dashboard passes.
The simulation lab validated the release: the findings gate at 10/10, plus a targeted check of every surface this release touches. A 600-second soak at 60 traces per second, 36,000 traces and 334,000 spans, held RSS drift at -4.1% with no shedding and no restart, exercising the new trace retention at its default cap. The export was verified to carry no raw statement, and the archived windows to carry no embedded_traces, the two properties a disclosure depends on. The scenario sweep was scoped to those surfaces rather than run in full, which the ledger entry records.
Verifying this release
# Binary integrity via SLSA Build L3 attestation
gh attestation verify perf-sentinel-linux-amd64 \
--repo robintra/perf-sentinel
# A periodic disclosure produced by this binary
perf-sentinel verify-hash --report perf-sentinel-report.json \
--expected-identity "https://github.com/robintra/perf-sentinel/.github/workflows/release.yml@refs/tags/v0.10.0" \
--expected-issuer "https://token.actions.githubusercontent.com" \
--verify-binary ./perf-sentinel-linux-amd64gh CLI 2.49 or newer required for gh attestation verify.
Full Changelog: v0.9.28...v0.10.0