v0.5.15

What's new in v0.5.15

v0.5.15 fixes two bugs that surfaced from upstream operators while staging the v0.5.14 release on a real cluster.

The daemon's /metrics endpoint now produces valid OpenMetrics 1.0.0 output when exemplars are present. Pre-0.5.15 the handler advertised Content-Type: application/openmetrics-text; version=1.0.0 but the body was malformed against the spec. The mandatory # EOF end-of-exposition marker was missing, and exemplar annotations omitted the mandatory numeric value. A Prometheus server negotiating OpenMetrics text format read the body, failed at parse time, and reported up=0 even though scrape_samples_scraped > 0. The fix appends # EOF\n after the last metric line and rewrites the exemplar form from # {trace_id="..."} to # {trace_id="..."} 1.0 (the 1.0 is a spec-required placeholder that consumers ignore in practice, Grafana exemplar click-through reads only the trace_id label). The text/plain; version=0.0.4 path served when no exemplars are recorded is unchanged.

The 32-frame MAX_JSON_DEPTH cap is now enforced uniformly across all JSON ingest formats. Pre-0.5.15 the depth guard only ran on the Native arm of JsonIngest::ingest, leaving Jaeger and Zipkin payloads on serde_json's default 128-frame stack. The asymmetry let JSON-bomb attempts targeting either format bypass the project ceiling. The 0.5.14 review surfaced this as a pre-existing security finding from 0.5.13, 0.5.15 closes it. Both formats now reject input above 32 frames with the same payload nesting exceeds maximum depth of 32 error message as the Native and Report JSON paths.

A small follow-up touches the MetricsState::render doc-comment and drops a dead defensive newline-guard branch. The doc-comment used to advertise # Panics, but the function returns the body "# error encoding metrics\n" on encoder failure rather than panicking. The doc now accurately reflects this contract.

Compatibility check

VictoriaMetrics has parsed Prometheus exemplars in the form metric value # {labels} value since 2020 (shape foo 123 # {bar="baz"} 1 per their changelog), so the new # {trace_id="..."} 1.0 annotation and the # EOF marker are accepted by both Prometheus and vmagent. vmagent does not advertise application/openmetrics-text in its scrape Accept header (intentional, per VictoriaMetrics issue #9239), but the perf-sentinel daemon still selects the OpenMetrics content type based on has_exemplars(), the body remains parseable by vmagent in practice. VictoriaTraces is unaffected because the MAX_JSON_DEPTH guard sits in JsonIngest::ingest (local file or stdin path), not in the remote jaeger-query subcommand that fetches traces from VictoriaTraces or upstream Jaeger.

Fixed

• /metrics endpoint emits valid OpenMetrics 1.0.0 when exemplars are present. Two non-conformities corrected: the mandatory # EOF end-of-exposition marker is appended, and exemplar annotations now include the mandatory numeric value (# {trace_id="..."} 1.0 instead of # {trace_id="..."}). Symptom pre-0.5.15: a Prometheus server in OpenMetrics negotiation mode reported up=0 despite a successful TCP connection.
• MAX_JSON_DEPTH = 32 enforced uniformly across Jaeger, Zipkin and Native ingest paths. The depth check moved from the Native arm of JsonIngest::ingest to before the match detect_format dispatch, so all three formats see the same cap.
• MetricsState::render doc-comment updated to reflect the actual error-string return on encoder failure (was advertising # Panics, the function does not panic). A dead defensive newline guard was dropped, the post-condition is enforced by inject_exemplars itself.

Behavior

• Plain Prometheus text format (text/plain; version=0.0.4, served when no exemplars are recorded) is unchanged. The # EOF marker only appears on the OpenMetrics negotiated path where it is mandatory.
• The exemplar numeric value 1.0 is a constant dummy. The OpenMetrics 1.0 spec (section 5.1.10) requires a numeric value after the labels block, no consumer of the perf-sentinel /metrics surface today reads it (Grafana, Prometheus, Mimir, and Tempo exemplar tooling all key on the trace_id label).
• Defense-in-depth on the JSON-bomb surface: the depth guard previously protected analyze --input native.json, report --input <Report JSON>, and inspect --input <Report JSON> since 0.5.14. It now also protects analyze --input jaeger.json, analyze --input zipkin.json, and the OTLP / Jaeger / Zipkin fallback in JsonIngest::ingest.
• No daemon code path changed beyond MetricsState::render and MetricsState::inject_exemplars. The HTTP handler, the metric registry, and the exemplar recording helpers are unchanged.

Security

• Closes a pre-existing observation from the 0.5.13 security review. Jaeger and Zipkin ingest paths previously fell through to serde_json's 128-frame default rather than the project's tighter 32-frame ceiling. Tightens JSON-bomb defense uniformly across the ingest surface.

Install

Prebuilt binaries (Linux amd64 / arm64, macOS arm64, Windows amd64):

curl -LO https://github.com/robintra/perf-sentinel/releases/download/v0.5.15/perf-sentinel-linux-amd64
chmod +x perf-sentinel-linux-amd64
sudo mv perf-sentinel-linux-amd64 /usr/local/bin/perf-sentinel

Linux binaries are statically linked against musl and run on any distribution (Alpine, Debian, RHEL, Ubuntu any version) regardless of glibc version, and inside FROM scratch images.

From crates.io:

cargo install perf-sentinel --version 0.5.15

Docker:

docker run --rm -p 4317:4317 -p 4318:4318 \
  ghcr.io/robintra/perf-sentinel:0.5.15 watch --listen-address 0.0.0.0

Also available on Docker Hub: robintrassard/perf-sentinel:0.5.15.

Helm (chart 0.2.18 ships 0.5.15 as its appVersion default):

helm install perf-sentinel oci://ghcr.io/robintra/charts/perf-sentinel \
  --version 0.2.18 \
  --namespace observability --create-namespace

Verify the binary against SHA256SUMS.txt:

curl -LO https://github.com/robintra/perf-sentinel/releases/download/v0.5.15/SHA256SUMS.txt
sha256sum -c SHA256SUMS.txt --ignore-missing

Full diff: v0.5.14...v0.5.15