Skip to content

v0.7.6

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 May 16:17
v0.7.6
2fb1781

What's new in v0.7.6

v0.7.6 hardens the Scaphandre and Redfish energy sources against silent over-attribution bugs of the v0.7.4 class, with two intentional config breaking changes (Scaphandre process_map and Redfish endpoints) and a new layered safety net that catches silent upstream renames at runtime, mirroring the Kepler net delivered in v0.7.5. Both process_map and endpoints are now closed-shape typed structs that reject unknown TOML fields at config load. The release adds runtime warn-once nets, a wire-conformance CI workflow that runs against the real upstream binaries (Kepler v0.11.4, Scaphandre v1.0.2, DMTF Redfish mockup), and an observation workflow that captures real Scaphandre output for follow-up parser design. No CLI surface change, no daemon wire protocol change, no report JSON schema change.

Scaphandre matcher upgraded to cmdline-aware (BREAKING)

The previous process_map flat-string form did not work against the real Scaphandre exporter. Scaphandre v1.0.2 emits exe as an absolute path (/usr/lib/jvm/temurin-25-jdk-amd64/bin/java), so a basename "java" never matched a single reading at runtime. Worse, multiple services sharing a runtime (JVM, .NET) collide on the same exe, the strict &p.exe == exe_name matcher attributed all readings to the first map entry and silently mis-attributed the rest. Captured empirically via the new observation workflow.

The new form is per-service typed:

[green.scaphandre.process_map."order-svc"]
exe_contains = "bin/java"
cmdline_contains = "order-svc.jar"

Substring containment on both labels. cmdline_contains is optional, for processes whose exe is already unique (native binaries). Multiple matches per service emit a warn-once latch keyed by service name, with debug-level follow-up on subsequent ambiguous ticks. #[serde(deny_unknown_fields)] on ProcessMatcher rejects typos like cmdline_containss at config load instead of silent over-attribution. Operators with [green.scaphandre] configured in v0.7.5 must rewrite the section, the legacy flat form is rejected with a serde type-mismatch error at load.

The ProcessPower parser now preserves the cmdline label, previously discarded. Scaphandre concatenates argv without separators (java -jar /tmp/svc-a.jar arrives as cmdline="java-jar/tmp/svc-a.jar"), substring matching must operate on this concatenated form, not on a POSIX command line with spaces.

Redfish typed schema per endpoint (BREAKING)

Endpoints are now per-chassis typed tables with a schema enum that selects the canonical JSON pointer at compile time. The legacy endpoints = { "id" = "url" } flat form and the top-level power_path knob are both removed:

[green.redfish.endpoints."chassis-legacy"]
url = "https://bmc-1/redfish/v1/Chassis/1/Power"
schema = "legacy_power"

[green.redfish.endpoints."chassis-modern"]
url = "https://bmc-2/redfish/v1/Chassis/2/EnvironmentMetrics"
schema = "environment_metrics"

legacy_power resolves the deprecated /PowerControl/0/PowerConsumedWatts. environment_metrics resolves the modern /PowerWatts/Reading introduced by DMTF Release 2020.4. Both schemas resolve to the same downstream redfish_bmc model tag, the choice is wire-shape only. A mixed fleet declares both schemas, one per chassis, without duplicating the surrounding section. Vendor OEM JSON pointers (HPE's Oem.Hpe.PowerSummary.Watts and similar) are no longer configurable, OEMs that publish wattage at a non-standard path must front the BMC with a reverse proxy that re-shapes the payload.

#[serde(deny_unknown_fields)] on both RedfishEndpoint and RedfishSection. Legacy flat-string endpoints, stale top-level power_path, and unknown schema variants all fail at config load with a serde error rather than silently degrading.

Scaphandre zero-sample warn-once net

A new track_zero_sample_streak in the Scaphandre scraper mirrors the Kepler analogue delivered in v0.7.5. After three consecutive HTTP-200 scrapes with zero parsed ProcessPower entries, the scraper emits one warn that points at the most common causes (silent upstream rename of the metric or the exe label, Scaphandre running with --no-procfs, host-only build with no per-process exposition). The RAPL-less case is explicitly excluded from the warn message because it produces a non-zero reading with power=0, a separate failure mode that deserves its own diagnostic.

scaphandre_last_scrape_age_seconds now climbs from boot on every failure tick (was set to 0 only on success, and frozen at the last good value otherwise). Parity with Kepler's kepler_last_scrape_age_seconds for Grafana alerting on rate(...).

Upstream wire-conformance CI

A new top-level workflow upstream-wire-conformance.yml runs three independent jobs against the real upstream binaries:

  • kepler-wire-conformance spins a kind cluster, applies the upstream v0.11.4 DaemonSet in fake-meter mode, port-forwards, and asserts both kepler_container_cpu_joules_total{container_name="..."} and kepler_process_cpu_joules_total{comm="..."} are present on the wire, then runs the perf-sentinel daemon against the live Kepler and asserts no zero-sample warn fires.
  • redfish-wire-conformance starts the DMTF mockup container, asserts both the legacy /PowerControl/0/PowerConsumedWatts and the modern /PowerWatts/Reading resolve to finite positive numbers, and runs the daemon with two chassis-ids one per schema to exercise the per-endpoint dispatch.
  • scaphandre-wire-conformance installs the upstream Scaphandre v1.0.2 deb in --vm mode against a fake powercap tree, spawns two co-located JVMs, configures process_map with cmdline_contains per service, and asserts the cmdline-aware matcher disambiguates without firing the ambiguity warn, plus a defensive assertion that the zero-sample warn never fires on a healthy run.

Three independent jobs, no needs: chain, so an upstream rename in one source cannot mask a regression in another. Triggered on push to main, on pull requests touching the relevant source paths, and weekly via cron. SHA-pinned upstream artifacts (Kepler image digest, DMTF mockup image digest, Scaphandre deb SHA256), the failure mode is loud (::error::) when an upstream rename slips through. Each job has a Checkout retry cascade (3 attempts) to defend against transient GitHub auth flakes.

A separate scaphandre-exe-observation.yml workflow runs manually via workflow_dispatch, builds 3 toy Java jars + 3 toy .NET assemblies + 1 native Rust binary, then dumps the full Scaphandre /metrics as a downloadable artifact and prints a five-question observation report. The data captured 2026-05-23 (PowerWatts.Reading = 374 on the DMTF mockup, Scaphandre v1.0.2 emits cmdline="java-jar/tmp/svc.jar") drove the cmdline-aware matcher and EnvironmentMetrics schema design.

Release tooling

A new scripts/release.sh is the canonical tag-and-push path. It runs every pre-flight check (clean working tree, signing identity, check-tag-version.sh, latest lab-validation freshness) before any destructive action, and aborts on the first failure. The release procedure docs (EN + FR) now point at it as the single entry point for the tag-and-push steps.

Documentation

  • docs/CONFIGURATION.md and its FR mirror: the [green.scaphandre] and [green.redfish] sections rewritten for the new TOML shapes, with side-by-side examples of multi-JVM Scaphandre attribution and mixed legacy + modern Redfish fleet.
  • docs/LIMITATIONS.md and its FR mirror: Scaphandre precision-bounds section documents v1.0.2 as the validated upstream reference (pinned by SHA256 of the upstream .deb artifact in the wire-conformance job). Redfish section gains methodology notes on the sensor-smoothing divergence between legacy_power (vendor-smoothed wattage, Dell iDRAC ~5s rolling, HPE iLO 1-5s) and environment_metrics (current-tick gauge), plus the EnergykWh.Reading cumulative-energy field that the modern schema exposes but perf-sentinel does not yet consume.
  • docs/METHODOLOGY.md and its FR mirror: new "Academic grounding" section citing the RAPL primary literature, the Scaphandre and Kepler software-meter literature, and the SPECpower / CCF dataset lineage that feeds the cloud_specpower path.
  • README.md: clarifies that perf-sentinel performs per-span attribution and exposes OTJAE (OpenTelemetry Java Auto-instrumentation Equivalent) detail in its outputs.

Tests

The diff adds 18 unit tests across the new code paths, plus adapts the existing tests to the new struct shapes:

  • ProcessPower parser covers cmdline preservation, argv-concatenation round-trip (java-jar/tmp/svc-a.jar), default-to-empty on absent label.
  • apply_scrape covers three-JVM disambiguation via cmdline_contains, exe-only matching when cmdline_contains is unset, ambiguous-matcher skip with warn-latch persistence across ticks and clear-on-clean-match.
  • RedfishSchema::json_pointer() dispatch table verified for both variants.
  • RedfishEndpoint and RedfishSection deserialization (rejection of legacy flat form, unknown schema variants, stale top-level power_path).
  • track_zero_sample_streak four edges identical to the Kepler test set: under threshold, at threshold, latch persistence, reset on non-empty scrape, plus a marker-stability test that anchors the literal CI grep target.

Why this is a patch and not a minor

Two operator-visible config breaking changes (Scaphandre process_map and Redfish endpoints), but the changed surface is opt-in: neither section is configured by default, and the previous Scaphandre form did not work against real Scaphandre output. Operators running v0.7.5 with [green.scaphandre] set were on the proxy fallback path anyway because of the basename-vs-fullpath mismatch documented above, so the rewrite they need to do to land on v0.7.6 also lights up the scaphandre_rapl model tag for the first time. The CLI subcommand surface, daemon HTTP routes, OTLP wire protocol, and report JSON schema are byte-for-byte identical to v0.7.5. The co2.model enum gains no new value, the existing scaphandre_rapl and redfish_bmc tags are preserved.

Verifying this release

# Binary integrity via SLSA Build L3 attestation
gh attestation verify perf-sentinel-linux-amd64 \
  --owner robintra --repo perf-sentinel

# A periodic disclosure produced by this binary
perf-sentinel verify-hash --report perf-sentinel-report.json \
  --expected-identity "https://github.com/robintra/perf-sentinel/.github/workflows/release.yml@refs/tags/v0.7.6" \
  --expected-issuer "https://token.actions.githubusercontent.com"

gh CLI 2.49 or newer required for gh attestation verify (unchanged from v0.7.2).

Full Changelog: v0.7.5...v0.7.6

Assets 7
Loading