perf(bench): add hot-path harness — baseline relay, reassembly, and derived-state costs

[Lythaeon]

Description

Adds a first SOF-owned Criterion benchmark harness for public hot paths and records a validated VPS baseline for future performance work.

This matters because the repo already has strong architectural guidance around hot-path changes, but it did not yet have a repeatable SOF-side benchmark entry point for relay/cache, dataset reassembly, or derived-state dispatch. This PR makes performance work more reviewable and gives future optimizations a concrete baseline to compare against.

Changes

• crates/sof-observer/benches/hot_paths.rs

  • Added a Criterion harness for four SOF-owned public hot paths:
    • relay cache insert
    • relay cache range query
    • dataset reassembly
    • derived-state host dispatch
  • Kept the harness on public crate surfaces instead of introducing benchmark-only API holes.
  • Made the bench crate fully CI-clean under the workspace lint policy without crate-level allow escapes.

• crates/sof-observer/Cargo.toml

  • Added criterion as a dev-dependency.
  • Registered the hot_paths bench target.

• CONTRIBUTING.md

  • Documented the hot-path benchmark workflow:
    • compile-only smoke check
    • when to run the harness
    • when to validate on the reference VPS
    • how to present benchmark evidence in PRs

• docs/architecture/ard/0006-performance-and-hot-path-playbook.md

  • Updated the performance playbook to point contributors at the shared harness instead of one-off microbenchmarks.
  • Clarified that host class must be recorded when comparing results because multicore runtime behavior is scheduler/topology-sensitive.

Affected slices:

• relay
• reassembly
• framework::derived_state
• contributor/performance workflow docs

Cross-slice communication changes:

• None. This PR adds measurement coverage only; it does not change runtime behavior or introduce new cross-slice dependencies.

Migration requirements:

• None.

Motivation

Business motivation:

• SOF is now close enough to serious external use that future performance changes need repeatable evidence, not ad hoc local observations.
• A stable benchmark harness reduces review friction and makes host-class regressions easier to catch before they affect real adopters.

Technical motivation:

• The repo already has ARD guidance that hot-path changes require benchmark/profiling evidence.
• Until now, there was no shared SOF-side harness covering relay/cache, dataset reassembly, and derived-state dispatch.
• Derived-state dispatch in particular is a likely optimization target, so it is worth baselining early.

Alternative approaches considered:

• Adding benchmark-only public APIs:
  • Rejected. This would weaken the public surface just to support microbenchmarks.
• Only using live-network runtime telemetry:
  • Rejected as insufficient for this branch. Live telemetry is useful for end-to-end health, but not for isolating internal path costs.
• Allowing bench-only Clippy escapes:
  • Rejected. The harness was made lint-clean under the same workspace policy as production code.

Scope and impact

• Affected slices: relay, reassembly, framework::derived_state, contributor/performance docs
• Data/API changes: adds a new bench target only; no runtime API or wire-format changes
• Backward compatibility: fully backward compatible
• Performance impact: no runtime behavior changes; this adds measurement infrastructure only
• Security impact: none

Testing

• Unit tests
• Integration tests
• Manual verification
• Performance checks (if applicable)
• Security checks (if applicable)

Commands/results:

cargo bench -p sof --bench hot_paths --no-run
cargo make ci

Manual / VPS verification:

• Rebuilt the bench binary locally and copied it to the reference VPS (91.99.102.201).
• Ran a 5-minute Criterion pass on the VPS:

./hot_paths_bench --bench --warm-up-time 5 --measurement-time 70 --sample-size 30

VPS baseline results:

• relay_cache_insert/shared_cache_insert_512

  • time: [287.33 us 289.35 us 291.96 us]
  • throughput: [1.7536 Melem/s 1.7695 Melem/s 1.7819 Melem/s]
  • result: no significant change vs previous short baseline

• relay_cache_query/shared_cache_query_range_64

  • time: [611.80 ns 1.2489 us 2.2808 us]
  • throughput: [28.060 Melem/s 51.243 Melem/s 104.61 Melem/s]
  • result: improved vs previous short baseline after the cleaned fixture path

• dataset_reassembly/contiguous_slot_32

  • time: [3.4682 us 3.4790 us 3.4880 us]
  • throughput: [9.1742 Melem/s 9.1980 Melem/s 9.2266 Melem/s]
  • result: stable on the VPS

• derived_state_dispatch/slot_status_batch_64

  • time: [64.297 us 66.909 us 69.193 us]
  • throughput: [924.95 Kelem/s 956.52 Kelem/s 995.38 Kelem/s]
  • result: regression vs previous short baseline; remains the slowest and most host-sensitive measured path on this VPS class

Key takeaway:

• Relay-cache insert and dataset reassembly are stable on the reference VPS.
• Derived-state dispatch is the clearest next optimization target for future performance work.

Related issues and documentation

• Fixes: none
• Related: future perf workstreams on feat/soak-fault-hardening and feat/busy-poll-experiments
• Architecture docs: docs/architecture/README.md
• Relevant ARD/ADR:
  • docs/architecture/ard/0006-performance-and-hot-path-playbook.md
  • docs/architecture/adr/0004-static-dispatch-and-performance-layout.md
• Operations/runbook updates: none

Reviewer checklist

• Code follows project standards and architecture constraints
• Slice boundaries are respected (docs/architecture/ard/0003-slice-dependency-contracts.md)
• Tests added/updated and passing
• Documentation updated (README/docs/operations as needed)
• No undocumented breaking change
• Performance trade-offs documented where relevant
• Security considerations addressed where relevant

Additional notes

• This branch intentionally does not benchmark private dedupe internals yet. The first slice focuses on public SOF-owned hot paths without adding benchmark-only API surface.
• The VPS benchmark is a microbenchmark baseline, not a live-network ingest proof. Live runtime/queue telemetry remains the right source for end-to-end network validation.