Skip to content

test: add criterion benchmark harness (#25)#40

Merged
joeldsouzax merged 3 commits into
mainfrom
perf/p0.1-benchmark-harness
Jun 30, 2026
Merged

test: add criterion benchmark harness (#25)#40
joeldsouzax merged 3 commits into
mainfrom
perf/p0.1-benchmark-harness

Conversation

@joeldsouzax

@joeldsouzax joeldsouzax commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Closes #25 — Phase 0 (P0.1) instrument.

Why

Performance is the project's paramount goal, but there was no way to measure it — no benches/, no bench dependency. Every later perf card (P1.x) depends on being able to answer "is this faster?". This adds a criterion-based harness over the realistic hot paths.

What

  • criterion 0.8.2 dev-dependency, default-features = false (drops rayon/plotters) to keep the cargo deny/cargo audit surface minimal. Three [[bench]] targets gated required-features = ["stream"].
  • benches/matter.rs — encode (matter_to_qb64, fixed codes) + decode (from_qualified_base64, fixed and variable-size StrB64_L0) + qb64qb2 conversion.
  • benches/counter.rs — counter encode (v1 + auto-big) + counter-led group parse (parse_group).
  • benches/stream.rs — full multi-primitive attachment-stream parse via groups().
  • README "Benchmarks" section documenting how to run them.

Design notes

  • Benches are std-only and feature-gated, so the no_std/WASM builds are untouchedcesr-wasm and cesr-nostd checks stay green.
  • The flake runs clippy --all-targets --all-features, so bench code obeys the crate's full deny-set (no unwrap/expect/panic/as): operations are benchmarked by black_box-ing the returned Result, and pre-built fixtures are guarded with if let Ok(..).
  • Fixtures are real qb64 vectors from src/core/matter/test_vectors.rs, so they parse without contrivance.

Research (per the strategy card's standing rule)

Tool choice (criterion vs divan/iai-callgrind/tango) and the CI-gate analysis (CodSpeed deferred) are captured with primary-source citations in the #25 research comment. The deterministic CI perf-gate (CodSpeed) is a deliberate follow-up.

Verification

nix flake check green (all 16 checks: clippy, fmt, taplo, audit, deny, nextest, doctest, wasm, nostd, + hygiene). All three suites run locally with tight confidence intervals (Matter decode ~60 ns / encode ~85 ns, counter parse ~26 ns, stream multi-group parse ~74 ns).

Note: second commit

This branch also carries one unrelated CI fix — ci: skip statix's broken test suite in dev shell — because statix 0.5.8 in the current nixpkgs rev fails its own tests and breaks nix develop/direnv. Happy to split it into its own PR if preferred.

🤖 Generated with Claude Code

joeldsouzax and others added 2 commits June 30, 2026 21:35
Phase 0 (P0.1) instrument: there was no way to measure performance, which
every later perf card depends on. Adds a criterion-based `benches/` suite
covering the realistic hot paths.

- criterion 0.8.2 dev-dependency (default-features = false → no rayon/plotters,
  keeping the cargo-deny/audit surface minimal) + three [[bench]] targets gated
  required-features = ["stream"].
- benches/matter.rs   — encode (matter_to_qb64, fixed codes) + decode
  (from_qualified_base64, fixed and variable-size StrB64_L0) + qb64<->qb2.
- benches/counter.rs  — counter encode (v1 + auto-big) + counter-led group parse.
- benches/stream.rs   — full multi-primitive attachment-stream parse via groups().
- README "Benchmarks" section documenting how to run them.

Benches are std-only and feature-gated, so the no_std/WASM builds are untouched.
Written clippy-clean against the crate's full deny-set (no unwrap/expect/panic):
operations are benchmarked by black_box-ing the returned Result, and pre-built
fixtures are guarded with `if let Ok(..)`.

Tool choice and CI-gate analysis (criterion vs divan/iai-callgrind/tango;
CodSpeed deferred) captured in the #25 research comment. `nix flake check` green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
statix 0.5.8 in the current nixpkgs rev fails its own insta snapshot tests
during build, which breaks `nix develop` / direnv (the dev shell can't
instantiate). Override it with doCheck = false — the binary builds fine and
statix is dev-shell-only (not part of the `nix flake check` gate), so skipping
its upstream-broken tests is safe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@joeldsouzax joeldsouzax merged commit d7ac2b1 into main Jun 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P0.1 · Benchmark harness

1 participant