test: add criterion benchmark harness (#25)#40
Merged
Conversation
Phase 0 (P0.1) instrument: there was no way to measure performance, which every later perf card depends on. Adds a criterion-based `benches/` suite covering the realistic hot paths. - criterion 0.8.2 dev-dependency (default-features = false → no rayon/plotters, keeping the cargo-deny/audit surface minimal) + three [[bench]] targets gated required-features = ["stream"]. - benches/matter.rs — encode (matter_to_qb64, fixed codes) + decode (from_qualified_base64, fixed and variable-size StrB64_L0) + qb64<->qb2. - benches/counter.rs — counter encode (v1 + auto-big) + counter-led group parse. - benches/stream.rs — full multi-primitive attachment-stream parse via groups(). - README "Benchmarks" section documenting how to run them. Benches are std-only and feature-gated, so the no_std/WASM builds are untouched. Written clippy-clean against the crate's full deny-set (no unwrap/expect/panic): operations are benchmarked by black_box-ing the returned Result, and pre-built fixtures are guarded with `if let Ok(..)`. Tool choice and CI-gate analysis (criterion vs divan/iai-callgrind/tango; CodSpeed deferred) captured in the #25 research comment. `nix flake check` green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
statix 0.5.8 in the current nixpkgs rev fails its own insta snapshot tests during build, which breaks `nix develop` / direnv (the dev shell can't instantiate). Override it with doCheck = false — the binary builds fine and statix is dev-shell-only (not part of the `nix flake check` gate), so skipping its upstream-broken tests is safe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7 tasks
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #25 — Phase 0 (P0.1) instrument.
Why
Performance is the project's paramount goal, but there was no way to measure it — no
benches/, no bench dependency. Every later perf card (P1.x) depends on being able to answer "is this faster?". This adds a criterion-based harness over the realistic hot paths.What
criterion0.8.2 dev-dependency,default-features = false(dropsrayon/plotters) to keep thecargo deny/cargo auditsurface minimal. Three[[bench]]targets gatedrequired-features = ["stream"].benches/matter.rs— encode (matter_to_qb64, fixed codes) + decode (from_qualified_base64, fixed and variable-sizeStrB64_L0) +qb64↔qb2conversion.benches/counter.rs— counter encode (v1 + auto-big) + counter-led group parse (parse_group).benches/stream.rs— full multi-primitive attachment-stream parse viagroups().Design notes
std-only and feature-gated, so the no_std/WASM builds are untouched —cesr-wasmandcesr-nostdchecks stay green.clippy --all-targets --all-features, so bench code obeys the crate's full deny-set (nounwrap/expect/panic/as): operations are benchmarked byblack_box-ing the returnedResult, and pre-built fixtures are guarded withif let Ok(..).src/core/matter/test_vectors.rs, so they parse without contrivance.Research (per the strategy card's standing rule)
Tool choice (criterion vs divan/iai-callgrind/tango) and the CI-gate analysis (CodSpeed deferred) are captured with primary-source citations in the #25 research comment. The deterministic CI perf-gate (CodSpeed) is a deliberate follow-up.
Verification
nix flake checkgreen (all 16 checks: clippy, fmt, taplo, audit, deny, nextest, doctest, wasm, nostd, + hygiene). All three suites run locally with tight confidence intervals (Matter decode ~60 ns / encode ~85 ns, counter parse ~26 ns, stream multi-group parse ~74 ns).Note: second commit
This branch also carries one unrelated CI fix —
ci: skip statix's broken test suite in dev shell— becausestatix 0.5.8in the current nixpkgs rev fails its own tests and breaksnix develop/direnv. Happy to split it into its own PR if preferred.🤖 Generated with Claude Code