Skip to content

ci: split benchstat threshold per-method sensitivity #62

@millerjp

Description

@millerjp

Summary

Current threshold (10% time/op at p≤0.05, any allocs/op regression) is too loose for O(n) methods (Range, Keys, Map, Values — 10% of 6–60 µs is noise-free) and too tight for sub-20 ns microbenchmarks (where CI jitter alone can cross 10%).

Scope

Either:

A. Tune scripts/check-bench-regression.sh to apply a stricter threshold (5–6%) to benchmarks tagged as O(n) and the current 10% to the fast-path ones.
B. Demote single-op benchmarks to advisory (continue-on-error: true) with a tracking issue.

Acceptance criteria

  1. Threshold policy documented in CONTRIBUTING.md §Performance baseline.
  2. 10 consecutive CI runs on unchanged main show zero false-positive failures.

Source: performance-reviewer + go-quality.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Can be deferredperformancePerformance-related changetestingTests: unit, BDD, benchmarks, fuzz

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions