Benchmarks‐And‐Validation

Benchmarks And Validation

ContextClean benchmark claims should be tied to fixtures and measured token counts.

Run Benchmarks

powershell -ExecutionPolicy Bypass -File .\scripts\benchmarks.ps1

Outputs:

• benchmarks/fixtures/
• benchmarks/results.json
• benchmarks/results.md

Current Fixture Types

Benchmarks cover:

• HTML scrape
• CI failure log
• provider CI mix
• stack trace dump
• dirty HTML article

Each fixture should have must-keep and must-remove expectations where practical.

What To Validate

For a benchmark claim to be useful, validate:

• input token count
• output token count
• tokens saved
• reduction percent
• important content preserved
• noise removed
• warnings reasonable
• truncation behavior when budgets are applied

Local Verification

Developer gate:

cargo fmt --all -- --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace --all-features --locked
cargo build --workspace --release --locked

Windows helper:

.\scripts\check.ps1

CI Expectations

CI should cover:

• format
• clippy
• MSRV
• build on Linux, macOS, and Windows
• test on Linux, macOS, and Windows

Dependency updates should not merge if they break MSRV or the supported platform matrix.