Skip to content

Refresh README and benchmark snapshots#27

Merged
benvinegar merged 1 commit intomainfrom
chore/refresh-readme-and-benchmarks
Apr 27, 2026
Merged

Refresh README and benchmark snapshots#27
benvinegar merged 1 commit intomainfrom
chore/refresh-readme-and-benchmarks

Conversation

@benvinegar
Copy link
Copy Markdown
Member

@benvinegar benvinegar commented Apr 26, 2026

Summary

  • bump slop-scan from 0.3.0 to 0.4.0 for the updated default rule pack
  • update the README rule list to match the current default pack
  • rerun the per-rule, pinned benchmark, and rolling-history benchmark jobs
  • refresh the README cohort medians and rolling leaderboard snapshot from the regenerated artifacts

What changed in the results

Compared with the previous README snapshot, the current default pack now separates the pinned AI and mature-OSS cohorts much more strongly:

  • blended-score median ratio: 3.02x -> 6.91x
  • score/file ratio: 4.11x -> 8.82x
  • score/KLOC ratio: 2.35x -> 7.38x
  • findings/file ratio: 3.74x -> 6.97x

The refreshed per-rule mini benchmark is also cleaner after weak default-pack removals. Current top isolated rules are:

  1. defensive.promise-default-fallbacks0.81
  2. api.generic-status-envelopes0.76
  3. defensive.error-swallowing0.72
  4. defensive.stringified-unknown-errors0.70

Validation

  • bun run benchmark:rules
  • bun run benchmark:scan
  • bun run benchmark:report
  • bun run benchmark:history
  • bun test
  • bun run lint

This PR description was generated by Pi using GPT-5 Codex

@benvinegar benvinegar merged commit 8254487 into main Apr 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant