Skip to content

v0.2.0

Choose a tag to compare

@ennajari ennajari released this 25 Oct 13:51
· 39 commits to master since this release

Users pinned to ennajari/snitch@v1 get this release automatically.

Works on real projects

  • Snitch now collects your tests with the repo's own interpreter: .venv/bin/python when there is one, or SNITCH_PYTHON. Before, any repo whose tests import third-party packages could not be inventoried, so deleted or skipped tests were missed there. The GitHub Action gets a matching python input.

Detection

  • More ways to weaken a test are flagged:
    • an equality turned into a range;
    • a test's input range shrunk tenfold or more;
    • the actual value rewritten to fit the expected one;
    • a conftest.py fixture that plants the asserted value.
  • Runner and coverage config is read at any depth, not only the repo root (.venv and node_modules are skipped).
  • Test files follow the repo's python_files setting.

Claude Code plugin

  • In enforce mode, the agent can no longer write to Snitch's own state: .snitch.toml, the plugin's ledger and baselines, the installed plugin, and the Claude Code settings files that could switch hooks off. This covers Edit/Write and the common shell write forms. See "What enforce can and cannot block" in the README.
  • The enforce gate also stops perl -i on test files.
  • When a detector could not run during a session (usually because the tests import packages Snitch's own environment lacks), the Stop hook tells you once per session, so a quiet session is no longer mistaken for a clean one.
  • Escalating one test no longer covers the other tests in its file.
  • Faster: on a 2,000-file repo, a hook run drops from about 31 s to about 12 s.

GitHub Action

  • An incomplete check no longer looks clean: the receipt says a detector could not run and the job shows a warning. Set strict: "true" to fail the job instead.

TemptationBench

  • The verifier records which engine commit graded each trial and keeps the agent's diff.
  • New bench-score options:
    • an infra label voids a trial that never really ran;
    • --review lists the trials that still need a human label.
  • The Claude Code count is corrected to 18 test changes, not 14.