Repository navigation
assertrust 0.2.0 adds an optional AI judge. The rules only read test code, so a test deleted along with the feature it tested looks just like a check quietly dropped to get CI green. With your own key, the judge reads each finding with the change around it and answers one question: does the test still check what it promises?
build/validate-exports.test.ts HIGH test-disabled
file deleted: its test no longer runs · fcd351b
judge: coverage lost: The validateExports logic was moved verbatim into a vite.config.ts build plugin that still runs in the build and throws on mismatch; the old unit test was deleted with its module and not replaced, though the behavior stays.
Turn it on
- uses: BeyondFolder/assertrust@v0
with:
judge: true
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}Or assertrust check --judge in a terminal. Findings it calls fine are folded away, and those it says lost coverage or look suspicious fail the check. Each change is judged once and the verdict cached, at about 2 cents a finding with the default, Sonnet 5.5. OpenAI, Gemini and local Ollama models work too. Without a key, assertrust works exactly as before. The README has the details.
Status: early. On the labelled history of 11 repos, 88% of the findings the judge fails are right, and it folds away 80 of the 97 false alarms.
Release Notes
The optional AI judge: with your own key, a model reads each finding with the change around it and says whether the test still checks what it promises.
assertrust check --judgegives each shown finding a verdict:fine,coverage-lost,suspiciousorunsure. Findings judged fine are folded away, andcoverage-lostandsuspiciousfail the check ([judge] fail-on). Changed expected values, hidden until now, show up when the judge doesn't call them fine.- Sonnet 5.5 at
mediumeffort by default.--judge-modeland--judge-effort, or[judge]inassertrust.toml, pick another: models of Anthropic, OpenAI, Google Gemini and Ollama work, namedprovider/model. - Verdicts are cached in
.git/assertrust/judge/, so a change is judged once and CI never flips. With no key, no network or a bad answer, findings show without verdicts and the check doesn't fail. --judge-dry-runprints exactly what a model would be sent, and needs no key.- Verdicts show in the terminal, in JSON (
"judge", still"schema": 1), and in the GitHub annotations and job summary. - The Action takes
judge,judge-modelandjudge-effort, and keeps verdicts between runs withactions/cache. assertrust backtest --judgemeasures the judge against labels. On the labelled history of 11 repos, with the default model, 88% of the findings it fails are right, and it folds away 80 of the 97 false alarms.- Building from source needs Rust 1.97.1.
Install assertrust 0.2.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/BeyondFolder/assertrust/releases/download/v0.2.0/assertrust-installer.sh | shDownload assertrust 0.2.0
| File | Platform | Checksum |
|---|---|---|
| assertrust-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| assertrust-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| assertrust-aarch64-unknown-linux-gnu.tar.xz | ARM64 Linux | checksum |
| assertrust-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |