Repository navigation
Releases
assertrust v0.2.0
assertrust 0.2.0 adds an optional AI judge. The rules only read test code, so a test deleted along with the feature it tested looks just like a check quietly dropped to get CI green. With your own key, the judge reads each finding with the change around it and answers one question: does the test still check what it promises?
build/validate-exports.test.ts HIGH test-disabled
file deleted: its test no longer runs · fcd351b
judge: coverage lost: The validateExports logic was moved verbatim into a vite.config.ts build plugin that still runs in the build and throws on mismatch; the old unit test was deleted with its module and not replaced, though the behavior stays.
Turn it on
- uses: BeyondFolder/assertrust@v0
with:
judge: true
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}Or assertrust check --judge in a terminal. Findings it calls fine are folded away, and those it says lost coverage or look suspicious fail the check. Each change is judged once and the verdict cached, at about 2 cents a finding with the default, Sonnet 5.5. OpenAI, Gemini and local Ollama models work too. Without a key, assertrust works exactly as before. The README has the details.
Status: early. On the labelled history of 11 repos, 88% of the findings the judge fails are right, and it folds away 80 of the 97 false alarms.
Release Notes
The optional AI judge: with your own key, a model reads each finding with the change around it and says whether the test still checks what it promises.
assertrust check --judgegives each shown finding a verdict:fine,coverage-lost,suspiciousorunsure. Findings judged fine are folded away, andcoverage-lostandsuspiciousfail the check ([judge] fail-on). Changed expected values, hidden until now, show up when the judge doesn't call them fine.- Sonnet 5.5 at
mediumeffort by default.--judge-modeland--judge-effort, or[judge]inassertrust.toml, pick another: models of Anthropic, OpenAI, Google Gemini and Ollama work, namedprovider/model. - Verdicts are cached in
.git/assertrust/judge/, so a change is judged once and CI never flips. With no key, no network or a bad answer, findings show without verdicts and the check doesn't fail. --judge-dry-runprints exactly what a model would be sent, and needs no key.- Verdicts show in the terminal, in JSON (
"judge", still"schema": 1), and in the GitHub annotations and job summary. - The Action takes
judge,judge-modelandjudge-effort, and keeps verdicts between runs withactions/cache. assertrust backtest --judgemeasures the judge against labels. On the labelled history of 11 repos, with the default model, 88% of the findings it fails are right, and it folds away 80 of the 97 false alarms.- Building from source needs Rust 1.97.1.
Install assertrust 0.2.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/BeyondFolder/assertrust/releases/download/v0.2.0/assertrust-installer.sh | shDownload assertrust 0.2.0
| File | Platform | Checksum |
|---|---|---|
| assertrust-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| assertrust-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| assertrust-aarch64-unknown-linux-gnu.tar.xz | ARM64 Linux | checksum |
| assertrust-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
assertrust v0.1.0
The first release of assertrust: it catches changes that quietly make a test suite weaker, like an assertion removed, a matcher loosened, a test skipped or a CI gate relaxed. It reads TypeScript and JavaScript tests written with vitest, jest or Playwright, names the commit that weakened a test, and needs no API key.
test/api/forwarding.test.ts:27 HIGH expectation-loosened
toMatchObject no longer checks domainIds, status · 10d8742
Use it on GitHub
name: assertrust
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
pull-requests: write # for the summary comment
jobs:
assertrust:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: BeyondFolder/assertrust@v0Findings show up as annotations on the diff and in one PR comment. The README has every input, command and setting.
Status: early. On the history of 10 public repos and one private one, half of what it shows is right, and 70% at high confidence. So it only warns unless you set fail-on. Linux and macOS only for now; Windows, npm and crates.io come later.
Release Notes
The first release: a CLI and a GitHub Action that flag changes weakening a TypeScript or JavaScript test suite (vitest, jest, Playwright).
assertrust checkfinds removed checks, looser matchers, checks that accept more, disabled tests, checks that can no longer fail, and CI or test config that lets failures through. Checks in helper functions, hooks and helper class methods count for the tests that use them.- PR-style checks name the commit behind each finding and hide weakening undone later in the PR, or done to tests the PR itself added.
- Output as text, JSON (
"schema": 1) or GitHub annotations with a job summary;fail-ondecides the exit code, warn-only by default. - Allow comments and
Assertrust-Allowcommit trailers acknowledge a deliberate change, with a reason. assertrust.tomlsetsfail-on, which files are tests, helpers and gates, and which rules run. A change can't loosen its own check.assertrust backtestreplays a branch's history PR by PR, and measures precision against labels.assertrust changes,testsandpairsshow what a check sees.- The GitHub Action (
BeyondFolder/assertrust@v0) checks pull requests and pushes, annotates the diff, and keeps one PR comment once there are findings. - Builds for Linux and macOS (x64 and arm64), with a shell installer.
Install assertrust 0.1.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/BeyondFolder/assertrust/releases/download/v0.1.0/assertrust-installer.sh | shDownload assertrust 0.1.0
| File | Platform | Checksum |
|---|---|---|
| assertrust-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| assertrust-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| assertrust-aarch64-unknown-linux-gnu.tar.xz | ARM64 Linux | checksum |
| assertrust-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |