Skip to content

Releases

assertrust v0.2.0

Choose a tag to compare

@github-actions github-actions released this 18 Nov 21:56
v0.2.0
c4a4d88

assertrust 0.2.0 adds an optional AI judge. The rules only read test code, so a test deleted along with the feature it tested looks just like a check quietly dropped to get CI green. With your own key, the judge reads each finding with the change around it and answers one question: does the test still check what it promises?

build/validate-exports.test.ts  HIGH  test-disabled
  file deleted: its test no longer runs · fcd351b
  judge: coverage lost: The validateExports logic was moved verbatim into a vite.config.ts build plugin that still runs in the build and throws on mismatch; the old unit test was deleted with its module and not replaced, though the behavior stays.

Turn it on

      - uses: BeyondFolder/assertrust@v0
        with:
          judge: true
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

Or assertrust check --judge in a terminal. Findings it calls fine are folded away, and those it says lost coverage or look suspicious fail the check. Each change is judged once and the verdict cached, at about 2 cents a finding with the default, Sonnet 5.5. OpenAI, Gemini and local Ollama models work too. Without a key, assertrust works exactly as before. The README has the details.

Status: early. On the labelled history of 11 repos, 88% of the findings the judge fails are right, and it folds away 80 of the 97 false alarms.

Release Notes

The optional AI judge: with your own key, a model reads each finding with the change around it and says whether the test still checks what it promises.

  • assertrust check --judge gives each shown finding a verdict: fine, coverage-lost, suspicious or unsure. Findings judged fine are folded away, and coverage-lost and suspicious fail the check ([judge] fail-on). Changed expected values, hidden until now, show up when the judge doesn't call them fine.
  • Sonnet 5.5 at medium effort by default. --judge-model and --judge-effort, or [judge] in assertrust.toml, pick another: models of Anthropic, OpenAI, Google Gemini and Ollama work, named provider/model.
  • Verdicts are cached in .git/assertrust/judge/, so a change is judged once and CI never flips. With no key, no network or a bad answer, findings show without verdicts and the check doesn't fail.
  • --judge-dry-run prints exactly what a model would be sent, and needs no key.
  • Verdicts show in the terminal, in JSON ("judge", still "schema": 1), and in the GitHub annotations and job summary.
  • The Action takes judge, judge-model and judge-effort, and keeps verdicts between runs with actions/cache.
  • assertrust backtest --judge measures the judge against labels. On the labelled history of 11 repos, with the default model, 88% of the findings it fails are right, and it folds away 80 of the 97 false alarms.
  • Building from source needs Rust 1.97.1.

Install assertrust 0.2.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/BeyondFolder/assertrust/releases/download/v0.2.0/assertrust-installer.sh | sh

Download assertrust 0.2.0

File Platform Checksum
assertrust-aarch64-apple-darwin.tar.xz Apple Silicon macOS checksum
assertrust-x86_64-apple-darwin.tar.xz Intel macOS checksum
assertrust-aarch64-unknown-linux-gnu.tar.xz ARM64 Linux checksum
assertrust-x86_64-unknown-linux-gnu.tar.xz x64 Linux checksum

assertrust v0.1.0

Choose a tag to compare

@github-actions github-actions released this 15 Nov 11:45
v0.1.0
fc6ac2b

The first release of assertrust: it catches changes that quietly make a test suite weaker, like an assertion removed, a matcher loosened, a test skipped or a CI gate relaxed. It reads TypeScript and JavaScript tests written with vitest, jest or Playwright, names the commit that weakened a test, and needs no API key.

test/api/forwarding.test.ts:27  HIGH  expectation-loosened
  toMatchObject no longer checks domainIds, status · 10d8742

Use it on GitHub

name: assertrust
on:
  pull_request:
  push:
    branches: [main]
permissions:
  contents: read
  pull-requests: write # for the summary comment
jobs:
  assertrust:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - uses: BeyondFolder/assertrust@v0

Findings show up as annotations on the diff and in one PR comment. The README has every input, command and setting.

Status: early. On the history of 10 public repos and one private one, half of what it shows is right, and 70% at high confidence. So it only warns unless you set fail-on. Linux and macOS only for now; Windows, npm and crates.io come later.

Release Notes

The first release: a CLI and a GitHub Action that flag changes weakening a TypeScript or JavaScript test suite (vitest, jest, Playwright).

  • assertrust check finds removed checks, looser matchers, checks that accept more, disabled tests, checks that can no longer fail, and CI or test config that lets failures through. Checks in helper functions, hooks and helper class methods count for the tests that use them.
  • PR-style checks name the commit behind each finding and hide weakening undone later in the PR, or done to tests the PR itself added.
  • Output as text, JSON ("schema": 1) or GitHub annotations with a job summary; fail-on decides the exit code, warn-only by default.
  • Allow comments and Assertrust-Allow commit trailers acknowledge a deliberate change, with a reason.
  • assertrust.toml sets fail-on, which files are tests, helpers and gates, and which rules run. A change can't loosen its own check.
  • assertrust backtest replays a branch's history PR by PR, and measures precision against labels.
  • assertrust changes, tests and pairs show what a check sees.
  • The GitHub Action (BeyondFolder/assertrust@v0) checks pull requests and pushes, annotates the diff, and keeps one PR comment once there are findings.
  • Builds for Linux and macOS (x64 and arm64), with a shell installer.

Install assertrust 0.1.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/BeyondFolder/assertrust/releases/download/v0.1.0/assertrust-installer.sh | sh

Download assertrust 0.1.0

File Platform Checksum
assertrust-aarch64-apple-darwin.tar.xz Apple Silicon macOS checksum
assertrust-x86_64-apple-darwin.tar.xz Intel macOS checksum
assertrust-aarch64-unknown-linux-gnu.tar.xz ARM64 Linux checksum
assertrust-x86_64-unknown-linux-gnu.tar.xz x64 Linux checksum