Repository navigation
honest-tests v0.1.0
honest-tests v0.1.0
Your AI wrote 200 tests. Many can't fail.
honest-tests is an agent skill plus a CLI for JS/TS test suites. It finds tests that can't fail, and
then proves which tests are weak by mutating your code and watching the suite stay green. It works
with Claude Code, Codex, Cursor or no agent at all.
npx honest-tests audit . # find tests that can't fail
npx honest-tests mutate src/pricing.ts # break the code on purpose, see which tests noticeHighlights
- 13 rules for fake and weak tests. The rules catch things like
expect(x).toBe(x), tests with
no assertions, asserting on a mock's configured return value, mocking the module under test,
try/catchblocks that swallow theAssertionError, assertions that only run inside acatch,
un-awaited.resolves,toBeTruthywithout the(), and leftover.onlyand.skip. - New:
asserts-on-own-literal. This rule catches
const p = { service: 's3' }; expect(p.service).toBe('s3'). Nothing from the code under test is
involved, so the test can never fail. The pattern is common in runtime "type tests", because
TypeScript types don't exist at runtime. The rule only fires when the asserted value traces back
purely to literals declared in the same test, and that binding is never passed to a function,
mutated or reassigned. - Built-in mutation testing.
mutateflips operators, booleans, constants, conditions and return
values. It runs only the tests that reach the file, and lists every change nobody noticed. It works
on a temporary copy of the project, so the working tree is never touched, even on Ctrl-C. - Agent-native.
- An Agent Skill teaches the rule a test is honest only if it fails when the behavior it names is
broken. It covers a writing workflow (self-audit before saying "done") and a phased audit
workflow (detect, verify, report, fix). - A Claude Code plugin adds
/honest-tests:auditand/honest-tests:mutate. honest-tests init codex|cursorwrites anAGENTS.mdsection or a Cursor rule.
- An Agent Skill teaches the rule a test is honest only if it fails when the behavior it names is
- Fits into CI. It provides
--jsonoutput (schema v1),--fail-on,--min-scoreand
suppression comments that take a reason.
What it found in the wild
- Open-source projects: across 5 popular OSS JS/TS projects (11,178 tests, all 13
current rules), 2.6% of tests got an error-level finding, and 0.6% had a high-confidence can't-fail
bug (never-failing-assertion,catch-only-assertion,tautological-assertion,
asserts-on-mockorasserts-on-own-literal). - 100% line coverage: in one app that enforces 100% line coverage, mutation testing still found
that 15% of the changes to a formatting module are caught by no test. - Precision of the new rule:
asserts-on-own-literalwas field-checked on 13 OSS JS/TS projects
(about 27,000 tests). It made 14 hits, and a manual review found every one was an assertion that
can't fail. There were no false positives. A prototype pass also found 15 hits in one real app.
Install
npx honest-tests audit . # CLI, Node 22+
npx honest-tests init claude # skill into .claude/skills//plugin marketplace add ranjan-mohanty/honest-tests
/plugin install honest-tests@honest-tests
Known limitations
- JS/TS only (vitest, jest, node:assert, chai). The pytest adapter is next.
- When
mutatepicks related tests, it doesn't follow path-alias imports (@/lib/x). Pass
--test-cmdin that case. - Mutants run one at a time. The bundled fixture takes about 12 s for 26 mutants.
Full changelog: CHANGELOG.md
