Repository navigation
v0.2.0
Immutable
release. Only release title and notes can be modified.
Added
- The accept label, the
accept-labelAction input (defaultpharn/accepted, at most 50 characters). In
test-integrity: blockon apull_requestorpr-numberrun, a pull request that carries the label, applied last by
a user account that is not the PR author, has its block signals downgraded towarn: the report gains the
TEST_INTEGRITY_ACCEPTEDmessage (info),testIntegrity.acceptedand, for a non-default name,
testIntegrity.acceptLabel, and the comment says who accepted it. A label that does not count (the author or a bot
applied it, no labeling event, or anything unreadable) addsTEST_INTEGRITY_ACCEPT_IGNORED(warn) and the blocks
stand. The Action reads the pull request and its issue events (at most 10 pages of 100) withpull-requests: reador
issues: read; the CLI never honours the label, andtest_integrity.accept_labelin.pharn/ci.ymlis refused. The
TI03, TI10 and TI13 hints now name the label. - The
test-integrityAction input and--test-integrityCLI flag (block,warnoroff, defaultwarn),
and atest_integrity.modekey in.pharn/ci.ymlthat can only make the check stricter than the input. The
report'stestIntegrity.modeandtestIntegrity.sourcesay which mode applies and where it came from, a
looser config value is ignored with the newTEST_INTEGRITY_MODE_IGNOREDmessage (warn), modeoffadds
TEST_INTEGRITY_OFF(info), and the PR comment and job summary gain a "🔍 Test integrity" section. - Signal TI01, test file deleted: a deleted test file whose removed lines define tests, or a test file renamed to
a path that is not a test file (edited or not, since a rename shows only the lines it edits), is reported in
testIntegrity.signalsand in oneTEST_INTEGRITY_TI01message,
blockin modeblock(a failing check),warnin the default mode. It isinfowhen the same change also
deletes the source file of the same name, deletes other files of the test's package, or adds a test file of the
same name (a migration); helpers, fixtures and an__init__.pyundertests/do not count. The "🔍 Test
integrity" section lists the findings. Test files are now recognized by the spec's globs:.test.and.spec.
with a JavaScript or TypeScript extension,.e2e.ts,.pw.ts,test_*.py,*_test.py,*_test.goand the
test,testsand__tests__directories.TEST_FILES_DELETEDis unchanged and now uses the same detection. - Signal TI14, diff-coverage exclusions widened: a change to the pharn-ci config in use (
.pharn/ci.yml, or the
configinput) that lowers or removescoverage.threshold, or adds a coverage exclusion, is reported in
testIntegrity.signalsand in oneTEST_INTEGRITY_TI14message,blockin modeblock,warnin the default
mode. Raising a threshold, removing an exclusion and unrelated settings never fire.CONFIG_CHANGED_IN_DIFFis
unchanged.
The config lines are read as the config reads them: a quoted key, a quoted or escaped value and any number of
decimals all count. - Signal TI11, CI stopped running tests, from the diff of workflow files (
.github/workflows/*.yml|yaml),
reported intestIntegrity.signalsand in oneTEST_INTEGRITY_TI11message.blockin modeblock: a removed
test command (pnpm test,vitest,jest,pytest,go test, ...) that no workflow adds again, and a removed
pull_requesttrigger.warnin every mode butoff: a shrunk flow-list matrix, an added
continue-on-error: true, an addedif:that is a literal false or leaves pull requests out, and a
timeout-minutesof 0 or 1. Which step a line belongs to cannot be seen without context lines, hence the warnings. - Signal TI03, a skip added to a test file, from the kept text of the diff, reported in
testIntegrity.signals
and in oneTEST_INTEGRITY_TI03message. It reads JS/TS (.skip,xitand the otherxheads,.todo,.fails,
skipIf,runIf,this.skip(), Python (@pytest.mark.skip,skipif,xfail,@unittest.skip*,pytest.skip() and
Go (t.Skip(,t.Skipf(,t.SkipNow(). An unconditional skip of a test the diff shows existed isblockin mode
block; a conditional skip, a skip inside a test body, and a skip in a test the change adds arewarn. A skip that
was already there (an edit, a move) never fires, and environment guards such asskipIf(!hasCamera)or
pytest.skip("REDIS_URL not set")staywarn.
At most 20 findings per file, with the skips of existing tests kept first, so warn findings never push a block out. - Signal TI04, a focused-run marker added to a JS/TS test file (
.only, barefit(orfdescribe(), reported
intestIntegrity.signalsand in oneTEST_INTEGRITY_TI04message,blockin modeblock,warnin the default
mode.solver.fit(x),def fit(self), comments, strings and files that are not tests never fire; a marker that was
already there (an edit, a move) does not fire either. - Signal TI10, a test script weakened, from the diff of a modified or renamed
package.json(root or nested),
reported intestIntegrity.signalsand in oneTEST_INTEGRITY_TI10message,blockin modeblock,warnin the
default mode. In a script whose key starts withtest, an added|| true, anechoorexit 0as a command,
--passWithNoTests,--testPathPatternand-tor--testNamePatterneach fire once per line, unless a removed
line of the same key already had that form; a removedpretestorposttestscript fires too.--passWithNoTests
stays quiet when no tracked file is a test file. Other scripts, an added or deletedpackage.jsonand a pure edit
that keeps the weakening never fire. - Signal TI13, a coverage-ignore comment added to source, reported in
testIntegrity.signalsand in one
TEST_INTEGRITY_TI13message,blockin modeblock,warnin the default mode. An added Istanbul, V8 or C8
ignore hint in a comment of a JS/TS file, or a# pragma: no coverin a Python file, that is not a test file fires;
a marker that was already there (an edit, a move) never does. - The unified-diff parser keeps the text of added and removed lines per hunk for test files, the pharn-ci config in
use, workflow files, everypackage.jsonand the JavaScript, TypeScript and Python source files (at most 5000 lines
per file, 400 characters per line;textTruncatedmarks a cut file), for the signals that read line text.readDiffChunkstakes an optional{ configPath, repoRoot }so a custom
config path works through the stream as it does throughcheck(). - The
accept-labelinput also refuses invisible formatting characters (zero-width marks, bidirectional overrides
and isolates, a byte order mark inside the name), which could make the hint around the label read differently from
what it says. A spelling ofpharn/acceptedin other letter case is read as the default, so the report, the hints
and the sign-off line name the same label. The comment shows those characters in a path or label as U+FFFD. - The PR comment and job summary follow the Report v2 layout. The summary table gains a Test integrity column (
off,
none (warn), or the number of findings and how many block), and the headline sentence counts the findings when
there are any. The "🔍 Test integrity" section lists block findings first, then warn, then info, shows 10 and
collapses the next 10 in a<details>block. The Messages section leaves out eachTEST_INTEGRITY_TI*message
whose findings the section lists, so a finding and its hint appear once;report.jsonkeeps every message. - Precision of the diff-only signals, from a review of the base-rate fixtures. TI11 reads a quoted
"on":key and the
test commandsnpm t,pnpm -r test,pnpm --filter <pkg> test,pnpm vitest,pnpm exec,bun x,
node --test,bun test,deno test,tox,noxand theuv,poetry,pipenv,pdm,hatchandrye run
wrappers. TI10 reads; true,|| :and; :as ignoring the exit status, no longer reads a banner
(echo start && vitest run) as a weakening, and no longer reads mocha's-t(its timeout) as a name filter. TI13
reads coverage.py's# pragma: nocover. TI03 reads a Python skip decorator above an editeddefline, or above an
existing test in a diff with context lines, as a skip of an existing test. TI01 no longer counts a deleted fixture,
snapshot or data file under a test directory because its text was cut. A hand-made hunk header that starts a side at
line 0 numbers its lines from 1. - The
TEST_INTEGRITY_TEXT_TRUNCATEDmessage (warn, not in modeoff): a file that a test-integrity signal reads
(a test file, a workflow, a modifiedpackage.json, a JavaScript, TypeScript or Python source file, or the pharn-ci
config) has more than 5000 changed lines, so the signals read only the first 5000. The message names those files,
so a change cannot pass in silence by padding a file past the cut. Deleted files are not named. - README: an Agent skills section on the
pharn-ciplugin in
pharn-dev/skills (install, explain a verdict, cover
uncovered lines), and a pointer to it from the Quick start and the Pharn family table. - The Skylos nine benchmark: the nine ways Skylos hides a failing test, reproduced in vitest under
fixtures/skylos-nine/and pinned by tests. The diff-only signals of this release (TI01 and TI03) catch 2 of 9;
the other seven are named with the planned signal that would catch them. - A "What test integrity does not catch" section in
docs/LIMITS.md, and a "Test integrity"
section in the README with a rendered example of a finding. The "Only added lines count" limit no longer says that
skipped tests are not detected. - A release smoke script,
scripts/smoke-release.mjs, that builds a throwaway repository fromfixtures/smoke/, runs
the built Action bundle against it with no token and no event, and checks the exit code, theverdictoutput, the
job summary and the footer version.docs/RELEASING.mddescribes running it in the dry run and after the release,
and adds a check with a pull request from a fork.
Changed
- Breaking for report consumers: the report is now Report v2
(schema/report.v2.json).schemaVersionis the number2(Report v1 used the
string"1"), every report carries atestIntegrityblock (the mode in effect and where it came from, the
verdict and the signals; modewarnfrom sourcedefaultunless set), and the Action'sreport.jsonadds ametablock with run
metadata (event, repository, pull request, head and base commits, run id and attempt). Every Report v1 field
keeps its name, type and meaning, and no message code changed.report.v1.jsonhas
additionalProperties: false, so a v2 report does not validate against it: tools that validate the report
must switch onschemaVersion. The CLI's--format jsonprints Report v2, and the comment footer reads
report schema v2.schema/report.v1.jsonis unchanged.