Repository navigation
Releases
v0.9.1
A release-process fix so pharn-ci can be listed on the GitHub Marketplace. The Action is unchanged from 0.9.0 apart from its version number.
Changed
- The release workflow now creates each GitHub Release as a draft, which the maintainer publishes with the
Marketplace box ticked, so pharn-ci can be listed on the GitHub Marketplace (immutable releases lock a published
release, and v0.9.0 could not be listed). The tag is still created at the verified commit; nothing in the Action
changed.
v0.9.0
Fix hints an agent can act on. Every message and every finding of Report v2 now carries an optional fix object that
says what kind of change would fix it (one of eight kinds, such as add-test or revert-config), with a fixed
sentence per message code or signal id and, on a finding, the path or path:line it already names. The hints are
generic and never lower a gate: verdicts, inputs and the PR comment are unchanged. Report v2 gains only the optional
fix field; no field, input or code was removed or renamed.
Added
- Fix hints: every message and every finding of Report v2 carries an optional
fixobject (kind,hintand, on a
finding,target), one of eight kinds and one fixed sentence per message code or signal id, so an agent can tell what
kind of change would fix a finding. No kind lowers a gate; the PR comment is unchanged.
v0.8.0
Two checks on what a pull request brings in and on how its tests ran. Dependency safety, signals D01 to D05
(DEPENDENCY_D01 to DEPENDENCY_D05, the report's dependencies object), checks the npm and Python dependencies a
pull request adds or changes: offline and on by default, D04 flags lockfile drift and D05 a dependency from a git
repository, a URL outside the npm and PyPI registries or a local file; with dependencies: lookup the Action also asks
the registries whether an added package exists (D01), is young (D02) or is rarely downloaded (D03). Flaky tests: the new
junit-paths input (CLI --junit) reads the JUnit XML the test step wrote, fills the report's flaky object and a
🧫 Flaky comment section and applies the org's quarantine list, and the new pharn-ci test command runs a test command
and exits 0 when only quarantined tests failed; test results never change the verdict. A behaviour change: dependency
safety runs by default (dependencies: auto) and D04 and D05 are errors, so a pull request that changes a manifest's
dependencies without its lockfile, or adds a git, URL or local-file dependency, now fails the check where it passed;
set dependencies: never to get the old behaviour back. A Dependabot bump and a lockfile-only refresh give no finding.
The CLI and the Action now read the committed text of the manifests and lockfiles a change touches, and with
dependencies: lookup the Action sends the names of added unscoped packages to registry.npmjs.org, api.npmjs.org and
pypi.org, the first hosts other than GitHub and pharn.dev it calls (Security below). A run upload also carries a
redacted junit/results.xml. Report v2 gains the optional dependencies object, fills the optional flaky object and
adds the new message codes; no field, input or code was removed or renamed.
Added
- Flaky tests (A6): the new
junit-pathsinput (CLI:--junit, repeatable) reads the JUnit XML the test step wrote,
with a hand-written, linear reader that refuses any DOCTYPE and expands no entity, capped at 50 files, 25 MiB and
20,000 test cases. Report v2 fills its optionalflakyobject (counts, and up to 200 quarantined test ids that
failed), the comment gains a 🧫 Flaky section, and four codes are added:JUNIT_NOT_FOUND,JUNIT_INVALID,
JUNIT_TOO_LARGE(warn) andFLAKY_QUARANTINED_FAILURES(info). The app's quarantine list from the policy
response is now applied unless the org policy setsflaky.quarantine: false; test results never change the
verdict. A run upload also carriesjunit/results.xml, rebuilt from the parsed cases with failure messages redacted
and cut to 2 KB (never the file, neversystem-outorsystem-err), unless the org setssettings.uploadJunit: false;sizes.junitis no longer always 0. The newpharn-ci test [--junit <glob>]... [--quarantine-file <path>] [--dry-run] -- <command>runs a test command without a shell and exits 0 when it failed only because of quarantined
tests, otherwise with the command's own code. A test id is<classname>::<name>(README, Flaky tests and JUnit
results). - Dependency safety (A13), signals D01 to D05, for the dependencies a pull request adds or changes in npm manifests and
lockfiles (package.json,package-lock.json,npm-shrinkwrap.json,pnpm-lock.yaml,yarn.lock) and Python
ones (requirements files,pyproject.toml,poetry.lock,uv.lock). Offline and on by default (new input
dependencies, defaultauto; CLI flag--dependencies): D04 lockfile drift (block: a manifest's dependencies
changed while its lockfile did not or does not record them, or a lockfile gives the same version a new hash) and D05
a source outside the npm and PyPI registries (block: git, another host, a local file, or a new package index).
Opt-in withdependencies: lookup(ordependencies.lookup: truein.pharn/ci.yml, or the org policy's new
dependencies.lookupkey inschema/policy.v1.json): the Action looks each added unscoped package up in
registry.npmjs.org,api.npmjs.organdpypi.org, sending only the package name in aGET, for D01 package does
not exist (block), D02 first published fewer thanmin_age_days(30) days before the head commit (warn) and D03
fewer thanmin_weekly_downloads(100) npm downloads last week (warn); lookups fail open to
DEPENDENCY_LOOKUP_UNAVAILABLE(warn)..pharn/ci.ymlmay only tighten (lookup: truenever overrules anever
input; the minimums only rise); an app policy with adependencieskey requires the check. Report v2 gains the
optionaldependenciesobject (checked,lookups,signals,truncated,unread) and the codes
DEPENDENCY_D01toDEPENDENCY_D05,DEPENDENCY_LOOKUP_UNAVAILABLEandDEPENDENCY_CHECK_INCOMPLETE; the PR comment
gains a 📦 Dependencies section. A block finding fails the verdict like an agent config secret. A Dependabot bump
and a lockfile-only refresh give no finding. Limits in docs/LIMITS.md, Dependency safety.
Security
- The CLI and the Action now read the committed text of the manifests and lockfiles a change touches (at most 5 MiB
each) at the merge base and at HEAD, unlessdependencies: never; a finding names a path, a line, fixed words and
only allow-listed package names and version specs (docs/THREAT-MODEL.mdrow C9). Withdependencies: lookupthe
Action sends the names of added unscoped packages toregistry.npmjs.org,api.npmjs.organdpypi.org, the first
requests to hosts other than GitHub and pharn.dev (SECURITY.md, Data handling; row D8, residual 43).
v0.7.0
Three checks on what the next agent will read and on whether the change is tested. The prompt-injection scan, signals
PI01 to PI05 (INJECTION_PI01 to INJECTION_PI05, the report's injection object), flags content a pull request adds
that is aimed at an agent: invisible or bidirectional Unicode, hidden Markdown that addresses an agent, known injection
phrases, long encoded blobs in an instruction file and an instruction file added in a vendored directory. Stale agent
instructions, signals SI01 to SI04 (SETUP_STALE_SI01 to SETUP_STALE_SI04, setupHealth.stale), flag an instruction
file that names a script, make target, path or file that does not exist, or that is old and names a file the change
edits. TESTS_NOT_EXERCISING_CHANGES warns when no test file the pull request adds or modifies imports a changed JS/TS
source file. A behaviour change: PI01 is an error by default, so a pull request that adds a zero-width character, a
bidi override or isolate, a tag character or a stray byte order mark now fails the check where it passed; test files,
fixtures/, __fixtures__/ and testdata/ are exempt, and an app policy can exempt paths (injection.exempt) or set
PI01's level (injection.signals). A pull request that changes JS/TS source without a test importing it now gets a
warn verdict where it got pass; it never fails the check. The CLI and the Action now read the committed text of the
agent instruction files, the root package.json and Makefile on every run with a local diff, with or without an org
policy, for the stale-instruction signals (Security below). Report v2 gains the optional injection, setupHealth and
testIntegrity.exercise fields and the new message codes; no field, input or code was removed or renamed.
Added
- Tests exercise the changed lines (A8 step 1,
TESTS_NOT_EXERCISING_CHANGES,warn): a changed JavaScript or
TypeScript source file counts as exercised when a JS/TS test file the pull request adds or modifies imports it
directly with a relative import and keeps or raises its assertion count. When at least one changed source file is not
exercised, the report carries one warning with the counts and up to 10 paths, the new optional
testIntegrity.exerciselists up to 200 paths, and the comment shows one line under "Test integrity" with a collapsed
list. It is a warning in every test-integrity mode and never fails the check, so a pull request that changes code
without a test now gets awarnverdict where it gotpass. Test files, declaration and config files, build output,
vendored, minified and generated files andcoverage.excludepaths are not asked; import aliases, workspace packages,
re-exports and per-test coverage are not followed (docs/LIMITS.md). It reads only the touched test files' text the
signals already read, and is silent in modeoff, without a diff and without that text. Report v2 gains the message
code and the optional field; nothing was removed or renamed. - Stale agent instructions, signals SI01 to SI04: on a change that touches an agent instruction file (
CLAUDE.md,
AGENTS.md,.cursor/rules,.github/copilot-instructions.mdand the other instruction kinds), or that deletes,
renames or removes what one names, pharn-ci reports a backticked command whose script is not in the root
package.jsonor whose target is not in the rootMakefile(SI01,SETUP_STALE_SI01), a backticked path that is not
tracked (SI02), a linked or@-imported file that is not tracked (SI03), allwarn, and an instruction file last
changed more than 90 days before HEAD's commit that names a file the change edits (SI04,info). Report v2 gains the
optionalsetupHealth.stalelist and the four codes; the PR comment gains a 📄 Instructions section. No finding
fails the check, and without a diff nothing is checked. - Prompt-injection surface scan (H4), signals PI01 to PI05: the lines a pull request adds are scanned for content aimed
at the next agent that reads the repository. PI01 (defaultblock, so it fails the check) flags invisible and
bidirectional Unicode (zero-width characters, bidi overrides and isolates as in Trojan Source, tag characters, a
stray byte order mark), with right-to-left marks, Persian and Indic joiners, emoji sequences and locale files left
alone; PI02 hidden Markdown (HTML comments, reference definitions, link titles) that addresses an agent; PI03 known
injection phrases from a versioned list (PH01toPH32); PI04 a base64 or hex blob over 200 characters in an agent
instruction file; PI05 an instruction file added undernode_modules/,vendor/,dist/and similar (PI02 to PI05
warn). Test files,fixtures/,__fixtures__/andtestdata/are exempt. Findings go into the new optional
injectionobject of the report and the comment's 🧬 Content section, as fixed words, phrase ids and code points,
never file text. New message codesINJECTION_PI01toINJECTION_PI05andINJECTION_TEXT_TRUNCATED(a file too
large to scan in full), and a newinjectionpolicy key (exempt,phrases,signals): an app policy can exempt
paths, add phrases and set severities; afile:policy can only add phrases and raise severities. Diff-only: no file
read, no new input, no new dependency, no network call.
Security
- The CLI and the Action now read the committed text of the agent instruction files (at most 50, 256 KiB scanned
each), the rootpackage.jsonandMakefile, andgit logtimes of up to 20 instruction files on every run with a
local diff, with or without an org policy. Only inline code, shell blocks, link targets and@imports are read; a
finding names a path or command made of repo-path characters only, and nothing else of the files leaves the job
(docs/THREAT-MODEL.mdrow C8, residual 40).
v0.6.0
Two org-policy rules for agent pull requests, and a .pharn/ci.yml that may only tighten. P13, agent
self-modification (POLICY_P13), blocks an agent pull request that widens its own or another agent's powers in an
agent settings file, an MCP config, a workflow or CODEOWNERS. P14, the agent PR size cap (POLICY_P14), fails an agent
pull request that changes more files or lines than the org policy's agents.max_files / agents.max_changed_lines;
it is opt-in, so a policy that sets neither sees no change. A behaviour change: a stricter coverage.threshold or
setup.require: true in .pharn/ci.yml now applies over a looser threshold input (or --threshold) or
require-agent-setup: false; to get the looser value back, lower it in .pharn/ci.yml itself. An added test file
pharn-ci cannot read is now named in the TEST_INTEGRITY_INDEX_UNAVAILABLE warning instead of dropped. Report v2 gains
the two message codes; no field, input or code was removed or renamed.
Added
- Org-policy rule P13, agent self-modification (
POLICY_P13, defaultblock): under an app orfile:policy, an
agent pull request that widens its own or another agent's powers is blocked. The Action compares each agent settings
file, MCP config, workflow and CODEOWNERS file the diff touches between the merge base and HEAD (read from committed
objects with the caps of the test sources): an allow rule added, a deny rule removed,defaultModechanged, a hook
removed or rewritten,disableAllHooksorenableAllProjectMcpServersturned on, an MCP server added or changed, a
secret newly referenced, a workflow permission widened or its block removed,pull_request_targetadded, or a
CODEOWNERS change other than owners added. One violation per file names the widening in fixed words, never file
text. A missing diff, or a file that cannot be read or parsed, fails closed. A person's pull request is never
subject to it, afile:policy cannot lower it, and the CLI never runs it. New message code in schema v2, no new
input, no new dependency, no network call. - Org-policy rule P14, agent PR size cap, with the new message code
POLICY_P14: an agent pull request that changes
more files thanagents.max_filesor more lines (added plus removed) thanagents.max_changed_linesfails, with one
path-less violation per exceeded cap that names the count, the cap and the top-level directories touched ("split it
by area: …"). It is opt-in: setagents.max_files/agents.max_changed_linesto turn it on. There is no default
cap, so a policy that sets neither sees no change. Lockfiles, snapshots,*.generated.*files andcoverage.exclude
paths do not count (coverage.excludeis not used when the pull request edits the config in use or a workflow,
unless the app policy sets the exclusions). A person's pull request is never subject to it, a missing diff fails
closed, the severity isblockand afile:policy can lower a cap but never raise or remove one. Report v2 gains
the code only; no field changes.
Changed
.pharn/ci.ymlmay now only tighten the coverage threshold and the agent setup requirement, as it already did the
test-integrity mode. A strictercoverage.thresholdnow applies over a looserthresholdinput or--threshold
flag (with both set, the larger applies; it was the input's), andsetup.require: truenow stays on when the
require-agent-setupinput isfalse(it was turned off). This is a behaviour change: a workflow that set a lower
threshold orrequire-agent-setup: falseto relax a stricter config now gets the config's value. An org policy that
raises the threshold over a config value that was in force namesconfiginPOLICY_OVERRIDE_IGNORED. The file,
root and format inputs still replace the config's, and exclusions are still the config's plus the input's. No new
message code and no schema change.- An added test file pharn-ci cannot read (the test index refuses it as
syntax-error-heavy,too-deepor
parser-threw, its text was dropped or missing at HEAD, or there is no Python index for a.pyfile) is now named in
TEST_INTEGRITY_INDEX_UNAVAILABLEwith its reason code, in a sentence of its own after the TI02 and TI05 one: "The
tests in these added test files could not be read, so they are not checked: …". It was dropped with no trace, so
its tests were checked by nothing and the report did not say so. The message stays a warning in every mode, is not
given in modeoff, and is unchanged when no added file is unread. No new message code and no schema change. - TI01's detail for a deleted test file with a same-stem test file added now says the base text "could not be parsed"
when it was read but the test index refused it, instead of saying it was not available. The severity is unchanged.
v0.5.0
The fixes of the 2026-11-01 review. Test integrity in block fails closed when it cannot see the change, TI03 blocks
the ways a skip was written to slip past it, unreadable agent settings and MCP files are errors, and the upload to
pharn.dev is bounded in time and never sent from a pull_request_target run. The Action input fail-on-threshold is
renamed fail-on-verdict; the old name still works, with a warning. Some warnings become errors on purpose: see
Changed and Security. No schema field or message code was added, removed or renamed.
Added
- The Action input
fail-on-verdict, the new name offail-on-threshold(review 2026-11-01, finding 27): it gates on
the verdict, whatever made it fail, not only on the coverage threshold.fail-on-thresholdstill works when
fail-on-verdictis not set, with a deprecation warning, and is ignored (with a warning) when both are; both stay
declared inaction.yml, which no longer gives either a default (the default,true, is applied in code, so a set
old name is never shadowed).
Changed
-
The Action applies two things the pharn.dev app contract already named (review finding 15 f and g). No schema change,
no new input, no new message code and no new host.- The app's
settings.messageMaxBytesnow lowers the upload's per-message cap: the cap is the smaller of 2048 bytes
and that setting, taken only from a policy response that validated whole in the same run (never under
policy: neveror afile:policy). The app's other settings (uploadJunit,redactMessages) and its
quarantinelist are validated and still not applied. meta.mergeBaseis recorded for a run with a local diff: the merge base of the resolved base andmeta.headSha
(the pull request's head, so on apull_requestrun the fork point, not the base tip), one extra local
git merge-baseper run at most. Every other run, and agit merge-basethat fails or prints no object id (a
shallow checkout), recordsnulland changes nothing else. Matrix legs of one job remain a documented limit
(docs/LIMITS.md).
- The app's
-
The Action's
packages/action/dist/index.jsshrinks from about 11.7 MB to about 1.7 MB: the TypeScript parser no
longer sits inside it.scripts/bundle.mjscopiestypescript@5.9.3'slib/typescript.jsbyte for byte beside the
bundle aspackages/action/dist/typescript.js(9.1 MB, the dist's second committed file), and the Action loads it
only when a pull request makes it parse a test file or test config, after checking its sha256 against a pin in
packages/action/src/vendored-typescript-pin.ts. A file that does not match the pin fails the step with an error
naming the file and both hashes, and nothing is reported. A run that parses no test file never reads it. The CLI
still inlines the parser. No schema change and no new message code. Seedocs/THREAT-MODEL.md, G1. -
One source for three text rules in
@pharn-ci/core(text.ts): the code-unit comparatorbyCodeUnit(about 50
copies, in core, node-io and the tests, are gone), the surrogate-safe cutcutCodeUnits(the…inside the limit,
an option for a cut without it, never inside a surrogate pair) andutf8ByteLength. No limit number, message code
or schema changes. Two byte-level effects: a TI09 value (base/head) is now cut to at most 80 code units,…
included (it was 81), and a key text to at most 100 (it was 101); and the three cuts that could leave a lone
surrogate (TI01 and TI06 details, the 500-unit test name) no longer do. -
The Action's
run.tsitems of the 2026-11-01 review. No schema change and no new message code.- A PR comment or job summary cut as a last resort (the messages alone over the limit) now closes every collapsed
<details>section it cuts into before the truncation note, so the note is no longer hidden inside it (37). - Fewer GitHub API calls (47): a pull request is read once per run (in
pr-numbermode it was read up to three
times: the target, the agent PR detection and the accept label), and the comments and the token's login are
listed once for the upload's comment link, the pointer and the sticky comment (they were listed twice when an
upload was tried, including when the app answered404). A failed listing is still asked again by the next
reader. Theconfigandbase-refinputs are read once, and the coverage file is read from the inputs after the
org policy, as the CLI reads it. - The base is decided once for the diff and for
report.json(45):meta.baseShais now the base the diff was taken
against, recorded only when it is a full commit SHA. With abase-refbranch name, apull_requestrun used to
record the PR base and apushrun the previous commit; both now recordnull. A 64-character (SHA-256) commit
SHA asbase-refis no longer also tried asorigin/<sha>. - The warning for a truncated PR-head file list in
pr-numbermode names what may miss files: the agent setup scan,
the secret scan and the org policy's forbidden-file rule (P08) (48).docs/THREAT-MODEL.mdresidual 36 records
why a bot's marker comment still counts as pharn-ci's own when the token's login is unknown: no API names an
installation token's own login. - An unexpected exception in the upload is still
failedwith the reasoninternaland never changes the check,
and its log line now names the error class (never its message); with debug logging (RUNNER_DEBUG=1) it then
fails the step with the message, so it can be found (28). A413from the app is still read without its body:
the app's API defines none, somaxBytesstaysnull(29,docs/LIMITS.md). - Not changed: finding 39 (the two test-sources tests
vi.mock("@pharn-ci/core")) is dropped. The mock is a pinned
spy-through to the realcheck(); replacing it would delete the test that pins it, which TI02 correctly blocks.
- A PR comment or job summary cut as a last resort (the messages alone over the limit) now closes every collapsed
-
Docs and metadata (second review, 2026-11-01): README says nothing phones home only without
id-token: write, the
quick precedence sentence no longer claims.pharn/ci.yml"can only tighten" (only the test-integrity mode is
merged; an input replaces the config's threshold, file, root, format andsetup.require), the token-footprint
bullet names the accept-label reads, and the AI-agnostic and "fully stateless" sentences name P11 and the opt-in
upload.SECURITY.mdbest practices name whatid-token: writeturns on.docs/THREAT-MODEL.md§1 names the
pharn.dev app, its responses,app-urland the OIDC token, and §3 gains item 37 (the threshold comes from the PR).
docs/LIMITS.md§1 points at TI02, TI05 and TI06 instead of "not detected yet", gains "What P01 misses", and says
fail-on-threshold: falseun-gates every cause of afailverdict;action.ymlsays so too, and its
github-tokentext names the commits read and its fail-closed agent-PR detection.AGENTS.mddescribes Report v2,
the schemas, the Python index subprocess and the opt-in pharn.dev calls.docs/MARKETPLACE.mdno longer says the
listing is refreshed on each release: the Action is not listed yet.CONTRIBUTING.mdsays why secret-pattern
tests build tokens at test time. -
Release verifyrefuses a run whoseHEADis not thechore: release X.Y.Z (#N)merge commit, so a tag can no
longer land on a later PR (v0.2.0 was tagged four PRs after its release PR). -
The skylos-nine fixture pins now cover
fixtures/skylos-nine-pytest/andfixtures/skylos-nine-sources/too, and
name the commits and reasons of the twochange.diffedits made before the pins existed (docs/PRECISION.md§3). -
Test-only, recorded late: 0.4.0 (#111) moved the growth tests to CPU time and raised four vitest timeouts to 300 s
(ac-ti0205-growth,ac-ti06-ti12-linear,ac-py-index-growth,limits); the growth ceilings themselves did not
change. -
An org policy that requires a deny rule or a hook (
setup.required_permissions_denyorsetup.required_hooks,
with P03 or P06 at block) now fails the check when the agent settings files could not be read, instead of warning
(second review, HIGH 4). The files are unread whenever the diff came from the compare API (a shallow checkout, the
actions/checkoutdefault;pr-numbermode; a failed local diff), so before this a default checkout turned the
policy's block into a warning and the step passed with the requirement unchecked.POLICY_SETTINGS_INVALIDis now
errorthere, as it already was underpolicy.required; usefetch-depth: 0. A rule the policy sets towarnor
offkeeps the warning. -
TI10 reports a test script replaced by a lone
trueor:("test": "true", ortrue || vitest run) as forcing a
successful exit, likeexit 0(second review, HIGH 5).docs/LIMITS.mdlists what TI10 still misses:
node -e "process.exit(0)", a positional path filter,--dirand--project. -
.pharn/ci.ymlrefuses a__proto__key, bare, quoted or nested: it set the prototype of the parsed map, so a
coverageblock under it escaped the unknown-key check (second review, finding 26). -
The PR comment shows the accepted line once:
TEST_INTEGRITY_ACCEPTEDis no longer repeated under Messages when the
Test integrity section shows the accepted line, and its hint moves under that line (finding 32). -
TEST_INTEGRITY_MODE_IGNOREDnames the config in use when theconfiginput points elsewhere than
.pharn/ci.yml; the upload bundle redacts secrets in object keys too;action.ymlstates thepython3
requirement; the comment renderer uses the shared unsafe-character set, and the Action reads logins with core's
LOGIN_PATTERN(findings 45, 46 and 48, cleanups). No schema change. -
Tests (findings 12, 35, 41): the uploaded run's report equals the offline one apart from
meta.upload; every
declared input and output must be in the never-removed list (test-integrityandaccept-labelwere...
v0.4.0
Org policy and the app link-up. The Action can fetch an org policy (policy input) and upload the redacted run bundle
(upload input) to the pharn.dev app, a new policy mode reads agent settings and MCP configs (rules P01–P11), agent PRs
are detected (meta.isAgentPr), and check A7 scans agent configs for secrets. With no policy and no app the report is
the one 0.3.0 produced; nothing was removed or renamed. See Security for the new network surface and file reads.
Added
- Org policy rules P02–P06, and policy mode: with a policy in force the CLI and the Action read the tracked agent
settings files (.claude/settings.jsonand.claude/settings.local.json, root and nested) and, when the policy lists
MCP servers,.mcp.json,.cursor/mcp.jsonand.vscode/mcp.json, at HEAD. P02 flags an allow rule that overlaps
a forbidden permission,defaultModebypassPermissionsorauto, andadditionalDirectories; P03 a missing
required deny rule; P04 an MCP server that is not allowlisted; P05 a forbidden MCP server; P06 a missing required
hook. New message codesPOLICY_P02toPOLICY_P06andPOLICY_SETTINGS_INVALID(a file the rules could not fully
use, or did not read); P09 now judges those edits as a block, warn or info. Nothing read from a file is printed. meta.isAgentPrin the Action's report, withmeta.prAuthor,meta.prLabels,meta.coAuthorsandmeta.agentHint:
whether the pull request counts as an agent PR, from its author, its commits' authors, committers and
Co-authored-bytrailers, and its labels (agentandai-generatedby default). Seven known agents are recognized.
Evidence that cannot be read in full (an unknown author, unreadable commits, more than 250 commits) counts as an agent
PR. Metadata only: the verdict, the messages, the comment and the exit code do not change. In core:detectAgentPr,
parseCoAuthorTrailers,agentPrMetaFields,withAgentPrMeta,KNOWN_AGENTSandDEFAULT_AGENT_LABELS.schema/policy.v1.json: the JSON Schema (draft 2020-12) for version 1 of the policy document, the file that will
tell pharn-ci what "safe to merge" means for a repository, shared with the pharn.dev app. A pure validator in core
(validatePolicy, andparsePolicyJsonfor JSON text) checks a document against it: it never throws, never
repeats a value from the document in an error, and bounds its work on any input. The org policy entry below is
what reads it.- Org policy. The Action takes two inputs,
policy(auto,never,requiredorfile:<path>, defaultauto) and
app-url(defaulthttps://pharn.dev), and the CLI two flags,--policy-file <path>and--no-policy. With
policy: autoorrequiredand an OIDC token (permissions: id-token: write), the Action makes oneGETof the
pharn.dev policy endpoint, sending only the OIDC token and the repository name, and applies the served policy over
the workflow inputs and.pharn/ci.yml: it raises the threshold,setup.requireand the test-integrity mode,
replaces the exclusions and the coverage root, sets per-signal severities and the accept label, and adds
setup.expecttools. Afile:policy carried by the pull request can only tighten. Two new message codes:
POLICY_UNAVAILABLE(warn,errorunderpolicy: required) andPOLICY_OVERRIDE_IGNORED(warn). With no
token,policy: neveror no policy, the report is the one it was. - Check A7, secrets in agent configs. The CLI and the Action read the committed text at
HEADof the
machine-readable agent setup files (.mcp.json,.cursor/mcp.json,.vscode/mcp.json,.claude/settings*.json,
.codex/config.toml,.gemini/settings.json,opencode.json,.aider.conf.yml, the files under.claude/hooks/and
the.json,.yamland.ymlfiles under.continue/rules/; never the Markdown kinds) and report ten shapes of
secret (GitHub, Anthropic, OpenAI, Google, AWS, Slack, Stripe, private keys, URL passwords and tokens, and secret-like
values under secret-named keys). Twelve message codes,AGENT_CONFIG_SECRET_S01toAGENT_CONFIG_SECRET_S11and
AGENT_CONFIG_SECRET_UNSCANNED: S01 to S09 areerrorand fail the check, S10 iswarn, S11 isinfo, and files that
could not be scanned are named in the unscanned code (errorwhen a drop could hide a secret). Report v2 only gains
codes; the message-code description inschema/report.v2.jsonnames them. In the Action, the files are read only for
a diff from the local checkout; in PR-number mode and for a compare-API diff the message says nothing was read. - A redaction module in
@pharn-ci/core:findSecretSpans,redactSecrets,truncateUtf8,redactMessageand
MESSAGE_MAX_BYTES. Nothing outside check A7 uses it yet. - Upload of the run to pharn.dev. The Action takes an
uploadinput (autoornever, defaultauto) and sets a
run-urloutput. Withauto, a job that grantsid-token: writeand an organization that installed the Pharn
GitHub App get the redactedreport.jsonand a newmeta.jsonuploaded; the App then comments on the pull request,
so the Action posts no comment of its own, and a404from the app puts an install hint under the comment and the
summary. A failed upload never fails the check; a report over the limit adds theUPLOAD_TOO_LARGEmessage code
(warn).schema/upload-meta.v1.jsondescribesmeta.json. In core:buildUploadMeta,buildUploadBundle,
buildUploadTar,withUploadOutcomeandUPLOAD_REPORT_MAX_BYTES. - Org policy rules P01, P08, P09 and P10. A policy that sets
setup.requireorsetup.expect(P01) raises
NO_AGENT_SETUP_FILESandAGENT_SETUP_EXPECTED_MISSINGto the rule's severity; a tracked file that matches a
setup.forbidden_filespattern (P08) is a newPOLICY_P08error; a pull request that edits the policy file it is
checked against, the config or a workflow while it sets a value looser than the policy, or an agent settings file
the policy checks (P09) is a newPOLICY_P09error (a warning for an agent settings file); an unreadable policy
(P10) staysPOLICY_UNAVAILABLE.policy.rulessets each rule's severity, and afile:policy can only raise it.
The findings are in the report'spolicyobject and a new Policy section of the PR comment; without a policy the
report and the comment are the ones they were. - Org policy rule P11, an agent pull request that changes a protected path. A policy that lists
agents.protected_paths
makes every path of an agent PR's diff that matches one of the globs (a rename counts by both its paths) a P11
violation, and one newPOLICY_P11message names up to 20 paths and the evidence that made the pull request an agent
PR, as fixed words.agents.protected_paths_modeandpolicy.rules.P11set the severity (blockby default; a
file:policy always blocks); a person's pull request is never subject to it, and a missing diff fails closed. The
policy'sagents.labelsandagents.extra_loginsnow add to the default agent labels and logins. A pull request
that counts as an agent PR only because its evidence could not be read gets oneAGENT_PR_DETECTION_INCOMPLETE
message (warn) that says why. CLI runs have no pull request, so P11 never fires there. - Org policy rule P07, agent instruction files that disagree. Under a policy, when two instruction files of one
directory (CLAUDE.md,AGENTS.md,GEMINI.md,.cursor/rules,.claude/rules,.github/copilot-instructions.md
and the other instruction kinds) name different package managers (npm,pnpm,yarn,bun) or test runners
(vitest,jest) in inline code or shell blocks, each is a P07 violation with its path, its line and the tools as
fixed words, and one newPOLICY_P07message names up to 20 files (warnby default;setup.consistencyand
policy.rules.P07set the severity, and afile:policy can only raise it). Files that could not be read or compared
are reported too. Without a policy, or with P07 switched off, nothing is read and the report is the one it was.
Changed
- Agent PR detection runs on every pull request run, not only on runs whose report carries
meta, so the Action reads the
pull request's commits (GET /repos/{owner}/{repo}/pulls/{n}/commits) on every pull request run. On its own the flag
still changes no verdict, message, comment or exit code; under a policy with protected paths, P11 uses it.
Security
-
A
file:org policy that the pull request edits is also read at the merge base, from committed git objects only
(never the working tree, no symbolic link followed, at most 1 MiB), and its labels, logins and protected paths count
together with the pull request's own: a pull request cannot remove a protected path or an agent label by editing the
policy file. When the merge-base version cannot be read, an edited policy file makes the pull request an agent PR. -
The Action makes one new read-only GitHub API call,
GET /repos/{owner}/{repo}/pulls/{n}/commits(at most 3 pages of
100), and reads the text of the pull request's commit messages in memory for the agent PR detection. A message is
capped at 65,536 UTF-16 code units, read for at most 50 trailers in linear time, and never stored, logged or sent;
only fixed words, valid logins and the trailers of known agents reach the report.docs/THREAT-MODEL.mdhas the new
surface A5 and residual risks 22 and 23. -
No A7 message, and nothing else the report, the comment or the summary carries, names a secret's value, its key or
any other text of an agent config file: a message holds the shape id, a path in a code span and a line number. -
The upload is a new network surface of the Action: two
POSTrequests to the app origin and onePUTto a signed
storage URL, only withuploadnotneverand a job that grantsid-token: write. Only the redactedreport.json
andmeta.jsonleav...
v0.3.0
Test integrity now reads the tests themselves. Nine new signals (TI02, TI05, TI06, TI07, TI08, TI09, TI12, TI15 and
TI16) compare what a change does to its test files, test-runner configs and sources, for JS/TS and Python alike, and the
report lists the tests a change adds, removes, modifies or renames. With test-integrity: block, a pull request that
passed under 0.2.0 can now fail: see the first entry under Changed for what, and how to get the 0.2.0 outcome back.
No input, output or message code was removed or renamed, the default mode is still warn, and the report is still
Report v2 with two optional fields added.
Added
- Signal TI02, a test removed, and the code
TEST_INTEGRITY_TI02. With TI05, the first signals that compare the
text of the test files a change modifies at the merge base and at HEAD (JS/TS and, through the Python test index,
.pyfiles). TI02 reports a test with no counterpart left in the touched test files by name or by a similar body,
blockin modeblockandwarnin modewarnand for a computed name, at its base path and base line. It clears a
test renamed in place in its describe, hand-written tests replaced by anit.eachtable, and a test moved to another
touched file under a new name when its body closely matches. No rule clears a test by its name alone: a removed test
is cleared by the same leaf name in another touched file only when that test holds the body. - Signal TI05, assertions emptied or dropped, and the code
TEST_INTEGRITY_TI05. It reports a test that had
assertions and has none at head (blockin modeblock), or fewer with the rest of its body unchanged (warn), at
its head path and head line. - Signal TI06, a test weakened, and the code
TEST_INTEGRITY_TI06, alwayswarn. TI06 compares each matched test at
the merge base and at HEAD and reports a strong matcher replaced by a weak one (toBetotoBeTruthy), an
expect.assertionsorexpect.hasAssertionsguard lowered or removed, an assertion that compares a literal with
itself added, and an assertion moved into atryblock with acatchor after an earlyreturn. It reads the
subject ofexpect(…)fortoBe,toEqualandtoStrictEqual, sotypeof,instanceof,.length,!!x,null
andundefinedrewrites to the matcher they equal are not a weakening. The detail is fixed text and matcher labels,
never a test name. - Signal TI07, an expected value edited to match, and the code
TEST_INTEGRITY_TI07. It warns, in modesblockand
warnalike and so never fails the check, when a test of a modified JS/TS or Python test file changes only literals
inside the expected-value arguments of its assertions (the Jest and Vitest matchers, chai'sequal, node:assert's
assert.equalfamily, unittest'sassertEqualfamily and the operand after==in a pytestassert) and the change
also edits a JS/TS or Python source file. The finding names the test and the matcher, never the literal. - Signal TI08, snapshot churn, and the code
TEST_INTEGRITY_TI08. It warns, in modesblockandwarnalike, when a
change modifies, deletes or rewrites more than 10 snapshot files (__snapshots__directories,.snapand.ambr
files), when a JS/TS test file rewrites 10 or moretoMatchInlineSnapshot(lines, or when a snapshot-update flag
(-u,--update,--updateSnapshot,--update-snapshots,--snapshot-update) is added to atest*script in
package.jsonor--snapshot-updateto a pytest config file. - Signal TI09, a test-runner setting loosened, and the code
TEST_INTEGRITY_TI09. TI09 compares the base and head
text of the touched test-runner config files:vitest.config.*,vite.config.*(now read as a test-config file by the
CLI and the Action),vitest.workspace.*,jest.config.*and thejestkey ofpackage.json,playwright.config.*,
.mocharc.*,pytest.ini,tox.ini,setup.cfgandpyproject.toml. A fixed list of keys is judged: a lowered
or removed coverage threshold, a narrowed test list, a widened exclusion (build directories,node_modulesand
.d.tsfiles excepted),passWithNoTests,allowOnlyorbailturned on, agrepadded,forbidOnlyturned off
and a pytestaddoptsthat gains a deselecting option areblockin modeblockandwarnin modewarn; a raised
retry count or timeout, and a value that is not a literal, are alwayswarn. The projects of a vitest or jestprojectslist are matched by name
(test.name,displayName) or by position and only their test-list keys are compared, so an unrelated key changed in
a project gives nothing. The finding sits at the head path and the head line of the key (no line for JSON). A config
that cannot be read, or the whole signal when a side of the test sources is unavailable, adds a TI09 sentence to
TEST_INTEGRITY_INDEX_UNAVAILABLE. - Signal TI12, a module under test mocked, and the code
TEST_INTEGRITY_TI12, alwayswarn. TI12 reports a new
vi.mock,jest.mockor Pythonpatchof a source file the same change edits, once per test file and target, with
the mock's specifier sanitized and cut to 120 characters. - Signal TI15, the test may be special-cased, and the code
TEST_INTEGRITY_TI15, at levelinfo: it never blocks or
warns, in modesblockandwarnalike. It fires when a condition added to a JavaScript, TypeScript or Python source
file (===,!==,==,!=, acaselabel or a Pythonin (...)group) compares with a string or number literal
that a test file in the same change also adds. It matches only inside the same package (the nearestpackage.jsonor
pyproject.toml), leaves outexamples,benchmarks,bench,docsandfixturesdirectories, needs 6 characters
or 4 significant digits, and lists at most 10 findings per change (the message says how many more). The finding names
the source line, the literal's kind and length and the test file, never the literal's text. It isinfobecause 0 of
30 sampled TI15 findings were right on the 600-PR precision corpus (docs/PRECISION.md). - Signal TI16, a lint or type suppression in tests, and the code
TEST_INTEGRITY_TI16. It gives aninfofinding,
which never blocks and leaves the verdict atpass, when a JS/TS or Python test file adds a comment that switches off
a linter or type checker. - At most 20 findings per file and signal are listed for TI02, TI05, TI06, TI09 and TI12, and the messages say how many
were left out. - The code
TEST_INTEGRITY_INDEX_UNAVAILABLE(warn). It names the touched test files that could not be compared, with a
reason code each, or says that TI02 and TI05 did not run. - Tests-changed summary:
testIntegrity.changesinreport.json(optional, inschema/report.v2.json) lists the tests a
change adds, removes, modifies or renames, with counts per test file, and the comment gains a collapsed "Tests changed"
table with the counts per test file. Test names are inreport.jsononly, cut to 120 characters, at most 200 entries.
An.eachreplacement is listed once under renamed. testIntegrity.languagesinreport.json(optional) lists the languages (js,py,ts) the test indexes parsed.- Python parity with JS/TS. TI03 and the Python test index resolve pytest marker and unittest decorator aliases
bound by a test file's own top-level imports (import pytest as pt,from pytest import mark as m,
from unittest import skipIf as si), so TI02, TI05 and TI06 see through them too; TI03 also reads
@unittest.expectedFailureandpytest.xfail(. TI09 reads pytest 9's native[tool.pytest]table of
pyproject.tomlbeside[tool.pytest.ini_options].docs/LIMITS.mdgains a Python parity table and the list of
Python misses. - The official precision results for v0.3.0, in
docs/PRECISION.md: 600 PRs, 0 errors, TI01, TI02 and TI11 each
falsely block 1 of 600 PRs and ship as block, and the reviewed CSV isfixtures/corpus/review.csv. - The Skylos nine reproduced for pytest under
fixtures/skylos-nine-pytest(eleven cases, the nine plus an aliased skip
and amock.ANYvariant of the always-true matcher), pinned by tests that runcheck()with the real Python index.
The signals catch 8 of 9 for pytest, and the vitestskylos-ninebenchmark underfixtures/skylos-nine/now
catches 7 of 9, up from 2 of 9 in 0.2.0 (emptied-assertion,deselect-setting,replaced-test,
loosened-assertionandedited-expected-valueare new).docs/PRECISION.mdgains a section with the case-by-case
table, anddocs/LIMITS.mdgains the TI17 proposal for the always-true matcher (a proposal only, nothing implements it).
Changed
- What can now fail a pull request in mode
blockthat passed under 0.2.0: a removed test (TI02), a test left with
no assertions (TI05), a loosened test-runner setting (TI09), a skip of an existing test written through an aliased
pytest marker or unittest decorator, or@unittest.expectedFailure(TI03), and a deleted test file replaced by a test
file of the same name whose tests are not found again by body (TI01, below). Modewarn, the default, reports the
same findings without failing the check. To get the 0.2.0 outcome back on one pull request, a reviewer who is not its
author applies the accept label (pharn/accepted, theaccept-labelinput), which downgrades its block findings to
warn; to get it back for every pull request, settest-integrity: warn. - TI01 no longer lets a name stand in for a test's code. A deleted test file replaced by a test file of the same
name isinfoonly when every test it held is found again by body in a new test, and stays ablockwhen the tests
are hollow or when no test sources are given (the twoinfo-migrationdiffs checked diff-only are now ablock).
TI01 also givesinfowhen every test a deleted test file held...
v0.2.0
Added
- The accept label, the
accept-labelAction input (defaultpharn/accepted, at most 50 characters). In
test-integrity: blockon apull_requestorpr-numberrun, a pull request that carries the label, applied last by
a user account that is not the PR author, has its block signals downgraded towarn: the report gains the
TEST_INTEGRITY_ACCEPTEDmessage (info),testIntegrity.acceptedand, for a non-default name,
testIntegrity.acceptLabel, and the comment says who accepted it. A label that does not count (the author or a bot
applied it, no labeling event, or anything unreadable) addsTEST_INTEGRITY_ACCEPT_IGNORED(warn) and the blocks
stand. The Action reads the pull request and its issue events (at most 10 pages of 100) withpull-requests: reador
issues: read; the CLI never honours the label, andtest_integrity.accept_labelin.pharn/ci.ymlis refused. The
TI03, TI10 and TI13 hints now name the label. - The
test-integrityAction input and--test-integrityCLI flag (block,warnoroff, defaultwarn),
and atest_integrity.modekey in.pharn/ci.ymlthat can only make the check stricter than the input. The
report'stestIntegrity.modeandtestIntegrity.sourcesay which mode applies and where it came from, a
looser config value is ignored with the newTEST_INTEGRITY_MODE_IGNOREDmessage (warn), modeoffadds
TEST_INTEGRITY_OFF(info), and the PR comment and job summary gain a "🔍 Test integrity" section. - Signal TI01, test file deleted: a deleted test file whose removed lines define tests, or a test file renamed to
a path that is not a test file (edited or not, since a rename shows only the lines it edits), is reported in
testIntegrity.signalsand in oneTEST_INTEGRITY_TI01message,
blockin modeblock(a failing check),warnin the default mode. It isinfowhen the same change also
deletes the source file of the same name, deletes other files of the test's package, or adds a test file of the
same name (a migration); helpers, fixtures and an__init__.pyundertests/do not count. The "🔍 Test
integrity" section lists the findings. Test files are now recognized by the spec's globs:.test.and.spec.
with a JavaScript or TypeScript extension,.e2e.ts,.pw.ts,test_*.py,*_test.py,*_test.goand the
test,testsand__tests__directories.TEST_FILES_DELETEDis unchanged and now uses the same detection. - Signal TI14, diff-coverage exclusions widened: a change to the pharn-ci config in use (
.pharn/ci.yml, or the
configinput) that lowers or removescoverage.threshold, or adds a coverage exclusion, is reported in
testIntegrity.signalsand in oneTEST_INTEGRITY_TI14message,blockin modeblock,warnin the default
mode. Raising a threshold, removing an exclusion and unrelated settings never fire.CONFIG_CHANGED_IN_DIFFis
unchanged.
The config lines are read as the config reads them: a quoted key, a quoted or escaped value and any number of
decimals all count. - Signal TI11, CI stopped running tests, from the diff of workflow files (
.github/workflows/*.yml|yaml),
reported intestIntegrity.signalsand in oneTEST_INTEGRITY_TI11message.blockin modeblock: a removed
test command (pnpm test,vitest,jest,pytest,go test, ...) that no workflow adds again, and a removed
pull_requesttrigger.warnin every mode butoff: a shrunk flow-list matrix, an added
continue-on-error: true, an addedif:that is a literal false or leaves pull requests out, and a
timeout-minutesof 0 or 1. Which step a line belongs to cannot be seen without context lines, hence the warnings. - Signal TI03, a skip added to a test file, from the kept text of the diff, reported in
testIntegrity.signals
and in oneTEST_INTEGRITY_TI03message. It reads JS/TS (.skip,xitand the otherxheads,.todo,.fails,
skipIf,runIf,this.skip(), Python (@pytest.mark.skip,skipif,xfail,@unittest.skip*,pytest.skip() and
Go (t.Skip(,t.Skipf(,t.SkipNow(). An unconditional skip of a test the diff shows existed isblockin mode
block; a conditional skip, a skip inside a test body, and a skip in a test the change adds arewarn. A skip that
was already there (an edit, a move) never fires, and environment guards such asskipIf(!hasCamera)or
pytest.skip("REDIS_URL not set")staywarn.
At most 20 findings per file, with the skips of existing tests kept first, so warn findings never push a block out. - Signal TI04, a focused-run marker added to a JS/TS test file (
.only, barefit(orfdescribe(), reported
intestIntegrity.signalsand in oneTEST_INTEGRITY_TI04message,blockin modeblock,warnin the default
mode.solver.fit(x),def fit(self), comments, strings and files that are not tests never fire; a marker that was
already there (an edit, a move) does not fire either. - Signal TI10, a test script weakened, from the diff of a modified or renamed
package.json(root or nested),
reported intestIntegrity.signalsand in oneTEST_INTEGRITY_TI10message,blockin modeblock,warnin the
default mode. In a script whose key starts withtest, an added|| true, anechoorexit 0as a command,
--passWithNoTests,--testPathPatternand-tor--testNamePatterneach fire once per line, unless a removed
line of the same key already had that form; a removedpretestorposttestscript fires too.--passWithNoTests
stays quiet when no tracked file is a test file. Other scripts, an added or deletedpackage.jsonand a pure edit
that keeps the weakening never fire. - Signal TI13, a coverage-ignore comment added to source, reported in
testIntegrity.signalsand in one
TEST_INTEGRITY_TI13message,blockin modeblock,warnin the default mode. An added Istanbul, V8 or C8
ignore hint in a comment of a JS/TS file, or a# pragma: no coverin a Python file, that is not a test file fires;
a marker that was already there (an edit, a move) never does. - The unified-diff parser keeps the text of added and removed lines per hunk for test files, the pharn-ci config in
use, workflow files, everypackage.jsonand the JavaScript, TypeScript and Python source files (at most 5000 lines
per file, 400 characters per line;textTruncatedmarks a cut file), for the signals that read line text.readDiffChunkstakes an optional{ configPath, repoRoot }so a custom
config path works through the stream as it does throughcheck(). - The
accept-labelinput also refuses invisible formatting characters (zero-width marks, bidirectional overrides
and isolates, a byte order mark inside the name), which could make the hint around the label read differently from
what it says. A spelling ofpharn/acceptedin other letter case is read as the default, so the report, the hints
and the sign-off line name the same label. The comment shows those characters in a path or label as U+FFFD. - The PR comment and job summary follow the Report v2 layout. The summary table gains a Test integrity column (
off,
none (warn), or the number of findings and how many block), and the headline sentence counts the findings when
there are any. The "🔍 Test integrity" section lists block findings first, then warn, then info, shows 10 and
collapses the next 10 in a<details>block. The Messages section leaves out eachTEST_INTEGRITY_TI*message
whose findings the section lists, so a finding and its hint appear once;report.jsonkeeps every message. - Precision of the diff-only signals, from a review of the base-rate fixtures. TI11 reads a quoted
"on":key and the
test commandsnpm t,pnpm -r test,pnpm --filter <pkg> test,pnpm vitest,pnpm exec,bun x,
node --test,bun test,deno test,tox,noxand theuv,poetry,pipenv,pdm,hatchandrye run
wrappers. TI10 reads; true,|| :and; :as ignoring the exit status, no longer reads a banner
(echo start && vitest run) as a weakening, and no longer reads mocha's-t(its timeout) as a name filter. TI13
reads coverage.py's# pragma: nocover. TI03 reads a Python skip decorator above an editeddefline, or above an
existing test in a diff with context lines, as a skip of an existing test. TI01 no longer counts a deleted fixture,
snapshot or data file under a test directory because its text was cut. A hand-made hunk header that starts a side at
line 0 numbers its lines from 1. - The
TEST_INTEGRITY_TEXT_TRUNCATEDmessage (warn, not in modeoff): a file that a test-integrity signal reads
(a test file, a workflow, a modifiedpackage.json, a JavaScript, TypeScript or Python source file, or the pharn-ci
config) has more than 5000 changed lines, so the signals read only the first 5000. The message names those files,
so a change cannot pass in silence by padding a file past the cut. Deleted files are not named. - README: an Agent skills section on the
pharn-ciplugin in
pharn-dev/skills (install, explain a verdict, cover
uncovered lines), and a pointer to it from the Quick start and the Pharn family table. - The Skylos nine benchmark: the nine ways Skylos hides a failing test, reproduced in vitest under
fixtures/skylos-nine/and pinned by tests. The diff-only signals of this release (TI01 and TI03) catch 2 of 9;
the other seven are named with the planned signal that would catch them. - A "What test integrity does not catch" section in
docs/LIMITS.md, and a "Test integrity"
section in the README with a rendered example of a finding. The "Only added lines count" limit no longer says that
skipped tests are not detected. - A release smoke script,
scripts/smoke-release.mjs, that builds a throwaway repository from `fixtu...
v0.1.0
Added
-
GitHub Action and
pharn-ci checkCLI. -
Diff coverage: the share of lines added in the change that the coverage report (lcov or
Cobertura XML) marks as executed, with an optional, inclusive threshold gate (thresholdinput,
--threshold,coverage.threshold). -
Agent setup inventory: the tracked agent setup files of Claude Code, Cursor, GitHub Copilot,
Codex, Gemini, Windsurf, Cline and MCP, listed by kind and never read. The kinds areclaude-md,
claude-local-md,agents-md,gemini-md,claude-settings,claude-settings-local,
claude-skill,claude-hook,claude-agent,claude-command,cursor-rules,cursor-mcp,
copilot-instructions,copilot-instructions-scoped,copilot-agents,copilot-prompts,
vscode-mcp,codex-config,windsurf-rules,cline-rulesandmcp-config, plus, for Aider,
Continue, Roo Code, Amazon Q Developer, Junie, Zed, Augment, Kiro, Goose, OpenCode and Gemini CLI
and for.claude/rules/,claude-rule,aider-config,conventions-md,continue-rules,
roo-rules,amazonq-rules,junie-guidelines,zed-rules,augment-rules,kiro-steering,
goosehints,opencode-configandgemini-settings..mcp.json,.claude/files and
.cursor/rules/are found in nested directories, and the more specific directory rule wins over
a file-name rule. -
Report v1: optional
agentSetup.summary(counts per kind and the tools involved), and an
Agent setup: N files · toolsline in the markdown, with the table collapsed above 25 files. -
Report v1 (
schema/report.v1.json), the stable JSON output, with message codes
DIFF_COVERAGE_BELOW_THRESHOLD,DIFF_COVERAGE_UNGATED,NO_COVERABLE_CHANGES,
COVERAGE_FILE_MISSING,COVERAGE_FILE_EMPTY,COVERAGE_FORMAT_UNSUPPORTED,
DIFF_UNAVAILABLEandNO_AGENT_SETUP_FILES. -
One sticky PR comment and a job summary; on fork PRs, where the token cannot comment, the job
summary only. -
.pharn/ci.ymlconfig file, overridden by Action inputs and CLI flags. -
Coverage exclusions: glob patterns (
*,**,?) fromcoverage.exclude, the CLI's repeatable
--exclude <glob>and the Action'sexcludeinput, merged (their union applies), leave matching
changed files out of diff coverage. The report gains the optionaldiffCoverage.excludedFiles
field, and the comment an "Excluded" summary listing them. -
Cobertura XML coverage parsing in core, read by the CLI and the Action.
-
Coverage format selection: the
coverage-formatAction input, the--coverage-formatCLI flag
andcoverage.formatin.pharn/ci.ymltakelcov,coberturaorauto. The flag or input
wins over the config file. -
autois the default: the format is detected from the first 1 KiB of the file, and the CLI
(stderr) and the Action (log) print which format was used, for example
coverage: cobertura (auto-detected). Clover and JaCoCo native XML are not supported. -
Message code
COVERAGE_FORMAT_UNSUPPORTED: underauto, a coverage file that is neither lcov
nor Cobertura XML is an error with a threshold, a warning without. -
Report v1: optional
diffCoverage.format(lcovorcobertura), the format the report was
read as. -
COVERAGE_LINES_SKIPPED(info): malformed lcovDArecords and Cobertura<line>elements that were skipped while reading the coverage files are now counted, and the report says how many (a count only, no line numbers or file names). The verdict and the diff coverage numbers do not change, and reports that end before diff coverage is computed do not carry the message. -
Coverage auto-discovery: with no
coverage-fileinput, no--coverageflag and nocoverage.file, the Action and the CLI look for the coverage report atcoverage/lcov.info,lcov.info,coverage.xml,coverage/cobertura-coverage.xmland**/coverage.cobertura.xml, and printcoverage: <path> (auto-discovered)for each file they read. More than one kind found is an error asking you to set the path; none found is aCOVERAGE_FILE_MISSINGmessage (previously a usage error). Complete workflows for vitest, jest, pytest, .NET and cargo are inexamples/workflows/. -
Expected agent tools:
setup.expectin.pharn/ci.yml, theexpect-agent-setupAction input and the CLI's--expect-agent-setupflag (repeatable) name agent tools (for exampleclaude-code,cursor) that should have a setup file. Any tool no setup file configures is listed in the newagentSetup.missingfield and reported asAGENT_SETUP_EXPECTED_MISSING(warn, or error when agent setup is required). A repo that expects nothing sees no change. -
AGENT_SETUP_FILE_EMPTY(warn): a listed agent setup file that is empty (size 0 inHEAD, fromgit ls-tree -r -l) is reported with the first path, and stays in the inventory. Whengit ls-treefails, none is reported (the CLI notes it on stderr, the Action warns). Because it is a warning, the verdict of a repo with an emptyCLAUDE.mdchanges frompasstowarn. -
TEST_FILES_DELETED(info): the change deletes a test file, or renames one to a name that is not a test file
(F-21). The message names up to five of them; the verdict is unchanged. Test files are recognized by name
(.test./.spec.in the basename, or atest,testsor__tests__directory). -
Agent setup inventory covers more tools (F-11): 13 new setup kinds,
claude-rule(.claude/rules/),aider-config,conventions-md,continue-rules,roo-rules,amazonq-rules,junie-guidelines,zed-rules,augment-rules,kiro-steering,goosehints,opencode-configandgemini-settings, so the files of Aider, Continue, Roo Code, Amazon Q Developer, Junie, Zed, Augment, Kiro, Goose, OpenCode and Gemini CLI are listed.agents-mdis now also credited to amp, jules, opencode and zed, andagentSetup.summary.toolsin Report v1 accepts new tool names within v1 (a lowercase name pattern replaces the closed list). -
PR-number mode: the
pr-numberandhead-shaAction inputs let pharn-ci measure and comment on a
pull request from aworkflow_runorworkflow_dispatchrun, with no PR code checked out. The PR,
its diff and its file list come from the GitHub API (two new read-only calls: get a pull request,
get a git tree),pr-numberis validated, andhead-shais checked against the PR head. The README
documents the two-workflow pattern for comments on fork PRs. -
Three warn-level messages for a change that touches the inputs of its own grade:
CONFIG_CHANGED_IN_DIFF(the diff edits the pharn-ci config in use or a file under.github/workflows/),COVERAGE_FILE_TRACKED(the coverage file is a tracked file, so the change can supply its own numbers) andDIFF_BINARY_COVERED_FILE(the diff shows a covered source file as binary, e.g. through-diffin.gitattributes, so its added lines were not measured). They lift apasstowarnand never fail the check. -
Several coverage files and globs: the
coverage-fileAction input takes one entry per line, the CLI's--coverageflag is repeatable andcoverage.filein.pharn/ci.ymlcan be a list. Each entry is a path or a glob (*,**,?, relative to the workspace, the current directory or the repo root), soTestResults/**/coverage.cobertura.xmlandpackages/*/coverage/lcov.infowork. The files are merged with hit counts summed;autodetects the format of each file. A glob that finds nothing isCOVERAGE_FILE_MISSING; a walk stops at 1,000 matched files or 100,000 visited entries, never follows a symbolic link and skips.gitandnode_modulesunless named. -
Require agent setup: the
require-agent-setupAction input, the CLI's--require-agent-setupflag andsetup.requirein.pharn/ci.yml(off by default) makeNO_AGENT_SETUP_FILESan error and the verdictfailwhen the repo has no agent setup files. -
DIFF_COVERAGE_UNGATED(info): diff coverage was measured, with its percent in the text, but no
threshold gates it (none is set, or it is 0). Before, an ungated run passed with no message. -
Data handling section in
SECURITY.md, linked from the README's Safety model: what the Action reads, what it sends to your GitHub API, what lands in the PR comment, job summary, step log andreport.json, and that nothing is sent to Pharn. -
The rendered report (PR comment, job summary, CLI markdown) adds a one-line hint under each message
saying how to fix it, and says what is missing (typicallyAGENTS.mdorCLAUDE.md) when no agent
setup files are found. Message text inreport.jsonis unchanged. -
coverage.excludein.pharn/ci.ymlcan be written as a one-line flow list, e.g.
exclude: ["**", "dist/**"], which used to stop the check with exit code 2. -
Coverage root:
coverage.rootin.pharn/ci.yml, thecoverage-rootAction input and the CLI's--coverage-rootflag name the repo-relative directory that relative coverage paths start from, so a package-relative lcov or Cobertura report in a monorepo is measured at the right files. Absolute paths are never remapped. -
COVERAGE_PATHS_PARTIALLY_UNMATCHED(info): some of the coverage report's file paths are not tracked files, so their coverage is not used. The verdict is unchanged. Thefixtures/goldenandfixtures/golden-coberturareports gain this message.
Changed
- When more than one coverage file is read (a glob, a list, or several
--coverageflags), thecoverage: <format> (<how>)log line names its file:coverage: coverage/a.info: lcov (auto-detected). With one file the line is unchanged, and so is the(auto-discovered)line. - The PR comment, job summary and CLI markdown now show the overview sentence in a GitHub alert under the verdict heading:
[!CAUTION]for fail,[!WARNING]for warn,[!TIP]for pass. Report data and message text are unchanged. - The PR comment, job summary and CLI markdown now show the pharn logo (a 48×48 image from the repository's raw
mainURL) above the verdict heading. Report data...