Skip to content

Releases

v0.9.1

Choose a tag to compare

@github-actions github-actions released this 08 Nov 14:08
Immutable release. Only release title and notes can be modified.
dec4a0e

A release-process fix so pharn-ci can be listed on the GitHub Marketplace. The Action is unchanged from 0.9.0 apart from its version number.

Changed

  • The release workflow now creates each GitHub Release as a draft, which the maintainer publishes with the
    Marketplace box ticked, so pharn-ci can be listed on the GitHub Marketplace (immutable releases lock a published
    release, and v0.9.0 could not be listed). The tag is still created at the verified commit; nothing in the Action
    changed.

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 08 Nov 12:28
Immutable release. Only release title and notes can be modified.
fd1795c

Fix hints an agent can act on. Every message and every finding of Report v2 now carries an optional fix object that
says what kind of change would fix it (one of eight kinds, such as add-test or revert-config), with a fixed
sentence per message code or signal id and, on a finding, the path or path:line it already names. The hints are
generic and never lower a gate: verdicts, inputs and the PR comment are unchanged. Report v2 gains only the optional
fix field; no field, input or code was removed or renamed.

Added

  • Fix hints: every message and every finding of Report v2 carries an optional fix object (kind, hint and, on a
    finding, target), one of eight kinds and one fixed sentence per message code or signal id, so an agent can tell what
    kind of change would fix a finding. No kind lowers a gate; the PR comment is unchanged.

v0.8.0

Choose a tag to compare

@github-actions github-actions released this 08 Nov 08:37
Immutable release. Only release title and notes can be modified.
7d88076

Two checks on what a pull request brings in and on how its tests ran. Dependency safety, signals D01 to D05
(DEPENDENCY_D01 to DEPENDENCY_D05, the report's dependencies object), checks the npm and Python dependencies a
pull request adds or changes: offline and on by default, D04 flags lockfile drift and D05 a dependency from a git
repository, a URL outside the npm and PyPI registries or a local file; with dependencies: lookup the Action also asks
the registries whether an added package exists (D01), is young (D02) or is rarely downloaded (D03). Flaky tests: the new
junit-paths input (CLI --junit) reads the JUnit XML the test step wrote, fills the report's flaky object and a
🧫 Flaky comment section and applies the org's quarantine list, and the new pharn-ci test command runs a test command
and exits 0 when only quarantined tests failed; test results never change the verdict. A behaviour change: dependency
safety runs by default (dependencies: auto) and D04 and D05 are errors, so a pull request that changes a manifest's
dependencies without its lockfile, or adds a git, URL or local-file dependency, now fails the check where it passed;
set dependencies: never to get the old behaviour back. A Dependabot bump and a lockfile-only refresh give no finding.
The CLI and the Action now read the committed text of the manifests and lockfiles a change touches, and with
dependencies: lookup the Action sends the names of added unscoped packages to registry.npmjs.org, api.npmjs.org and
pypi.org, the first hosts other than GitHub and pharn.dev it calls (Security below). A run upload also carries a
redacted junit/results.xml. Report v2 gains the optional dependencies object, fills the optional flaky object and
adds the new message codes; no field, input or code was removed or renamed.

Added

  • Flaky tests (A6): the new junit-paths input (CLI: --junit, repeatable) reads the JUnit XML the test step wrote,
    with a hand-written, linear reader that refuses any DOCTYPE and expands no entity, capped at 50 files, 25 MiB and
    20,000 test cases. Report v2 fills its optional flaky object (counts, and up to 200 quarantined test ids that
    failed), the comment gains a 🧫 Flaky section, and four codes are added: JUNIT_NOT_FOUND, JUNIT_INVALID,
    JUNIT_TOO_LARGE (warn) and FLAKY_QUARANTINED_FAILURES (info). The app's quarantine list from the policy
    response is now applied unless the org policy sets flaky.quarantine: false; test results never change the
    verdict. A run upload also carries junit/results.xml, rebuilt from the parsed cases with failure messages redacted
    and cut to 2 KB (never the file, never system-out or system-err), unless the org sets settings.uploadJunit: false; sizes.junit is no longer always 0. The new pharn-ci test [--junit <glob>]... [--quarantine-file <path>] [--dry-run] -- <command> runs a test command without a shell and exits 0 when it failed only because of quarantined
    tests, otherwise with the command's own code. A test id is <classname>::<name> (README, Flaky tests and JUnit
    results).
  • Dependency safety (A13), signals D01 to D05, for the dependencies a pull request adds or changes in npm manifests and
    lockfiles (package.json, package-lock.json, npm-shrinkwrap.json, pnpm-lock.yaml, yarn.lock) and Python
    ones (requirements files, pyproject.toml, poetry.lock, uv.lock). Offline and on by default (new input
    dependencies, default auto; CLI flag --dependencies): D04 lockfile drift (block: a manifest's dependencies
    changed while its lockfile did not or does not record them, or a lockfile gives the same version a new hash) and D05
    a source outside the npm and PyPI registries (block: git, another host, a local file, or a new package index).
    Opt-in with dependencies: lookup (or dependencies.lookup: true in .pharn/ci.yml, or the org policy's new
    dependencies.lookup key in schema/policy.v1.json): the Action looks each added unscoped package up in
    registry.npmjs.org, api.npmjs.org and pypi.org, sending only the package name in a GET, for D01 package does
    not exist (block), D02 first published fewer than min_age_days (30) days before the head commit (warn) and D03
    fewer than min_weekly_downloads (100) npm downloads last week (warn); lookups fail open to
    DEPENDENCY_LOOKUP_UNAVAILABLE (warn). .pharn/ci.yml may only tighten (lookup: true never overrules a never
    input; the minimums only rise); an app policy with a dependencies key requires the check. Report v2 gains the
    optional dependencies object (checked, lookups, signals, truncated, unread) and the codes
    DEPENDENCY_D01 to DEPENDENCY_D05, DEPENDENCY_LOOKUP_UNAVAILABLE and DEPENDENCY_CHECK_INCOMPLETE; the PR comment
    gains a 📦 Dependencies section. A block finding fails the verdict like an agent config secret. A Dependabot bump
    and a lockfile-only refresh give no finding. Limits in docs/LIMITS.md, Dependency safety.

Security

  • The CLI and the Action now read the committed text of the manifests and lockfiles a change touches (at most 5 MiB
    each) at the merge base and at HEAD, unless dependencies: never; a finding names a path, a line, fixed words and
    only allow-listed package names and version specs (docs/THREAT-MODEL.md row C9). With dependencies: lookup the
    Action sends the names of added unscoped packages to registry.npmjs.org, api.npmjs.org and pypi.org, the first
    requests to hosts other than GitHub and pharn.dev (SECURITY.md, Data handling; row D8, residual 43).

v0.7.0

Choose a tag to compare

@github-actions github-actions released this 08 Nov 07:39
Immutable release. Only release title and notes can be modified.
ef7b8ff

Three checks on what the next agent will read and on whether the change is tested. The prompt-injection scan, signals
PI01 to PI05 (INJECTION_PI01 to INJECTION_PI05, the report's injection object), flags content a pull request adds
that is aimed at an agent: invisible or bidirectional Unicode, hidden Markdown that addresses an agent, known injection
phrases, long encoded blobs in an instruction file and an instruction file added in a vendored directory. Stale agent
instructions, signals SI01 to SI04 (SETUP_STALE_SI01 to SETUP_STALE_SI04, setupHealth.stale), flag an instruction
file that names a script, make target, path or file that does not exist, or that is old and names a file the change
edits. TESTS_NOT_EXERCISING_CHANGES warns when no test file the pull request adds or modifies imports a changed JS/TS
source file. A behaviour change: PI01 is an error by default, so a pull request that adds a zero-width character, a
bidi override or isolate, a tag character or a stray byte order mark now fails the check where it passed; test files,
fixtures/, __fixtures__/ and testdata/ are exempt, and an app policy can exempt paths (injection.exempt) or set
PI01's level (injection.signals). A pull request that changes JS/TS source without a test importing it now gets a
warn verdict where it got pass; it never fails the check. The CLI and the Action now read the committed text of the
agent instruction files, the root package.json and Makefile on every run with a local diff, with or without an org
policy, for the stale-instruction signals (Security below). Report v2 gains the optional injection, setupHealth and
testIntegrity.exercise fields and the new message codes; no field, input or code was removed or renamed.

Added

  • Tests exercise the changed lines (A8 step 1, TESTS_NOT_EXERCISING_CHANGES, warn): a changed JavaScript or
    TypeScript source file counts as exercised when a JS/TS test file the pull request adds or modifies imports it
    directly with a relative import and keeps or raises its assertion count. When at least one changed source file is not
    exercised, the report carries one warning with the counts and up to 10 paths, the new optional
    testIntegrity.exercise lists up to 200 paths, and the comment shows one line under "Test integrity" with a collapsed
    list. It is a warning in every test-integrity mode and never fails the check, so a pull request that changes code
    without a test now gets a warn verdict where it got pass. Test files, declaration and config files, build output,
    vendored, minified and generated files and coverage.exclude paths are not asked; import aliases, workspace packages,
    re-exports and per-test coverage are not followed (docs/LIMITS.md). It reads only the touched test files' text the
    signals already read, and is silent in mode off, without a diff and without that text. Report v2 gains the message
    code and the optional field; nothing was removed or renamed.
  • Stale agent instructions, signals SI01 to SI04: on a change that touches an agent instruction file (CLAUDE.md,
    AGENTS.md, .cursor/rules, .github/copilot-instructions.md and the other instruction kinds), or that deletes,
    renames or removes what one names, pharn-ci reports a backticked command whose script is not in the root
    package.json or whose target is not in the root Makefile (SI01, SETUP_STALE_SI01), a backticked path that is not
    tracked (SI02), a linked or @-imported file that is not tracked (SI03), all warn, and an instruction file last
    changed more than 90 days before HEAD's commit that names a file the change edits (SI04, info). Report v2 gains the
    optional setupHealth.stale list and the four codes; the PR comment gains a 📄 Instructions section. No finding
    fails the check, and without a diff nothing is checked.
  • Prompt-injection surface scan (H4), signals PI01 to PI05: the lines a pull request adds are scanned for content aimed
    at the next agent that reads the repository. PI01 (default block, so it fails the check) flags invisible and
    bidirectional Unicode (zero-width characters, bidi overrides and isolates as in Trojan Source, tag characters, a
    stray byte order mark), with right-to-left marks, Persian and Indic joiners, emoji sequences and locale files left
    alone; PI02 hidden Markdown (HTML comments, reference definitions, link titles) that addresses an agent; PI03 known
    injection phrases from a versioned list (PH01 to PH32); PI04 a base64 or hex blob over 200 characters in an agent
    instruction file; PI05 an instruction file added under node_modules/, vendor/, dist/ and similar (PI02 to PI05
    warn). Test files, fixtures/, __fixtures__/ and testdata/ are exempt. Findings go into the new optional
    injection object of the report and the comment's 🧬 Content section, as fixed words, phrase ids and code points,
    never file text. New message codes INJECTION_PI01 to INJECTION_PI05 and INJECTION_TEXT_TRUNCATED (a file too
    large to scan in full), and a new injection policy key (exempt, phrases, signals): an app policy can exempt
    paths, add phrases and set severities; a file: policy can only add phrases and raise severities. Diff-only: no file
    read, no new input, no new dependency, no network call.

Security

  • The CLI and the Action now read the committed text of the agent instruction files (at most 50, 256 KiB scanned
    each), the root package.json and Makefile, and git log times of up to 20 instruction files on every run with a
    local diff, with or without an org policy. Only inline code, shell blocks, link targets and @ imports are read; a
    finding names a path or command made of repo-path characters only, and nothing else of the files leaves the job
    (docs/THREAT-MODEL.md row C8, residual 40).

v0.6.0

Choose a tag to compare

@github-actions github-actions released this 08 Nov 01:31
Immutable release. Only release title and notes can be modified.
6d49c27

Two org-policy rules for agent pull requests, and a .pharn/ci.yml that may only tighten. P13, agent
self-modification (POLICY_P13), blocks an agent pull request that widens its own or another agent's powers in an
agent settings file, an MCP config, a workflow or CODEOWNERS. P14, the agent PR size cap (POLICY_P14), fails an agent
pull request that changes more files or lines than the org policy's agents.max_files / agents.max_changed_lines;
it is opt-in, so a policy that sets neither sees no change. A behaviour change: a stricter coverage.threshold or
setup.require: true in .pharn/ci.yml now applies over a looser threshold input (or --threshold) or
require-agent-setup: false; to get the looser value back, lower it in .pharn/ci.yml itself. An added test file
pharn-ci cannot read is now named in the TEST_INTEGRITY_INDEX_UNAVAILABLE warning instead of dropped. Report v2 gains
the two message codes; no field, input or code was removed or renamed.

Added

  • Org-policy rule P13, agent self-modification (POLICY_P13, default block): under an app or file: policy, an
    agent pull request that widens its own or another agent's powers is blocked. The Action compares each agent settings
    file, MCP config, workflow and CODEOWNERS file the diff touches between the merge base and HEAD (read from committed
    objects with the caps of the test sources): an allow rule added, a deny rule removed, defaultMode changed, a hook
    removed or rewritten, disableAllHooks or enableAllProjectMcpServers turned on, an MCP server added or changed, a
    secret newly referenced, a workflow permission widened or its block removed, pull_request_target added, or a
    CODEOWNERS change other than owners added. One violation per file names the widening in fixed words, never file
    text. A missing diff, or a file that cannot be read or parsed, fails closed. A person's pull request is never
    subject to it, a file: policy cannot lower it, and the CLI never runs it. New message code in schema v2, no new
    input, no new dependency, no network call.
  • Org-policy rule P14, agent PR size cap, with the new message code POLICY_P14: an agent pull request that changes
    more files than agents.max_files or more lines (added plus removed) than agents.max_changed_lines fails, with one
    path-less violation per exceeded cap that names the count, the cap and the top-level directories touched ("split it
    by area: …"). It is opt-in: set agents.max_files / agents.max_changed_lines to turn it on. There is no default
    cap, so a policy that sets neither sees no change. Lockfiles, snapshots, *.generated.* files and coverage.exclude
    paths do not count (coverage.exclude is not used when the pull request edits the config in use or a workflow,
    unless the app policy sets the exclusions). A person's pull request is never subject to it, a missing diff fails
    closed, the severity is block and a file: policy can lower a cap but never raise or remove one. Report v2 gains
    the code only; no field changes.

Changed

  • .pharn/ci.yml may now only tighten the coverage threshold and the agent setup requirement, as it already did the
    test-integrity mode. A stricter coverage.threshold now applies over a looser threshold input or --threshold
    flag (with both set, the larger applies; it was the input's), and setup.require: true now stays on when the
    require-agent-setup input is false (it was turned off). This is a behaviour change: a workflow that set a lower
    threshold or require-agent-setup: false to relax a stricter config now gets the config's value. An org policy that
    raises the threshold over a config value that was in force names config in POLICY_OVERRIDE_IGNORED. The file,
    root and format inputs still replace the config's, and exclusions are still the config's plus the input's. No new
    message code and no schema change.
  • An added test file pharn-ci cannot read (the test index refuses it as syntax-error-heavy, too-deep or
    parser-threw, its text was dropped or missing at HEAD, or there is no Python index for a .py file) is now named in
    TEST_INTEGRITY_INDEX_UNAVAILABLE with its reason code, in a sentence of its own after the TI02 and TI05 one: "The
    tests in these added test files could not be read, so they are not checked: …". It was dropped with no trace, so
    its tests were checked by nothing and the report did not say so. The message stays a warning in every mode, is not
    given in mode off, and is unchanged when no added file is unread. No new message code and no schema change.
  • TI01's detail for a deleted test file with a same-stem test file added now says the base text "could not be parsed"
    when it was read but the test index refused it, instead of saying it was not available. The severity is unchanged.

v0.5.0

Choose a tag to compare

@github-actions github-actions released this 03 Nov 21:22
Immutable release. Only release title and notes can be modified.
cb87afd

The fixes of the 2026-11-01 review. Test integrity in block fails closed when it cannot see the change, TI03 blocks
the ways a skip was written to slip past it, unreadable agent settings and MCP files are errors, and the upload to
pharn.dev is bounded in time and never sent from a pull_request_target run. The Action input fail-on-threshold is
renamed fail-on-verdict; the old name still works, with a warning. Some warnings become errors on purpose: see
Changed and Security. No schema field or message code was added, removed or renamed.

Added

  • The Action input fail-on-verdict, the new name of fail-on-threshold (review 2026-11-01, finding 27): it gates on
    the verdict, whatever made it fail, not only on the coverage threshold. fail-on-threshold still works when
    fail-on-verdict is not set, with a deprecation warning, and is ignored (with a warning) when both are; both stay
    declared in action.yml, which no longer gives either a default (the default, true, is applied in code, so a set
    old name is never shadowed).

Changed

  • The Action applies two things the pharn.dev app contract already named (review finding 15 f and g). No schema change,
    no new input, no new message code and no new host.

    • The app's settings.messageMaxBytes now lowers the upload's per-message cap: the cap is the smaller of 2048 bytes
      and that setting, taken only from a policy response that validated whole in the same run (never under
      policy: never or a file: policy). The app's other settings (uploadJunit, redactMessages) and its
      quarantine list are validated and still not applied.
    • meta.mergeBase is recorded for a run with a local diff: the merge base of the resolved base and meta.headSha
      (the pull request's head, so on a pull_request run the fork point, not the base tip), one extra local
      git merge-base per run at most. Every other run, and a git merge-base that fails or prints no object id (a
      shallow checkout), records null and changes nothing else. Matrix legs of one job remain a documented limit
      (docs/LIMITS.md).
  • The Action's packages/action/dist/index.js shrinks from about 11.7 MB to about 1.7 MB: the TypeScript parser no
    longer sits inside it. scripts/bundle.mjs copies typescript@5.9.3's lib/typescript.js byte for byte beside the
    bundle as packages/action/dist/typescript.js (9.1 MB, the dist's second committed file), and the Action loads it
    only when a pull request makes it parse a test file or test config, after checking its sha256 against a pin in
    packages/action/src/vendored-typescript-pin.ts. A file that does not match the pin fails the step with an error
    naming the file and both hashes, and nothing is reported. A run that parses no test file never reads it. The CLI
    still inlines the parser. No schema change and no new message code. See docs/THREAT-MODEL.md, G1.

  • One source for three text rules in @pharn-ci/core (text.ts): the code-unit comparator byCodeUnit (about 50
    copies, in core, node-io and the tests, are gone), the surrogate-safe cut cutCodeUnits (the … inside the limit,
    an option for a cut without it, never inside a surrogate pair) and utf8ByteLength. No limit number, message code
    or schema changes. Two byte-level effects: a TI09 value (base/head) is now cut to at most 80 code units, …
    included (it was 81), and a key text to at most 100 (it was 101); and the three cuts that could leave a lone
    surrogate (TI01 and TI06 details, the 500-unit test name) no longer do.

  • The Action's run.ts items of the 2026-11-01 review. No schema change and no new message code.

    • A PR comment or job summary cut as a last resort (the messages alone over the limit) now closes every collapsed
      <details> section it cuts into before the truncation note, so the note is no longer hidden inside it (37).
    • Fewer GitHub API calls (47): a pull request is read once per run (in pr-number mode it was read up to three
      times: the target, the agent PR detection and the accept label), and the comments and the token's login are
      listed once for the upload's comment link, the pointer and the sticky comment (they were listed twice when an
      upload was tried, including when the app answered 404). A failed listing is still asked again by the next
      reader. The config and base-ref inputs are read once, and the coverage file is read from the inputs after the
      org policy, as the CLI reads it.
    • The base is decided once for the diff and for report.json (45): meta.baseSha is now the base the diff was taken
      against, recorded only when it is a full commit SHA. With a base-ref branch name, a pull_request run used to
      record the PR base and a push run the previous commit; both now record null. A 64-character (SHA-256) commit
      SHA as base-ref is no longer also tried as origin/<sha>.
    • The warning for a truncated PR-head file list in pr-number mode names what may miss files: the agent setup scan,
      the secret scan and the org policy's forbidden-file rule (P08) (48). docs/THREAT-MODEL.md residual 36 records
      why a bot's marker comment still counts as pharn-ci's own when the token's login is unknown: no API names an
      installation token's own login.
    • An unexpected exception in the upload is still failed with the reason internal and never changes the check,
      and its log line now names the error class (never its message); with debug logging (RUNNER_DEBUG=1) it then
      fails the step with the message, so it can be found (28). A 413 from the app is still read without its body:
      the app's API defines none, so maxBytes stays null (29, docs/LIMITS.md).
    • Not changed: finding 39 (the two test-sources tests vi.mock("@pharn-ci/core")) is dropped. The mock is a pinned
      spy-through to the real check(); replacing it would delete the test that pins it, which TI02 correctly blocks.
  • Docs and metadata (second review, 2026-11-01): README says nothing phones home only without id-token: write, the
    quick precedence sentence no longer claims .pharn/ci.yml "can only tighten" (only the test-integrity mode is
    merged; an input replaces the config's threshold, file, root, format and setup.require), the token-footprint
    bullet names the accept-label reads, and the AI-agnostic and "fully stateless" sentences name P11 and the opt-in
    upload. SECURITY.md best practices name what id-token: write turns on. docs/THREAT-MODEL.md §1 names the
    pharn.dev app, its responses, app-url and the OIDC token, and §3 gains item 37 (the threshold comes from the PR).
    docs/LIMITS.md §1 points at TI02, TI05 and TI06 instead of "not detected yet", gains "What P01 misses", and says
    fail-on-threshold: false un-gates every cause of a fail verdict; action.yml says so too, and its
    github-token text names the commits read and its fail-closed agent-PR detection. AGENTS.md describes Report v2,
    the schemas, the Python index subprocess and the opt-in pharn.dev calls. docs/MARKETPLACE.md no longer says the
    listing is refreshed on each release: the Action is not listed yet. CONTRIBUTING.md says why secret-pattern
    tests build tokens at test time.

  • Release verify refuses a run whose HEAD is not the chore: release X.Y.Z (#N) merge commit, so a tag can no
    longer land on a later PR (v0.2.0 was tagged four PRs after its release PR).

  • The skylos-nine fixture pins now cover fixtures/skylos-nine-pytest/ and fixtures/skylos-nine-sources/ too, and
    name the commits and reasons of the two change.diff edits made before the pins existed (docs/PRECISION.md §3).

  • Test-only, recorded late: 0.4.0 (#111) moved the growth tests to CPU time and raised four vitest timeouts to 300 s
    (ac-ti0205-growth, ac-ti06-ti12-linear, ac-py-index-growth, limits); the growth ceilings themselves did not
    change.

  • An org policy that requires a deny rule or a hook (setup.required_permissions_deny or setup.required_hooks,
    with P03 or P06 at block) now fails the check when the agent settings files could not be read, instead of warning
    (second review, HIGH 4). The files are unread whenever the diff came from the compare API (a shallow checkout, the
    actions/checkout default; pr-number mode; a failed local diff), so before this a default checkout turned the
    policy's block into a warning and the step passed with the requirement unchecked. POLICY_SETTINGS_INVALID is now
    error there, as it already was under policy.required; use fetch-depth: 0. A rule the policy sets to warn or
    off keeps the warning.

  • TI10 reports a test script replaced by a lone true or : ("test": "true", or true || vitest run) as forcing a
    successful exit, like exit 0 (second review, HIGH 5). docs/LIMITS.md lists what TI10 still misses:
    node -e "process.exit(0)", a positional path filter, --dir and --project.

  • .pharn/ci.yml refuses a __proto__ key, bare, quoted or nested: it set the prototype of the parsed map, so a
    coverage block under it escaped the unknown-key check (second review, finding 26).

  • The PR comment shows the accepted line once: TEST_INTEGRITY_ACCEPTED is no longer repeated under Messages when the
    Test integrity section shows the accepted line, and its hint moves under that line (finding 32).

  • TEST_INTEGRITY_MODE_IGNORED names the config in use when the config input points elsewhere than
    .pharn/ci.yml; the upload bundle redacts secrets in object keys too; action.yml states the python3
    requirement; the comment renderer uses the shared unsafe-character set, and the Action reads logins with core's
    LOGIN_PATTERN (findings 45, 46 and 48, cleanups). No schema change.

  • Tests (findings 12, 35, 41): the uploaded run's report equals the offline one apart from meta.upload; every
    declared input and output must be in the never-removed list (test-integrity and accept-label were...

Read more

v0.4.0

Choose a tag to compare

@github-actions github-actions released this 27 Oct 10:33
Immutable release. Only release title and notes can be modified.
a9314e9

Org policy and the app link-up. The Action can fetch an org policy (policy input) and upload the redacted run bundle
(upload input) to the pharn.dev app, a new policy mode reads agent settings and MCP configs (rules P01–P11), agent PRs
are detected (meta.isAgentPr), and check A7 scans agent configs for secrets. With no policy and no app the report is
the one 0.3.0 produced; nothing was removed or renamed. See Security for the new network surface and file reads.

Added

  • Org policy rules P02–P06, and policy mode: with a policy in force the CLI and the Action read the tracked agent
    settings files (.claude/settings.json and .claude/settings.local.json, root and nested) and, when the policy lists
    MCP servers, .mcp.json, .cursor/mcp.json and .vscode/mcp.json, at HEAD. P02 flags an allow rule that overlaps
    a forbidden permission, defaultMode bypassPermissions or auto, and additionalDirectories; P03 a missing
    required deny rule; P04 an MCP server that is not allowlisted; P05 a forbidden MCP server; P06 a missing required
    hook. New message codes POLICY_P02 to POLICY_P06 and POLICY_SETTINGS_INVALID (a file the rules could not fully
    use, or did not read); P09 now judges those edits as a block, warn or info. Nothing read from a file is printed.
  • meta.isAgentPr in the Action's report, with meta.prAuthor, meta.prLabels, meta.coAuthors and meta.agentHint:
    whether the pull request counts as an agent PR, from its author, its commits' authors, committers and
    Co-authored-by trailers, and its labels (agent and ai-generated by default). Seven known agents are recognized.
    Evidence that cannot be read in full (an unknown author, unreadable commits, more than 250 commits) counts as an agent
    PR. Metadata only: the verdict, the messages, the comment and the exit code do not change. In core: detectAgentPr,
    parseCoAuthorTrailers, agentPrMetaFields, withAgentPrMeta, KNOWN_AGENTS and DEFAULT_AGENT_LABELS.
  • schema/policy.v1.json: the JSON Schema (draft 2020-12) for version 1 of the policy document, the file that will
    tell pharn-ci what "safe to merge" means for a repository, shared with the pharn.dev app. A pure validator in core
    (validatePolicy, and parsePolicyJson for JSON text) checks a document against it: it never throws, never
    repeats a value from the document in an error, and bounds its work on any input. The org policy entry below is
    what reads it.
  • Org policy. The Action takes two inputs, policy (auto, never, required or file:<path>, default auto) and
    app-url (default https://pharn.dev), and the CLI two flags, --policy-file <path> and --no-policy. With
    policy: auto or required and an OIDC token (permissions: id-token: write), the Action makes one GET of the
    pharn.dev policy endpoint, sending only the OIDC token and the repository name, and applies the served policy over
    the workflow inputs and .pharn/ci.yml: it raises the threshold, setup.require and the test-integrity mode,
    replaces the exclusions and the coverage root, sets per-signal severities and the accept label, and adds
    setup.expect tools. A file: policy carried by the pull request can only tighten. Two new message codes:
    POLICY_UNAVAILABLE (warn, error under policy: required) and POLICY_OVERRIDE_IGNORED (warn). With no
    token, policy: never or no policy, the report is the one it was.
  • Check A7, secrets in agent configs. The CLI and the Action read the committed text at HEAD of the
    machine-readable agent setup files (.mcp.json, .cursor/mcp.json, .vscode/mcp.json, .claude/settings*.json,
    .codex/config.toml, .gemini/settings.json, opencode.json, .aider.conf.yml, the files under .claude/hooks/ and
    the .json, .yaml and .yml files under .continue/rules/; never the Markdown kinds) and report ten shapes of
    secret (GitHub, Anthropic, OpenAI, Google, AWS, Slack, Stripe, private keys, URL passwords and tokens, and secret-like
    values under secret-named keys). Twelve message codes, AGENT_CONFIG_SECRET_S01 to AGENT_CONFIG_SECRET_S11 and
    AGENT_CONFIG_SECRET_UNSCANNED: S01 to S09 are error and fail the check, S10 is warn, S11 is info, and files that
    could not be scanned are named in the unscanned code (error when a drop could hide a secret). Report v2 only gains
    codes; the message-code description in schema/report.v2.json names them. In the Action, the files are read only for
    a diff from the local checkout; in PR-number mode and for a compare-API diff the message says nothing was read.
  • A redaction module in @pharn-ci/core: findSecretSpans, redactSecrets, truncateUtf8, redactMessage and
    MESSAGE_MAX_BYTES. Nothing outside check A7 uses it yet.
  • Upload of the run to pharn.dev. The Action takes an upload input (auto or never, default auto) and sets a
    run-url output. With auto, a job that grants id-token: write and an organization that installed the Pharn
    GitHub App get the redacted report.json and a new meta.json uploaded; the App then comments on the pull request,
    so the Action posts no comment of its own, and a 404 from the app puts an install hint under the comment and the
    summary. A failed upload never fails the check; a report over the limit adds the UPLOAD_TOO_LARGE message code
    (warn). schema/upload-meta.v1.json describes meta.json. In core: buildUploadMeta, buildUploadBundle,
    buildUploadTar, withUploadOutcome and UPLOAD_REPORT_MAX_BYTES.
  • Org policy rules P01, P08, P09 and P10. A policy that sets setup.require or setup.expect (P01) raises
    NO_AGENT_SETUP_FILES and AGENT_SETUP_EXPECTED_MISSING to the rule's severity; a tracked file that matches a
    setup.forbidden_files pattern (P08) is a new POLICY_P08 error; a pull request that edits the policy file it is
    checked against, the config or a workflow while it sets a value looser than the policy, or an agent settings file
    the policy checks (P09) is a new POLICY_P09 error (a warning for an agent settings file); an unreadable policy
    (P10) stays POLICY_UNAVAILABLE. policy.rules sets each rule's severity, and a file: policy can only raise it.
    The findings are in the report's policy object and a new Policy section of the PR comment; without a policy the
    report and the comment are the ones they were.
  • Org policy rule P11, an agent pull request that changes a protected path. A policy that lists agents.protected_paths
    makes every path of an agent PR's diff that matches one of the globs (a rename counts by both its paths) a P11
    violation, and one new POLICY_P11 message names up to 20 paths and the evidence that made the pull request an agent
    PR, as fixed words. agents.protected_paths_mode and policy.rules.P11 set the severity (block by default; a
    file: policy always blocks); a person's pull request is never subject to it, and a missing diff fails closed. The
    policy's agents.labels and agents.extra_logins now add to the default agent labels and logins. A pull request
    that counts as an agent PR only because its evidence could not be read gets one AGENT_PR_DETECTION_INCOMPLETE
    message (warn) that says why. CLI runs have no pull request, so P11 never fires there.
  • Org policy rule P07, agent instruction files that disagree. Under a policy, when two instruction files of one
    directory (CLAUDE.md, AGENTS.md, GEMINI.md, .cursor/rules, .claude/rules, .github/copilot-instructions.md
    and the other instruction kinds) name different package managers (npm, pnpm, yarn, bun) or test runners
    (vitest, jest) in inline code or shell blocks, each is a P07 violation with its path, its line and the tools as
    fixed words, and one new POLICY_P07 message names up to 20 files (warn by default; setup.consistency and
    policy.rules.P07 set the severity, and a file: policy can only raise it). Files that could not be read or compared
    are reported too. Without a policy, or with P07 switched off, nothing is read and the report is the one it was.

Changed

  • Agent PR detection runs on every pull request run, not only on runs whose report carries meta, so the Action reads the
    pull request's commits (GET /repos/{owner}/{repo}/pulls/{n}/commits) on every pull request run. On its own the flag
    still changes no verdict, message, comment or exit code; under a policy with protected paths, P11 uses it.

Security

  • A file: org policy that the pull request edits is also read at the merge base, from committed git objects only
    (never the working tree, no symbolic link followed, at most 1 MiB), and its labels, logins and protected paths count
    together with the pull request's own: a pull request cannot remove a protected path or an agent label by editing the
    policy file. When the merge-base version cannot be read, an edited policy file makes the pull request an agent PR.

  • The Action makes one new read-only GitHub API call, GET /repos/{owner}/{repo}/pulls/{n}/commits (at most 3 pages of
    100), and reads the text of the pull request's commit messages in memory for the agent PR detection. A message is
    capped at 65,536 UTF-16 code units, read for at most 50 trailers in linear time, and never stored, logged or sent;
    only fixed words, valid logins and the trailers of known agents reach the report. docs/THREAT-MODEL.md has the new
    surface A5 and residual risks 22 and 23.

  • No A7 message, and nothing else the report, the comment or the summary carries, names a secret's value, its key or
    any other text of an agent config file: a message holds the shape id, a path in a code span and a line number.

  • The upload is a new network surface of the Action: two POST requests to the app origin and one PUT to a signed
    storage URL, only with upload not never and a job that grants id-token: write. Only the redacted report.json
    and meta.json leav...

Read more

v0.3.0

Choose a tag to compare

@github-actions github-actions released this 26 Oct 12:33
Immutable release. Only release title and notes can be modified.
ad3b75f

Test integrity now reads the tests themselves. Nine new signals (TI02, TI05, TI06, TI07, TI08, TI09, TI12, TI15 and
TI16) compare what a change does to its test files, test-runner configs and sources, for JS/TS and Python alike, and the
report lists the tests a change adds, removes, modifies or renames. With test-integrity: block, a pull request that
passed under 0.2.0 can now fail: see the first entry under Changed for what, and how to get the 0.2.0 outcome back.
No input, output or message code was removed or renamed, the default mode is still warn, and the report is still
Report v2 with two optional fields added.

Added

  • Signal TI02, a test removed, and the code TEST_INTEGRITY_TI02. With TI05, the first signals that compare the
    text of the test files a change modifies at the merge base and at HEAD (JS/TS and, through the Python test index,
    .py files). TI02 reports a test with no counterpart left in the touched test files by name or by a similar body,
    block in mode block and warn in mode warn and for a computed name, at its base path and base line. It clears a
    test renamed in place in its describe, hand-written tests replaced by an it.each table, and a test moved to another
    touched file under a new name when its body closely matches. No rule clears a test by its name alone: a removed test
    is cleared by the same leaf name in another touched file only when that test holds the body.
  • Signal TI05, assertions emptied or dropped, and the code TEST_INTEGRITY_TI05. It reports a test that had
    assertions and has none at head (block in mode block), or fewer with the rest of its body unchanged (warn), at
    its head path and head line.
  • Signal TI06, a test weakened, and the code TEST_INTEGRITY_TI06, always warn. TI06 compares each matched test at
    the merge base and at HEAD and reports a strong matcher replaced by a weak one (toBe to toBeTruthy), an
    expect.assertions or expect.hasAssertions guard lowered or removed, an assertion that compares a literal with
    itself added, and an assertion moved into a try block with a catch or after an early return. It reads the
    subject of expect(…) for toBe, toEqual and toStrictEqual, so typeof, instanceof, .length, !!x, null
    and undefined rewrites to the matcher they equal are not a weakening. The detail is fixed text and matcher labels,
    never a test name.
  • Signal TI07, an expected value edited to match, and the code TEST_INTEGRITY_TI07. It warns, in modes block and
    warn alike and so never fails the check, when a test of a modified JS/TS or Python test file changes only literals
    inside the expected-value arguments of its assertions (the Jest and Vitest matchers, chai's equal, node:assert's
    assert.equal family, unittest's assertEqual family and the operand after == in a pytest assert) and the change
    also edits a JS/TS or Python source file. The finding names the test and the matcher, never the literal.
  • Signal TI08, snapshot churn, and the code TEST_INTEGRITY_TI08. It warns, in modes block and warn alike, when a
    change modifies, deletes or rewrites more than 10 snapshot files (__snapshots__ directories, .snap and .ambr
    files), when a JS/TS test file rewrites 10 or more toMatchInlineSnapshot( lines, or when a snapshot-update flag
    (-u, --update, --updateSnapshot, --update-snapshots, --snapshot-update) is added to a test* script in
    package.json or --snapshot-update to a pytest config file.
  • Signal TI09, a test-runner setting loosened, and the code TEST_INTEGRITY_TI09. TI09 compares the base and head
    text of the touched test-runner config files: vitest.config.*, vite.config.* (now read as a test-config file by the
    CLI and the Action), vitest.workspace.*, jest.config.* and the jest key of package.json, playwright.config.*,
    .mocharc.*, pytest.ini, tox.ini, setup.cfg and pyproject.toml. A fixed list of keys is judged: a lowered
    or removed coverage threshold, a narrowed test list, a widened exclusion (build directories, node_modules and
    .d.ts files excepted), passWithNoTests, allowOnly or bail turned on, a grep added, forbidOnly turned off
    and a pytest addopts that gains a deselecting option are block in mode block and warn in mode warn; a raised
    retry count or timeout, and a value that is not a literal, are always warn. The projects of a vitest or jest projects list are matched by name
    (test.name, displayName) or by position and only their test-list keys are compared, so an unrelated key changed in
    a project gives nothing. The finding sits at the head path and the head line of the key (no line for JSON). A config
    that cannot be read, or the whole signal when a side of the test sources is unavailable, adds a TI09 sentence to
    TEST_INTEGRITY_INDEX_UNAVAILABLE.
  • Signal TI12, a module under test mocked, and the code TEST_INTEGRITY_TI12, always warn. TI12 reports a new
    vi.mock, jest.mock or Python patch of a source file the same change edits, once per test file and target, with
    the mock's specifier sanitized and cut to 120 characters.
  • Signal TI15, the test may be special-cased, and the code TEST_INTEGRITY_TI15, at level info: it never blocks or
    warns, in modes block and warn alike. It fires when a condition added to a JavaScript, TypeScript or Python source
    file (===, !==, ==, !=, a case label or a Python in (...) group) compares with a string or number literal
    that a test file in the same change also adds. It matches only inside the same package (the nearest package.json or
    pyproject.toml), leaves out examples, benchmarks, bench, docs and fixtures directories, needs 6 characters
    or 4 significant digits, and lists at most 10 findings per change (the message says how many more). The finding names
    the source line, the literal's kind and length and the test file, never the literal's text. It is info because 0 of
    30 sampled TI15 findings were right on the 600-PR precision corpus (docs/PRECISION.md).
  • Signal TI16, a lint or type suppression in tests, and the code TEST_INTEGRITY_TI16. It gives an info finding,
    which never blocks and leaves the verdict at pass, when a JS/TS or Python test file adds a comment that switches off
    a linter or type checker.
  • At most 20 findings per file and signal are listed for TI02, TI05, TI06, TI09 and TI12, and the messages say how many
    were left out.
  • The code TEST_INTEGRITY_INDEX_UNAVAILABLE (warn). It names the touched test files that could not be compared, with a
    reason code each, or says that TI02 and TI05 did not run.
  • Tests-changed summary: testIntegrity.changes in report.json (optional, in schema/report.v2.json) lists the tests a
    change adds, removes, modifies or renames, with counts per test file, and the comment gains a collapsed "Tests changed"
    table with the counts per test file. Test names are in report.json only, cut to 120 characters, at most 200 entries.
    An .each replacement is listed once under renamed.
  • testIntegrity.languages in report.json (optional) lists the languages (js, py, ts) the test indexes parsed.
  • Python parity with JS/TS. TI03 and the Python test index resolve pytest marker and unittest decorator aliases
    bound by a test file's own top-level imports (import pytest as pt, from pytest import mark as m,
    from unittest import skipIf as si), so TI02, TI05 and TI06 see through them too; TI03 also reads
    @unittest.expectedFailure and pytest.xfail(. TI09 reads pytest 9's native [tool.pytest] table of
    pyproject.toml beside [tool.pytest.ini_options]. docs/LIMITS.md gains a Python parity table and the list of
    Python misses.
  • The official precision results for v0.3.0, in docs/PRECISION.md: 600 PRs, 0 errors, TI01, TI02 and TI11 each
    falsely block 1 of 600 PRs and ship as block, and the reviewed CSV is fixtures/corpus/review.csv.
  • The Skylos nine reproduced for pytest under fixtures/skylos-nine-pytest (eleven cases, the nine plus an aliased skip
    and a mock.ANY variant of the always-true matcher), pinned by tests that run check() with the real Python index.
    The signals catch 8 of 9 for pytest, and the vitest skylos-nine benchmark under fixtures/skylos-nine/ now
    catches 7 of 9, up from 2 of 9 in 0.2.0 (emptied-assertion, deselect-setting, replaced-test,
    loosened-assertion and edited-expected-value are new). docs/PRECISION.md gains a section with the case-by-case
    table, and docs/LIMITS.md gains the TI17 proposal for the always-true matcher (a proposal only, nothing implements it).

Changed

  • What can now fail a pull request in mode block that passed under 0.2.0: a removed test (TI02), a test left with
    no assertions (TI05), a loosened test-runner setting (TI09), a skip of an existing test written through an aliased
    pytest marker or unittest decorator, or @unittest.expectedFailure (TI03), and a deleted test file replaced by a test
    file of the same name whose tests are not found again by body (TI01, below). Mode warn, the default, reports the
    same findings without failing the check. To get the 0.2.0 outcome back on one pull request, a reviewer who is not its
    author applies the accept label (pharn/accepted, the accept-label input), which downgrades its block findings to
    warn; to get it back for every pull request, set test-integrity: warn.
  • TI01 no longer lets a name stand in for a test's code. A deleted test file replaced by a test file of the same
    name is info only when every test it held is found again by body in a new test, and stays a block when the tests
    are hollow or when no test sources are given (the two info-migration diffs checked diff-only are now a block).
    TI01 also gives info when every test a deleted test file held...
Read more

v0.2.0

Choose a tag to compare

@github-actions github-actions released this 25 Oct 10:24
Immutable release. Only release title and notes can be modified.
0873ce2

Added

  • The accept label, the accept-label Action input (default pharn/accepted, at most 50 characters). In
    test-integrity: block on a pull_request or pr-number run, a pull request that carries the label, applied last by
    a user account that is not the PR author, has its block signals downgraded to warn: the report gains the
    TEST_INTEGRITY_ACCEPTED message (info), testIntegrity.accepted and, for a non-default name,
    testIntegrity.acceptLabel, and the comment says who accepted it. A label that does not count (the author or a bot
    applied it, no labeling event, or anything unreadable) adds TEST_INTEGRITY_ACCEPT_IGNORED (warn) and the blocks
    stand. The Action reads the pull request and its issue events (at most 10 pages of 100) with pull-requests: read or
    issues: read; the CLI never honours the label, and test_integrity.accept_label in .pharn/ci.yml is refused. The
    TI03, TI10 and TI13 hints now name the label.
  • The test-integrity Action input and --test-integrity CLI flag (block, warn or off, default warn),
    and a test_integrity.mode key in .pharn/ci.yml that can only make the check stricter than the input. The
    report's testIntegrity.mode and testIntegrity.source say which mode applies and where it came from, a
    looser config value is ignored with the new TEST_INTEGRITY_MODE_IGNORED message (warn), mode off adds
    TEST_INTEGRITY_OFF (info), and the PR comment and job summary gain a "🔍 Test integrity" section.
  • Signal TI01, test file deleted: a deleted test file whose removed lines define tests, or a test file renamed to
    a path that is not a test file (edited or not, since a rename shows only the lines it edits), is reported in
    testIntegrity.signals and in one TEST_INTEGRITY_TI01 message,
    block in mode block (a failing check), warn in the default mode. It is info when the same change also
    deletes the source file of the same name, deletes other files of the test's package, or adds a test file of the
    same name (a migration); helpers, fixtures and an __init__.py under tests/ do not count. The "🔍 Test
    integrity" section lists the findings. Test files are now recognized by the spec's globs: .test. and .spec.
    with a JavaScript or TypeScript extension, .e2e.ts, .pw.ts, test_*.py, *_test.py, *_test.go and the
    test, tests and __tests__ directories. TEST_FILES_DELETED is unchanged and now uses the same detection.
  • Signal TI14, diff-coverage exclusions widened: a change to the pharn-ci config in use (.pharn/ci.yml, or the
    config input) that lowers or removes coverage.threshold, or adds a coverage exclusion, is reported in
    testIntegrity.signals and in one TEST_INTEGRITY_TI14 message, block in mode block, warn in the default
    mode. Raising a threshold, removing an exclusion and unrelated settings never fire. CONFIG_CHANGED_IN_DIFF is
    unchanged.
    The config lines are read as the config reads them: a quoted key, a quoted or escaped value and any number of
    decimals all count.
  • Signal TI11, CI stopped running tests, from the diff of workflow files (.github/workflows/*.yml|yaml),
    reported in testIntegrity.signals and in one TEST_INTEGRITY_TI11 message. block in mode block: a removed
    test command (pnpm test, vitest, jest, pytest, go test, ...) that no workflow adds again, and a removed
    pull_request trigger. warn in every mode but off: a shrunk flow-list matrix, an added
    continue-on-error: true, an added if: that is a literal false or leaves pull requests out, and a
    timeout-minutes of 0 or 1. Which step a line belongs to cannot be seen without context lines, hence the warnings.
  • Signal TI03, a skip added to a test file, from the kept text of the diff, reported in testIntegrity.signals
    and in one TEST_INTEGRITY_TI03 message. It reads JS/TS (.skip, xit and the other x heads, .todo, .fails,
    skipIf, runIf, this.skip(), Python (@pytest.mark.skip, skipif, xfail, @unittest.skip*, pytest.skip() and
    Go (t.Skip(, t.Skipf(, t.SkipNow(). An unconditional skip of a test the diff shows existed is block in mode
    block; a conditional skip, a skip inside a test body, and a skip in a test the change adds are warn. A skip that
    was already there (an edit, a move) never fires, and environment guards such as skipIf(!hasCamera) or
    pytest.skip("REDIS_URL not set") stay warn.
    At most 20 findings per file, with the skips of existing tests kept first, so warn findings never push a block out.
  • Signal TI04, a focused-run marker added to a JS/TS test file (.only, bare fit( or fdescribe(), reported
    in testIntegrity.signals and in one TEST_INTEGRITY_TI04 message, block in mode block, warn in the default
    mode. solver.fit(x), def fit(self), comments, strings and files that are not tests never fire; a marker that was
    already there (an edit, a move) does not fire either.
  • Signal TI10, a test script weakened, from the diff of a modified or renamed package.json (root or nested),
    reported in testIntegrity.signals and in one TEST_INTEGRITY_TI10 message, block in mode block, warn in the
    default mode. In a script whose key starts with test, an added || true, an echo or exit 0 as a command,
    --passWithNoTests, --testPathPattern and -t or --testNamePattern each fire once per line, unless a removed
    line of the same key already had that form; a removed pretest or posttest script fires too. --passWithNoTests
    stays quiet when no tracked file is a test file. Other scripts, an added or deleted package.json and a pure edit
    that keeps the weakening never fire.
  • Signal TI13, a coverage-ignore comment added to source, reported in testIntegrity.signals and in one
    TEST_INTEGRITY_TI13 message, block in mode block, warn in the default mode. An added Istanbul, V8 or C8
    ignore hint in a comment of a JS/TS file, or a # pragma: no cover in a Python file, that is not a test file fires;
    a marker that was already there (an edit, a move) never does.
  • The unified-diff parser keeps the text of added and removed lines per hunk for test files, the pharn-ci config in
    use, workflow files, every package.json and the JavaScript, TypeScript and Python source files (at most 5000 lines
    per file, 400 characters per line; textTruncated marks a cut file), for the signals that read line text. readDiffChunks takes an optional { configPath, repoRoot } so a custom
    config path works through the stream as it does through check().
  • The accept-label input also refuses invisible formatting characters (zero-width marks, bidirectional overrides
    and isolates, a byte order mark inside the name), which could make the hint around the label read differently from
    what it says. A spelling of pharn/accepted in other letter case is read as the default, so the report, the hints
    and the sign-off line name the same label. The comment shows those characters in a path or label as U+FFFD.
  • The PR comment and job summary follow the Report v2 layout. The summary table gains a Test integrity column (off,
    none (warn), or the number of findings and how many block), and the headline sentence counts the findings when
    there are any. The "🔍 Test integrity" section lists block findings first, then warn, then info, shows 10 and
    collapses the next 10 in a <details> block. The Messages section leaves out each TEST_INTEGRITY_TI* message
    whose findings the section lists, so a finding and its hint appear once; report.json keeps every message.
  • Precision of the diff-only signals, from a review of the base-rate fixtures. TI11 reads a quoted "on": key and the
    test commands npm t, pnpm -r test, pnpm --filter <pkg> test, pnpm vitest, pnpm exec, bun x,
    node --test, bun test, deno test, tox, nox and the uv, poetry, pipenv, pdm, hatch and rye run
    wrappers. TI10 reads ; true, || : and ; : as ignoring the exit status, no longer reads a banner
    (echo start && vitest run) as a weakening, and no longer reads mocha's -t (its timeout) as a name filter. TI13
    reads coverage.py's # pragma: nocover. TI03 reads a Python skip decorator above an edited def line, or above an
    existing test in a diff with context lines, as a skip of an existing test. TI01 no longer counts a deleted fixture,
    snapshot or data file under a test directory because its text was cut. A hand-made hunk header that starts a side at
    line 0 numbers its lines from 1.
  • The TEST_INTEGRITY_TEXT_TRUNCATED message (warn, not in mode off): a file that a test-integrity signal reads
    (a test file, a workflow, a modified package.json, a JavaScript, TypeScript or Python source file, or the pharn-ci
    config) has more than 5000 changed lines, so the signals read only the first 5000. The message names those files,
    so a change cannot pass in silence by padding a file past the cut. Deleted files are not named.
  • README: an Agent skills section on the pharn-ci plugin in
    pharn-dev/skills (install, explain a verdict, cover
    uncovered lines), and a pointer to it from the Quick start and the Pharn family table.
  • The Skylos nine benchmark: the nine ways Skylos hides a failing test, reproduced in vitest under
    fixtures/skylos-nine/ and pinned by tests. The diff-only signals of this release (TI01 and TI03) catch 2 of 9;
    the other seven are named with the planned signal that would catch them.
  • A "What test integrity does not catch" section in docs/LIMITS.md, and a "Test integrity"
    section in the README with a rendered example of a finding. The "Only added lines count" limit no longer says that
    skipped tests are not detected.
  • A release smoke script, scripts/smoke-release.mjs, that builds a throwaway repository from `fixtu...
Read more

v0.1.0

Choose a tag to compare

@github-actions github-actions released this 19 Oct 07:11
Immutable release. Only release title and notes can be modified.
a88e62a

Added

  • GitHub Action and pharn-ci check CLI.

  • Diff coverage: the share of lines added in the change that the coverage report (lcov or
    Cobertura XML) marks as executed, with an optional, inclusive threshold gate (threshold input,
    --threshold, coverage.threshold).

  • Agent setup inventory: the tracked agent setup files of Claude Code, Cursor, GitHub Copilot,
    Codex, Gemini, Windsurf, Cline and MCP, listed by kind and never read. The kinds are claude-md,
    claude-local-md, agents-md, gemini-md, claude-settings, claude-settings-local,
    claude-skill, claude-hook, claude-agent, claude-command, cursor-rules, cursor-mcp,
    copilot-instructions, copilot-instructions-scoped, copilot-agents, copilot-prompts,
    vscode-mcp, codex-config, windsurf-rules, cline-rules and mcp-config, plus, for Aider,
    Continue, Roo Code, Amazon Q Developer, Junie, Zed, Augment, Kiro, Goose, OpenCode and Gemini CLI
    and for .claude/rules/, claude-rule, aider-config, conventions-md, continue-rules,
    roo-rules, amazonq-rules, junie-guidelines, zed-rules, augment-rules, kiro-steering,
    goosehints, opencode-config and gemini-settings. .mcp.json, .claude/ files and
    .cursor/rules/ are found in nested directories, and the more specific directory rule wins over
    a file-name rule.

  • Report v1: optional agentSetup.summary (counts per kind and the tools involved), and an
    Agent setup: N files · tools line in the markdown, with the table collapsed above 25 files.

  • Report v1 (schema/report.v1.json), the stable JSON output, with message codes
    DIFF_COVERAGE_BELOW_THRESHOLD, DIFF_COVERAGE_UNGATED, NO_COVERABLE_CHANGES,
    COVERAGE_FILE_MISSING, COVERAGE_FILE_EMPTY, COVERAGE_FORMAT_UNSUPPORTED,
    DIFF_UNAVAILABLE and NO_AGENT_SETUP_FILES.

  • One sticky PR comment and a job summary; on fork PRs, where the token cannot comment, the job
    summary only.

  • .pharn/ci.yml config file, overridden by Action inputs and CLI flags.

  • Coverage exclusions: glob patterns (*, **, ?) from coverage.exclude, the CLI's repeatable
    --exclude <glob> and the Action's exclude input, merged (their union applies), leave matching
    changed files out of diff coverage. The report gains the optional diffCoverage.excludedFiles
    field, and the comment an "Excluded" summary listing them.

  • Cobertura XML coverage parsing in core, read by the CLI and the Action.

  • Coverage format selection: the coverage-format Action input, the --coverage-format CLI flag
    and coverage.format in .pharn/ci.yml take lcov, cobertura or auto. The flag or input
    wins over the config file.

  • auto is the default: the format is detected from the first 1 KiB of the file, and the CLI
    (stderr) and the Action (log) print which format was used, for example
    coverage: cobertura (auto-detected). Clover and JaCoCo native XML are not supported.

  • Message code COVERAGE_FORMAT_UNSUPPORTED: under auto, a coverage file that is neither lcov
    nor Cobertura XML is an error with a threshold, a warning without.

  • Report v1: optional diffCoverage.format (lcov or cobertura), the format the report was
    read as.

  • COVERAGE_LINES_SKIPPED (info): malformed lcov DA records and Cobertura <line> elements that were skipped while reading the coverage files are now counted, and the report says how many (a count only, no line numbers or file names). The verdict and the diff coverage numbers do not change, and reports that end before diff coverage is computed do not carry the message.

  • Coverage auto-discovery: with no coverage-file input, no --coverage flag and no coverage.file, the Action and the CLI look for the coverage report at coverage/lcov.info, lcov.info, coverage.xml, coverage/cobertura-coverage.xml and **/coverage.cobertura.xml, and print coverage: <path> (auto-discovered) for each file they read. More than one kind found is an error asking you to set the path; none found is a COVERAGE_FILE_MISSING message (previously a usage error). Complete workflows for vitest, jest, pytest, .NET and cargo are in examples/workflows/.

  • Expected agent tools: setup.expect in .pharn/ci.yml, the expect-agent-setup Action input and the CLI's --expect-agent-setup flag (repeatable) name agent tools (for example claude-code, cursor) that should have a setup file. Any tool no setup file configures is listed in the new agentSetup.missing field and reported as AGENT_SETUP_EXPECTED_MISSING (warn, or error when agent setup is required). A repo that expects nothing sees no change.

  • AGENT_SETUP_FILE_EMPTY (warn): a listed agent setup file that is empty (size 0 in HEAD, from git ls-tree -r -l) is reported with the first path, and stays in the inventory. When git ls-tree fails, none is reported (the CLI notes it on stderr, the Action warns). Because it is a warning, the verdict of a repo with an empty CLAUDE.md changes from pass to warn.

  • TEST_FILES_DELETED (info): the change deletes a test file, or renames one to a name that is not a test file
    (F-21). The message names up to five of them; the verdict is unchanged. Test files are recognized by name
    (.test./.spec. in the basename, or a test, tests or __tests__ directory).

  • Agent setup inventory covers more tools (F-11): 13 new setup kinds, claude-rule (.claude/rules/), aider-config, conventions-md, continue-rules, roo-rules, amazonq-rules, junie-guidelines, zed-rules, augment-rules, kiro-steering, goosehints, opencode-config and gemini-settings, so the files of Aider, Continue, Roo Code, Amazon Q Developer, Junie, Zed, Augment, Kiro, Goose, OpenCode and Gemini CLI are listed. agents-md is now also credited to amp, jules, opencode and zed, and agentSetup.summary.tools in Report v1 accepts new tool names within v1 (a lowercase name pattern replaces the closed list).

  • PR-number mode: the pr-number and head-sha Action inputs let pharn-ci measure and comment on a
    pull request from a workflow_run or workflow_dispatch run, with no PR code checked out. The PR,
    its diff and its file list come from the GitHub API (two new read-only calls: get a pull request,
    get a git tree), pr-number is validated, and head-sha is checked against the PR head. The README
    documents the two-workflow pattern for comments on fork PRs.

  • Three warn-level messages for a change that touches the inputs of its own grade: CONFIG_CHANGED_IN_DIFF (the diff edits the pharn-ci config in use or a file under .github/workflows/), COVERAGE_FILE_TRACKED (the coverage file is a tracked file, so the change can supply its own numbers) and DIFF_BINARY_COVERED_FILE (the diff shows a covered source file as binary, e.g. through -diff in .gitattributes, so its added lines were not measured). They lift a pass to warn and never fail the check.

  • Several coverage files and globs: the coverage-file Action input takes one entry per line, the CLI's --coverage flag is repeatable and coverage.file in .pharn/ci.yml can be a list. Each entry is a path or a glob (*, **, ?, relative to the workspace, the current directory or the repo root), so TestResults/**/coverage.cobertura.xml and packages/*/coverage/lcov.info work. The files are merged with hit counts summed; auto detects the format of each file. A glob that finds nothing is COVERAGE_FILE_MISSING; a walk stops at 1,000 matched files or 100,000 visited entries, never follows a symbolic link and skips .git and node_modules unless named.

  • Require agent setup: the require-agent-setup Action input, the CLI's --require-agent-setup flag and setup.require in .pharn/ci.yml (off by default) make NO_AGENT_SETUP_FILES an error and the verdict fail when the repo has no agent setup files.

  • DIFF_COVERAGE_UNGATED (info): diff coverage was measured, with its percent in the text, but no
    threshold gates it (none is set, or it is 0). Before, an ungated run passed with no message.

  • Data handling section in SECURITY.md, linked from the README's Safety model: what the Action reads, what it sends to your GitHub API, what lands in the PR comment, job summary, step log and report.json, and that nothing is sent to Pharn.

  • The rendered report (PR comment, job summary, CLI markdown) adds a one-line hint under each message
    saying how to fix it, and says what is missing (typically AGENTS.md or CLAUDE.md) when no agent
    setup files are found. Message text in report.json is unchanged.

  • coverage.exclude in .pharn/ci.yml can be written as a one-line flow list, e.g.
    exclude: ["**", "dist/**"], which used to stop the check with exit code 2.

  • Coverage root: coverage.root in .pharn/ci.yml, the coverage-root Action input and the CLI's --coverage-root flag name the repo-relative directory that relative coverage paths start from, so a package-relative lcov or Cobertura report in a monorepo is measured at the right files. Absolute paths are never remapped.

  • COVERAGE_PATHS_PARTIALLY_UNMATCHED (info): some of the coverage report's file paths are not tracked files, so their coverage is not used. The verdict is unchanged. The fixtures/golden and fixtures/golden-cobertura reports gain this message.

Changed

  • When more than one coverage file is read (a glob, a list, or several --coverage flags), the coverage: <format> (<how>) log line names its file: coverage: coverage/a.info: lcov (auto-detected). With one file the line is unchanged, and so is the (auto-discovered) line.
  • The PR comment, job summary and CLI markdown now show the overview sentence in a GitHub alert under the verdict heading: [!CAUTION] for fail, [!WARNING] for warn, [!TIP] for pass. Report data and message text are unchanged.
  • The PR comment, job summary and CLI markdown now show the pharn logo (a 48×48 image from the repository's raw main URL) above the verdict heading. Report data...
Read more