v0.1.1: a green check can no longer mean nothing was reviewed

What Gate is

Gate is a GitHub Action that runs a pull request's preview build inside a hardened sandbox, hands the verified preview URL to a critique service you supply, and publishes that service's design review back to the pull request as one sticky comment plus a Check Run. It judges and reports: it never edits code, never commits, never opens fix PRs, and never asks for contents: write.

Gate is the GitHub-facing half of a two-part system. It does not screenshot the page and does not run the vision model. Both sit behind an HTTP contract, and no implementation of that contract ships in this repository. verdict is a working one, and its matching v0.1.1 is released alongside this.

Upgrading from v0.1.0 is strongly recommended. At v0.1.0 Gate could publish a green check for a review that never happened. This release makes that impossible.

A green check can no longer mean nothing was reviewed

This is the change worth reading if you read only one.

A critique service returns a structured result with a grade in it, and a result with no findings grades ship. That is the right answer when the service looked at your pages and found nothing wrong. It is badly wrong when the service looked at nothing at all, because both cases produce the same empty result, and v0.1.0 published both as a passing check.

Two independent conditions now have to hold before Gate shows you a grade:

1. Something judged the page. The service stamps whether a model was actually involved. If it says no, or says nothing at all, Gate withholds the grade and publishes a neutral check titled Not judged or Judgment not stated, naming the field that would restore it.
2. Something was reviewed. The service now reports which routes and viewports it actually looked at, by name. A run that reviewed none of them gets a neutral check titled Nothing reviewed, which says in words that this is not a pass and not a failure, and lists every route it skipped with the reason.

Here is what the second case now publishes, instead of a green Ship:

No grade. The engine reviewed nothing: 0 of 2 requested routes were judged on this run, so the result below describes no page. This run is not a pass and not a failure.

A result with no findings grades ship by construction, which is why this is neutral rather than green: there was nothing to find because nothing was looked at. The routes and the reason each was skipped are listed below.

📋 Coverage: 0 of 2 route(s) reviewed; skipped /pricing, /checkout; viewports skipped: mobile, tablet, desktop.

A review that honestly covered part of your UI is still a real review, still passes, and now names what it missed:

Grade: Ship

No issues found.

📋 Coverage: 1 of 2 route(s) reviewed; reviewed /pricing; skipped /checkout; viewports skipped: tablet.

Not reviewed

• route /checkout (no preview deployment matched the head SHA)
• viewport tablet (not configured)

That distinction is deliberate. A partial review should not be punished for being honest about being partial; a review that covered nothing is not a review. The Check Run and the sticky comment apply the identical rule, so the two surfaces cannot disagree, and the Action's own run status reports the same thing as the check.

Setup mistakes now read as setup mistakes

At v0.1.0, several configuration errors surfaced as "The design engine is temporarily unavailable. The PR is not blocked; Gate will retry." That sentence is false for a wrong secret or a mistyped URL. Retrying sends the same wrong value.

• A wrong shared secret now reads Review not submitted ... HTTP 401 signature_mismatch on the pull request, carrying the service's own status and error code, what to check for that code, and no promise of a retry.
• An endpoint that is not a URL is caught before a review is attempted. That covers a bare verdict-acme.fly.dev (which is what the Fly, Render and Railway dashboards print at you), a typo'd htp://, and a host:port with no scheme. The neutral check is titled Engine endpoint invalid, shows the value it could not parse, suggests a corrected form, and says "This one does not clear by itself" rather than promising a retry that would change nothing.
• No critique service configured at all names the two variables you have to set, and says in words that this is not a pass.
• pnpm demo:live refuses a bad endpoint before starting a browser or a server, so you catch this on a laptop instead of in CI.

Every one of these still ends in a neutral Check Run. No failure of Gate's own is allowed to turn a pull request red.

The listing itself now says a critique service is required

action.yml's description read "Judgment-only design review for your PR preview deployments." It now reads "Judgment-only design review for your PR preview deployments. Requires a critique service you host; none ships here, and without one this action reviews nothing and says so." You see the requirement before you install, not after.

The README workflow now runs as written

The copy-paste workflow on the landing page referenced a deploy step that was never defined, and read .gate.yml out of a workspace it never checked out. Both are fixed. A test now finds every fenced workflow snippet in every markdown file in the repository and fails if any of them reads an output from an undefined step, reads the workspace without actions/checkout, or triggers on pull_request_target, so a snippet added to a new document later is covered without anyone remembering to add it.

Also in this release

• Dependency advisories cleared across the tree.
• Routes your critique service skipped are listed on the review, including routes dropped by a per-PR route cap.
• The pre-rename .designreview.yml config filename and the JUDGMENT_ENGINE_* variables still work, each with a one-line warning naming both the old name and the new one. This is a deprecation, not a supported alias.
• A result whose schema version does not match is reported as a setup error rather than as an outage.

Verified for this release

From a clean clone of the tagged commit: pnpm install --frozen-lockfile, pnpm build, pnpm typecheck and pnpm lint (eslint --max-warnings=0) all clean, and pnpm test passing at 102 test files, 717 tests, 0 failures. CI is green on the release commit.

Requirements

Node 24 or newer, pnpm 10.34.3, macOS or Linux. In the calling workflow, checks: write and pull-requests: write, and nothing more.

- uses: apatureai/gate@v1

The v1 major tag points at this release.

Honest limitations

None of these are fixed by this release, and they are worth reading before you wire Gate in.

• Half the system is behind an HTTP contract you have to implement. Every claim about screenshot quality, prompt design or finding accuracy belongs to the critique service. Gate's tests prove Gate's orchestration and delivery. They prove nothing about review quality.
• Review quality is entirely the service's. The judgment stamp answers "did anything judge this?", not "was the judgment any good?". A service running a real but bad model gets a real, bad review published verbatim.
• The coverage guarantee depends on the service reporting coverage. A service that omits the field keeps its grade, and the Check Run states in band that Gate cannot confirm the result covers your whole UI. A third-party service that both claims a model ran and omits coverage cannot be caught from the payload, and Gate says so rather than implying a completeness it cannot verify. verdict always reports coverage and refuses to serve a result without it.
• The Action path constrains hostile pull request code; it does not sandbox it. The ulimit caps, default-deny environment, loopback-redirect refusal and fork gating are real mitigations. The aggregate cgroup-v2 caps that would make them airtight are not implemented. Read the threat model before running this on a repository that accepts fork pull requests.
• The resource cap is Linux-only, and the process-count half of it needs bash: ulimit -v does not apply on macOS, and ulimit -u does not exist in dash.
• Windows is not supported. The supervisor relies on POSIX process groups.
• Nothing fails CI on a new dependency advisory. Both trees audit clean today, but no job enforces that, so the guarantee is only as fresh as the last manual pnpm audit.
• Billing has never processed a real charge. The Stripe plumbing and tier limits are unit-tested against fakes.
• Some source comments cite documents that are not in this repository. The load-bearing parts are absorbed into the README; the citations are left in place as the record of why each piece exists.
• The internal package manifests still read 0.1.0. Nothing here is published to npm, and for a GitHub Action the tag is the version.

MIT licensed.