Skip to content

ci(port): track upstream HEAD instead of the latest published release - #24

Merged
LukasParke merged 3 commits into
mainfrom
LukasParke/port-head-tracking
Aug 3, 2026
Merged

ci(port): track upstream HEAD instead of the latest published release#24
LukasParke merged 3 commits into
mainfrom
LukasParke/port-head-tracking

Conversation

@LukasParke

@LukasParke LukasParke commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

The problem this fixes

Today's weekly cron run reported success and opened no PR. That looked healthy. It wasn't:

resolved latest npm release: @openrouter/agent@0.8.0
target: 680bceb…   last port: 680bceb…
Upstream changes since last run: none.

Meanwhile upstream main had one unported commit — doom-loop detection (#73), ~7,500 lines, ~4,700 of it tests, rewriting a large part of model-result.ts. The pipeline was blind to it because it resolved the latest published npm release and ported that tag, and #73 is unreleased.

Release tracking sounds conservative. In practice it produces exactly the failure this pipeline exists to prevent: the port sits blind while unreleased work accumulates, then absorbs the entire delta in one automated run against the most load-bearing module in the package. Tracking HEAD keeps each delta small enough to actually review.

Ref resolution

  • The workflow no longer queries the npm registry. Blank ref = upstream default-branch HEAD, which scripts/upstream already resolves on its own.
  • The publish dispatch's client_payload.ref is now deliberately ignored (with a ::notice:: explaining why). It carries the release tag, which becomes an ancestor of HEAD as soon as the port is ahead — honoring it would revert landed work. The dispatch still does its real job: waking the pipeline promptly after a release. It just syncs to HEAD like every other trigger.
  • Fixed a latent bug: the Run port step passed --ref "" when no ref resolved. The arg parser consumes the empty value, so that is not equivalent to omitting the flag. Now only passed when there's a ref. Verified all four arg combinations.

Refusing to port backwards

This is the hazard HEAD-tracking introduces, so it needed a guard rather than a note.

scripts/upstream now exits 3 when the target is an ancestor of state.yaml's upstream_commit:

ERROR: refusing to port backwards.
  target:      680bceb… (@openrouter/agent@0.8.0)
  last ported: d030602…
  target is 4 commit(s) BEHIND what is already ported.

Verified against the real repo: fires on the actual 0.8.0 tag once state claims HEAD, and --force bypasses it.

Version honesty when ahead of a release

Being ahead is now normal — and invisible to the old check. package.json on main still carries the last released number, so pyproject.toml matches it and the check passes while the tree contains unreleased work.

  • verify.sh now also compares the ported commit against the release tag for the declared version. Level → PASS: ... (publishable). Ahead → a NOTE with the commit count and an explicit do-not-publish warning. Ahead is the intended state, so it does not fail the run.
  • publish.yaml gets a hard gate, because verify.sh's NOTE only appears when a sync run left an upstream checkout in tmp/ — and there is none at publish time, i.e. the one moment it matters most. It clones upstream, requires a release tag matching the declared version, and refuses if the ported commit is ahead.
Scenario Result
version 0.8.0, ported = 0.8.0 tag ALLOW
version 0.8.0, ported = HEAD (4 ahead) REFUSE
version 0.9.0, no such upstream tag REFUSE

Publishing X.Y.Z from a tree ahead of upstream's X.Y.Z tag ships unreleased work under a released number — permanently, since a PyPI version can never be reused.

Also: exclude tmp/ from ruff

Found while testing this. A port run writes scratch files under tmp/ — gitignored, but ruff still walks it. One stray probe script produced 22 ruff check errors and a format failure, meaning a sync run could fail its own mechanical gate on its own scratch space. Verified fixed with a deliberately broken file in tmp/. mypy was already scoped to src/tests and unaffected.

Verification

verify.sh PASS (0 failures) · all three workflows parse · backwards-guard and publish-gate each verified in three states · ruff/mypy clean with a deliberately broken file in tmp/ · 114 passed, coverage 83.89%.

What this means for the next run

The next cron will see upstream HEAD (d030602, 4 commits ahead) and port the doom-loop delta. That is a large first delta — expected, since it's the backlog release-tracking hid. Two notes for reviewing it:

  1. It will legitimately declare version 0.8.0 while being ahead of the 0.8.0 tag. verify.sh will print the NOTE, and publish.yaml will refuse to release from that state until upstream ships 0.9.0.
  2. Per the Test Parity rules added in test+ci: bulletproof the port's CI gate and add porting test-quality guardrails #22, it must also port doom-loop's ~4,700 lines of upstream tests — not just the source.

If you'd rather not have that land as one generated PR, porting doom-loop deliberately first would be much easier to review.

🤖 Generated with Claude Code


Update: port PRs now open via a GitHub App

Applying branch protection to main (6 required checks, PR required, 0 approvals) exposed a blocker in the porting automation, so this PR now fixes it too.

The problem, measured

Only pull_request-event runs satisfy required status checks. GitHub doesn't trigger workflows from events created with GITHUB_TOKEN (recursion guard), so a PR opened by the port workflow gets no pull_request checks — and can never merge.

The old workaround dispatched ci.yaml at the PR branch. That doesn't work, and I measured it rather than reasoning about it:

Count
check-runs on commit ce2b239 14
checks in PR #24's rollup 7

A workflow_dispatch run of the same workflow, on the same commit, succeeding — entirely invisible to branch protection.

A correction: I first claimed this citing runs on LukasParke/e2e-CI, but that PR was already merged and closed when they ran, so nothing would have appeared regardless of event type. I retested on open PR #24: 7 checks before the dispatch, 7 after. Same conclusion, but it needed real evidence.

"Can't we just not require a PR?"

Tested on a throwaway protected branch with required checks and no PR requirement:

remote: error: GH006: Protected branch update failed
remote: - Required status check "types" is expected.

Required checks gate pushes too. There's no branch-protection configuration that lets GITHUB_TOKEN-created work through while keeping the gates.

The fix

  • New Mint App token step (actions/create-github-app-token@v1), gated on vars.PORT_BOT_APP_ID.
  • create-pull-request uses the App token when present, GITHUB_TOKEN otherwise.
  • The dispatch step becomes a fallback, running only when no App token was minted, and emits a ::warning:: stating the PR will get no checks and cannot merge. An unconfigured bot should degrade loudly, not look healthy while producing stuck PRs.

App token over PAT: scoped to this repo, not tied to a personal account, independently revocable.

Setup required (two secrets, not yet present)

Create a GitHub App — org Settings → Developer settings → GitHub Apps:

  • Repository permissions: Contents: Read and write, Pull requests: Read and write. Nothing else.
  • Install on OpenRouterTeam/python-agent.
  • Generate a private key, then add:
Name Kind
PORT_BOT_APP_ID repo variable
PORT_BOT_PRIVATE_KEY repo secret (full PEM)

Until these exist, the fallback path runs — the pipeline still works, it just warns that the PR can't merge.

Already applied outside this PR

  • main branch protection: 6 required checks (check (py3.10/3.11/3.13), types, build, verify-port), require-up-to-date, include-administrators, no force-push/deletion, 0 required approvals per the automated-maintenance intent. e2e deliberately excluded — it exits 0 without the secret.
  • pypi / testpypi environments repinned from protected_branches: true to a custom policy naming main explicitly. The former means "any protected branch", so protecting a release branch later would have silently made it publish-eligible.

LukasParke and others added 3 commits August 3, 2026 12:19
The pipeline resolved the latest npm release and ported its tag. That is why
today's cron run found "no changes" while upstream main carried an unported
doom-loop commit (#73) — ~7,500 lines, ~4,700 of it tests, rewriting a large part
of model-result.ts. Release tracking sounds conservative and produces exactly the
failure this pipeline exists to prevent: the port stays blind to unreleased work,
then absorbs the whole delta in one automated run against the most load-bearing
module in the package. Tracking HEAD keeps each delta reviewable.

Ref resolution

- The workflow no longer queries the npm registry. A blank ref means upstream
  default-branch HEAD, which scripts/upstream already resolves on its own.
- The publish dispatch's client_payload.ref is now deliberately IGNORED (with a
  ::notice:: saying so). It carries the release tag, which becomes an ancestor of
  HEAD as soon as the port is ahead of the release — honoring it would revert
  landed work. The dispatch still does its real job: waking the pipeline promptly
  after a release. It just syncs to HEAD like every other trigger.
- Fixed a latent bug in the Run port step: it passed `--ref ""` when no ref was
  resolved. The arg parser consumes the empty value, so that is not the same as
  omitting the flag. Now the flag is only passed when there is a ref. Verified all
  four arg combinations parse correctly.

Refusing to port backwards

scripts/upstream now exits 3 if the target commit is an ancestor of
state.yaml's upstream_commit, printing how many commits behind it is and pointing
at --force as the deliberate override. Without this, a single stale --ref (or a
dispatch payload, before the change above) would instruct the converter to
faithfully port an older tree and silently revert everything since. Verified: the
guard fires on the real 0.8.0 tag once state claims HEAD, and --force bypasses it.

Version honesty when ahead of a release

Being ahead is now the normal state, and it is invisible to the old check:
package.json on main still carries the last released number, so pyproject.toml
matches it and the check passes — while the tree contains unreleased work.

- verify.sh now also compares the ported commit against the release tag for the
  declared version. Level with the tag PASSes as publishable; ahead prints a NOTE
  with the commit count and an explicit "do not publish this version" warning.
  Ahead is not a failure — it is the intended state — so it does not fail the run.
- publish.yaml gets a hard gate, because verify.sh's NOTE only appears when a sync
  run has left an upstream checkout in tmp/, and there is none at publish time. It
  clones upstream, requires a release tag matching the declared version, and
  refuses if the ported commit is ahead of it. Verified in all three states: level
  allows, ahead refuses, missing tag refuses.

Publishing X.Y.Z from a tree ahead of upstream's X.Y.Z tag ships unreleased work
under a released number, and a PyPI version can never be reused.

Also: exclude tmp/ from ruff

Found while testing: a port run writes scratch files under tmp/ (gitignored, but
ruff still walks it), and one stray probe script failed `ruff check` and
`ruff format` — 22 errors that had nothing to do with the port. A sync run could
fail its own mechanical gate on its own scratch space. mypy was already scoped to
src/tests and unaffected.

Docs

PORTING.md's pipeline diagram now shows the real trigger set and the backwards
refusal, plus a section on why HEAD over releases and the two consequences. The
contract's Package Version section says between-releases is now the normal state,
tells a sync to report how far ahead it is, and states the publishing rule.

Verification: verify.sh PASS (0 failures) · all three workflows parse · guard and
publish-gate logic each verified in three states · ruff/mypy clean with a
deliberately broken file in tmp/ · 114 passed, coverage 83.89%.

Co-Authored-By: Claude <noreply@anthropic.com>
Branch protection on main requires six status checks, and only pull_request-event
runs satisfy them. GitHub does not trigger workflows from events created with the
native GITHUB_TOKEN (recursion guard), so a PR opened with it gets no
pull_request checks and can never become mergeable — the automated port PR would
sit permanently stuck.

The previous workaround dispatched ci.yaml at the PR branch. That does not work,
and I measured it rather than assuming: on PR #24 the commit carried 14
check-runs while the PR's rollup showed 7. A workflow_dispatch run of the same
workflow, on the same commit, succeeding, was entirely invisible to branch
protection.

(I also had to correct myself here. I first claimed dispatch runs never attach,
citing runs on LukasParke/e2e-CI — but that PR was already merged and closed when
they ran, so nothing would have shown regardless. Retested on an open PR: 7 checks
before the dispatch, 7 after. Same conclusion, real evidence.)

Also verified that dropping the PR requirement does not help. On a throwaway
protected branch with required checks and no PR requirement, a direct push is
rejected outright:

  remote: error: GH006: Protected branch update failed
  remote: - Required status check "types" is expected.

Required checks gate pushes too, so there is no configuration of branch
protection that lets GITHUB_TOKEN-created work through while keeping the gates.

An App installation token is not recursion-guarded, so the PR it opens gets real
checks. Preferred over a PAT: scoped to this repo, not tied to a personal account,
revocable on its own.

- New "Mint App token" step (actions/create-github-app-token@v1), gated on
  vars.PORT_BOT_APP_ID being set.
- create-pull-request uses the App token when present, GITHUB_TOKEN otherwise.
- The dispatch step is now a FALLBACK, running only when no App token was minted,
  and it emits a ::warning:: stating plainly that the PR will get no checks and
  cannot merge as-is. An unconfigured bot should degrade loudly rather than look
  healthy while producing permanently stuck PRs.

Setup is documented in PORTING.md: App with Contents + Pull requests read/write
(nothing else), installed on this repo, then PORT_BOT_APP_ID (variable) and
PORT_BOT_PRIVATE_KEY (secret). Neither exists yet, so the fallback path is what
runs until they are added.

Verification: all three workflows parse · mint step gated correctly · PR token
falls back as intended · verify.sh PASS (0 failures) · 114 passed.

Co-Authored-By: Claude <noreply@anthropic.com>
Two bugs in the App-token gate, both surfaced by actually adding the App ID.

1. The gate tested only vars.PORT_BOT_APP_ID. The ID is a variable and the key is
   a secret, so they are added in different places and one lands first — which is
   exactly what happened: the ID arrived, the key did not. With only the ID set,
   the mint step would run and fail on the missing key, converting a working
   fallback into a broken pipeline. Worse than having no App configured.

2. The obvious fix — `secrets.PORT_BOT_PRIVATE_KEY != ''` in the step's `if:` —
   does not work. The `secrets` context is NOT available in a step-level
   conditional (only github/needs/strategy/matrix/job/runner/env/vars/steps/
   inputs are), so it evaluates to empty and the condition silently never
   matches. That would have been a quiet no-op: no mint, no error, and the
   fallback firing forever while the App looked configured.

Surfaced as a job-level env boolean (HAS_APP_KEY) instead, which the step `if:`
can read. Only ever compared against '', never echoed.

Verified against GitHub's context-availability documentation rather than assumed.

Co-Authored-By: Claude <noreply@anthropic.com>
@LukasParke
LukasParke merged commit fdf1004 into main Aug 3, 2026
7 checks passed
@LukasParke
LukasParke deleted the LukasParke/port-head-tracking branch August 3, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant