feat: automated Upstreamer port pipeline for @openrouter/agent - #1
Conversation
This repo is a port of @openrouter/agent but had no automation keeping it in sync: one commit (the manual 0.7.2 port), no CI, no drift signal. Upstream is now at 0.8.0, so the port is a minor version behind and missing HooksManager, versioned state serialization, and the #61-#68 fixes. Wires up Upstreamer (github.com/mountgram/upstreamer, MIT) so the port tracks upstream automatically, with gates strong enough to run unattended. - .upstreamer/upstreamer.md — the rewrite contract: scope (packages/agent only, MCP explicitly out), required exported API as the parity floor, the TS->Go naming map, permanent idiomatic divergences (context.Context over AbortSignal, explicit errors, struct tags + jsonschema over Zod, middleware for SDKHooks), the go-sdk pin, and the service_tier=auto workaround to preserve. - .upstreamer/scripts/verify.sh — mechanical gate: gofmt/build/vet/test, exported API presence via go doc, 0.8.0 parity surface, substrate pin. - .upstreamer/eval.md — parity gate, run in a fresh context against the TS reference. This is what catches a port that compiles but sits a version behind. Emphasizes the Go-specific traps: mixed approval/HITL turn ordering and stream errors reaching every consumer. - scripts/upstream — wrapper adapted for in-repo downstream: this repo IS the output, so no publish leg is needed. - .github/workflows/upstreamer-port.yaml — triggered by repository_dispatch from typescript-agent on npm publish (ports track releases, not every commit), weekly cron as backstop. Opens a PR; never pushes to main. - .github/workflows/ci.yaml — this repo had no CI at all, so the verifier's test calls only ever ran inside the sync job. State is seeded at 0.7.2 (adc7939) so the first run is an incremental 0.7.2 -> 0.8.0 delta rather than an unreviewable full regeneration. Failed gates deliberately leave state.yaml stale: the next run retries the same delta instead of skipping the gap, and the PR is labelled eval-failed. Credentials are OPENROUTER_API_KEY + OPENCODE_MODEL, read from .upstreamer/port.env locally (gitignored) or repo secret/variable in CI. See PORTING.md.
The required-API check fails by design until the port catches up to upstream 0.8.0. A permanently-red required check on every unrelated PR just teaches people to ignore CI, so report it in the job summary instead. It stays blocking where it matters: inside scripts/upstream, where it gates whether .upstreamer/state.yaml advances. Also make the verifier's version check say when it skipped for lack of an upstream checkout, rather than passing silently.
![perry-the-pr-reviewer[bot]](https://avatars.githubusercontent.com/in/3222961?s=60&v=4){kind=small}
There was a problem hiding this comment.
Perry's Review
Verdict: 💬 Comments / questions
Solid, well-documented automation infrastructure. The two-gate design (mechanical verify + fresh-context parity eval) with stale-on-failure state semantics is the right call — a false PASS advancing state is the exact failure mode that silently drops a port a version behind, and this pipeline makes that hard. CI is green, the existing SDK code is untouched, and the e2e tests skip cleanly without credentials.
Findings
Suggestion — Go verifier HooksManager pattern contradicts its own comment (.upstreamer/scripts/verify.sh:65)
The verifier comment says "Named loosely because the Go equivalents may not match TS spelling exactly," but the grep pattern \bHooksManager\b is the exact TypeScript name. The Go contract's Required Public API section says "A HooksManager equivalent" and the naming map table does not pin a Go name for it. When the 0.8.0 port lands, if the Go port uses a different idiomatic name (e.g. HookManager, HooksMgr), this check will falsely fail — blocking state advancement on a correct port.
Either add HooksManager to the naming map as the canonical Go name (making the verifier correct), or loosen the pattern to match a range of plausible names. The conversation_?state_?version check below it already does this well with case-insensitive -i and optional underscores.
Suggestion — full clone of upstream on every CI run (scripts/upstream:112)
git clone "$upstream_url" "$upstream_dir" does a full clone with all branches and tags. Since the upstream checkout is ephemeral in CI (fresh checkout each run), every sync does a full clone of typescript-agent. Adding --single-branch (or --depth=N if you only need recent history for diff resolution) would cut fetch time without affecting correctness. Not a blocker — just a CI cost consideration.
Nit — upstream clone assumes typescript-agent stays public
The clone passes no credentials. This works today because OpenRouterTeam/typescript-agent is public (confirmed), but if it's ever made private the pipeline breaks with a git auth error rather than a clear message. Worth a one-line note in PORTING.md under prerequisites.
Notes
- The
service_tier=autoworkaround verifier check (grep -rq 'service_tier\|ServiceTier') confirms presence but not theautovalue. Acceptable for a mechanical gate — the parity eval checks behavior. - The
perl -e 'alarm shift; exec @ARGV'timeout correctly propagates SIGALRM to the exec'd opencode process.pipefail+set -eensures the exit code propagates throughtee. upstreamer-changelog.mdis referenced by the contract and SKILL.md but already exists from the manual port — correctly not recreated here.
| # Hooks and versioned state are the 0.8.0 parity floor. Named loosely because | ||
| # the Go equivalents may not match TS spelling exactly; the eval checks behavior. | ||
| echo "-- 0.8.0 parity surface present" | ||
| grep -rqE '\bHooksManager\b' --include='*.go' . 2>/dev/null \ |
There was a problem hiding this comment.
▶ Prompt for agents
Suggestion: The comment says "Named loosely because the Go equivalents may not match TS spelling exactly," but \bHooksManager\b is the exact TS name. The contract says "A HooksManager equivalent" and the naming map doesn't pin a Go name. When the 0.8.0 port uses a different Go-idiomatic name, this check will falsely fail and block state advancement.
Either add HooksManager to the naming map table as the canonical Go name (making this grep correct), or loosen the pattern — the conversation_?state_?version check two lines below already does this well with -i and optional underscores.
There was a problem hiding this comment.
Addressed in follow-up PR #2. The merged 0.8.0 port chose the upstream spelling HooksManager verbatim, so rather than loosening the grep we pinned the name explicitly: the contract now lists HooksManager as the required Go type and the verify.sh comment states a rename is a contract change (naming map + check updated together), not drift the gate should tolerate.
| else | ||
| target_commit="$(git -C "$upstream_dir" rev-parse origin/HEAD 2>/dev/null || git -C "$upstream_dir" rev-parse origin/main)" | ||
| fi | ||
| git -C "$upstream_dir" checkout -q --detach "$target_commit" |
There was a problem hiding this comment.
▶ Prompt for agents
Suggestion: git clone without --single-branch fetches the entire ref namespace (all branches + tags) on every CI run, since tmp/upstreamer/ is ephemeral. Adding --single-branch would halve fetch time without affecting correctness — the script only ever checks out origin/HEAD or a specific --ref.
There was a problem hiding this comment.
Fixed in follow-up PR #2: --single-branch added, plus a fetch fallback so a --ref outside the default branch still resolves.
…ncode auth The wrapper only seeded auth.json when no openrouter credential existed. Anyone who had previously run 'opencode /connect' would have their port.env key silently ignored and the run billed to the old key — close to undebuggable. An explicitly provided key now always wins. Other providers in auth.json are preserved rather than clobbered, and a corrupt auth.json recovers instead of failing the run.
- Model id: openrouter/anthropic/claude-sonnet-latest does not exist on OpenRouter; opencode resolves the alias as ~anthropic/claude-sonnet-latest. Fixed in the contract frontmatter, port.env.example, PORTING.md, and the workflow comment. Set the OPENCODE_MODEL repo variable with the ~ form. - Committed opencode.json denying external_directory: without it, the model writing any scratch file outside the workspace (observed on the python port: /tmp/hookspec) trips an interactive permission ask that a headless `opencode run` can never answer — the run wedges silently until the job timeout. Deny makes opencode refuse the write so the agent course-corrects. - Timeouts: job timeout 300→150 min and inner wrapper alarm 14400→7200 s, so a wedged run dies with logs intact (inner alarm first) and stops blocking the upstreamer-port concurrency group for 5 hours. Same fixes verified end-to-end on the python-agent 0.7.2→0.8.0 port run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generated by the Upstreamer pipeline in this PR, run locally end to end (scripts/upstream --ref @openrouter/agent@0.8.0). Both gates passed: mechanical verifier === PASS: 0 failures ===, parity eval PASS WITH WARNINGS on round 4 (.upstreamer/eval-report.md). Ports the 0.7.2 -> 0.8.0 delta: lifecycle hooks system (HooksManager + nine built-in hooks), versioned conversation-state serialization, awaiting_client_tools for unresolved manual tool calls, default-on allow_final_response, strict_final_response / empty-final-retry tolerance, and MCP tool-result source discrimination. state.yaml advances to 680bceb, so the first CI run after merge is a no-op until the next upstream release. The eval's fix loop caught and fixed a real bug in round 3: an approval/HITL resume that itself re-paused never called StateAccessor.Save, so a StateAccessor-only caller could reload stale state and re-execute an already-approved tool. Remaining warning (pre-existing since the 0.7.2 hand port, not from this delta): no per-turn incremental StateAccessor persistence — worth a follow-up issue; see eval report section 4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Two commits added on top of the machinery:
One warning worth a follow-up issue (pre-existing since the 0.7.2 hand port, not from this delta): no per-turn incremental Remaining before this goes live: 🤖 Generated with Claude Code |
OPENCODE_MODEL repo variable updated to openrouter/~anthropic/claude-opus-latest to match; contract frontmatter, port.env.example, PORTING.md, and the workflow comment all now reference the same alias. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two changes so the pipeline runs entirely on the native GITHUB_TOKEN: - Cron and blank manual dispatches now resolve the latest published @openrouter/agent version from the public npm registry and port that release tag, instead of porting upstream main HEAD. This makes the weekly cron fully equivalent to the repository_dispatch fast path (same tag either way), so the cross-repo PAT dispatch in typescript-agent becomes a latency optimization, not a requirement — and it also fixes cron runs violating the 'ports track releases' principle by syncing to mid-flight upstream commits. - PRs opened with the native GITHUB_TOKEN don't trigger other workflows (GitHub's recursion guard), which was the standard argument for a PAT here. workflow_dispatch is exempt from that guard, so the port job now explicitly dispatches ci.yaml at the upstreamer/sync branch after opening the PR. Requires actions: write, added to the permissions block. No PAT or external secret is referenced anywhere in this repo now; the only secrets are OPENROUTER_API_KEY (inference) and the built-in token. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nager pinning) (#2) * fix: address both review suggestions from #1 - scripts/upstream: clone upstream --single-branch (ephemeral CI clone, full ref namespace doubles clone time), with a fetch fallback so a --ref outside the default branch still resolves. - verify.sh / contract: the HooksManager check grepped for the exact TS spelling while its comment claimed the name was loose — a future Go-idiomatic rename would falsely fail the gate. The 0.8.0 port chose the upstream spelling verbatim, so pin it explicitly: the contract now names HooksManager as the required Go type and the verify.sh comment says a rename is a contract change, not a drift to tolerate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(upstream): guard ref arguments against option injection Review nit on #2: separate the ref from options in the fallback fetch (-- terminator) and rev-parse (--end-of-options) so a manually supplied --ref value starting with '-' cannot be parsed as an option. CI refs always start with '@'; this hardens the manual path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* ci(release): dispatch the Python and Go ports on publish (HOP C) The Python and Go ports of @openrouter/agent track this repo as their reference spec, but nothing told them when a version shipped. Both were generated by hand against 0.7.2 and have sat a minor version behind since — missing HooksManager, versioned state serialization, and the #61-#68 fixes. Adds a HOP C dispatch beside the existing HOP B monorepo dispatch. On a real publish, python-agent and go-agent each receive openrouter-agent-published and open a PR porting the delta. Their pipelines gate the result on a mechanical verifier plus a behavioral parity eval before advancing sync state, so a bad port cannot land silently. Details worth noting: - Sends the release TAG (@openrouter/agent@X.Y.Z), not a branch, so a port reproduces the exact published tree rather than whatever main drifted to. - continue-on-error: the packages are already on npm when this runs, so a failed dispatch must not turn a successful release red. It warns instead, and names the manual recovery path. - !cancelled() so a failed HOP B dispatch doesn't also skip the ports. - Partial failure is tolerated: one port failing still dispatches the other. Reuses the same GH_TOKEN PAT as HOP B, which additionally needs contents:write on OpenRouterTeam/python-agent and OpenRouterTeam/go-agent. Also documents the full publish fan-out in the changeset-versioning skill, since a breaking callModel change now produces port PRs in two other repos. Companion PRs: OpenRouterTeam/python-agent#19, OpenRouterTeam/go-agent#1 * fix(release): address review findings on HOP C dispatch - Use -f (--raw-field) for all dispatch payload fields: -F treats values starting with @ (the release tag) as filenames, which made every port dispatch fail before the request was sent - Push release tags on the manual mode=publish path so the dispatched ref actually exists on the remote - Restore set -e; the gh api call sits in an if-condition and is already -e-exempt, so the rest of the script stays strict - Pass source_run_url via env like VERSION instead of inline ${{ }} Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(release): fall back to commit SHA when the release tag is not on origin Address review findings from cortex and Devin on the HOP C dispatch: - Make the manual-publish tag push non-fatal. It runs after packages are already on npm, so a rejected push (tag protection, or a re-run where the tag exists at a different commit) turned a successful publish red and, because the HOP B/C steps carry no status guard, skipped both dispatches. - Verify the tag is on origin before dispatching it, falling back to the run's commit SHA. Previously the manual path could dispatch a ref the port repos cannot resolve, making them fail on checkout instead of degrading. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: LukasParke <luke@openrouter.ai>
1 participant
Wires up automated porting of
@openrouter/agentinto this repo using Upstreamer (MIT) — the same tool that produced the existingupstreamer-changelog.md, but running on a schedule with real gates instead of by hand.Companion to OpenRouterTeam/python-agent#19 (same design, Go-specific contract).
Why
This repo has one commit (the manual 0.7.2 port), no CI, and no drift signal. Upstream is now at 0.8.0, so the port is a minor version behind and missing:
HooksManagerlifecycle system (upstream #7, #67) — the 0.8.0 headline featureawaiting_client_toolsmanual-tool persistence (#64), empty-final tolerance (#63), string-input resume (#61), default final-answer directive (#68)1. Fill in credentials. Two values, same names locally and in CI:
OPENROUTER_API_KEY.upstreamer/port.envOPENCODE_MODEL.upstreamer/port.envcp .upstreamer/port.env.example .upstreamer/port.env # gitignored.upstreamer/port.env.exampledocuments both. The wrapper writes the key into~/.local/share/opencode/auth.json, so headless runs work without the interactiveopencode /connectflow.OPENCODE_MODELoverrides the contract'smodel:field, so you can switch models without a code change — recommend a strong coding model, this is a load-bearing SDK port behind a strict eval.2. Add HOP C to
typescript-agent. This workflow listens forrepository_dispatchtypeopenrouter-agent-published, which nothing sends yet. It needs a dispatch step intypescript-agent'spublish.yaml, right next to the existing HOP B dispatch toopenrouter-web. Until then the weekly cron and manual dispatch cover it.What's here
.upstreamer/upstreamer.md.upstreamer/scripts/verify.sh.upstreamer/eval.md.upstreamer/state.yamladc7939)scripts/upstream.github/workflows/upstreamer-port.yaml.github/workflows/ci.yamlPORTING.mdGo-specific contract content
context.ContextoverAbortSignal, explicit(T, error)over exceptions, struct tags +invopop/jsonschemaover Zod,SDKHooksas public middleware (the Go SDK doesn't expose its generated internal hooks package), generics where TS used conditional types.service_tier=autoworkaround is documented as load-bearing, so a future run doesn't "clean it up" and silently break request serialization.ModelResultconsumer rather than one hanging silently.Design decisions worth reviewing
This repo is the downstream. Upstreamer normally writes generated output into
codebases/<name>/downstream/inside the upstreamer repo. Putting the machinery here instead means output lands in place and there's no publish/rsync leg that could clobber CI or release config. The converter skill has an explicit do-not-touch list for repo-owned files.Ports track releases, not commits. Trigger is npm publish, not every push to upstream main. Porting mid-flight commits creates churn and reviewer fatigue; at a publish event the upstream HEAD is also the release SHA, so change detection lands on a clean boundary.
Seeded state. Without seeding, run #1 regenerates the whole SDK — a huge unreviewable diff. Seeded at 0.7.2, run #1 is an incremental 0.7.2→0.8.0 delta.
Failure leaves state stale, on purpose. If either gate fails,
state.yamlis not advanced,.upstreamer/eval-report.mdexplains why, and the PR is titledEVAL FAILED — do not mergewith aneval-failedlabel. The next run retries the same delta rather than skipping the gap. This is the property that makes unattended runs safe; a false PASS is worse than no eval.The contract is the product. When a port is wrong, fix
.upstreamer/upstreamer.md, not just the generated code — a code-only fix gets re-broken next sync.Verified
Ran the full pipeline locally against
@openrouter/agent@0.7.2:go build ./...,go vet ./...,go test ./...go doc, not source grep).upstreamer/port.envis gitignored whileport.env.examplestays trackedTwo bugs found and fixed while testing: the seed was an annotated tag SHA rather than a commit SHA (would have forced a full regeneration every run), and the prompt heredoc tripped a bash 3.2 parsing bug on macOS.
Expectations
The first few runs will be contract-tuning, not clean automation. Budget review time. Also note a publishing blocker this pipeline doesn't solve:
go get github.com/OpenRouterTeam/go-agent404s at the module proxy because this repo is internal, so the documented install path doesn't work.Opened as a draft — needs the credentials above before the workflow can do anything.