Skip to content

chore(deps): move to the published cohort, and the one decision that blocks it - #749

Merged
drewstone merged 3 commits into
mainfrom
chore/cohort-refs
Aug 10, 2026
Merged

chore(deps): move to the published cohort, and the one decision that blocks it#749
drewstone merged 3 commits into
mainfrom
chore/cohort-refs

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Supersedes the diagnosis in #746, which named the wrong fix.

What actually breaks packed-cohort

The job does not install the published cohort. It checks out three sibling repos at pinned refs and packs those, then installs them beside @tangle-network/sandbox, which floats inside this repo's peer range:

repo pinned ref on main published today
agent-sdk 36b8e3ec — interface 0.43.1 0.46.1
agent-eval cd4c3195v0.144.4 v0.144.6
agent-knowledge b5c82eb9v7.0.11 v7.1.2

sandbox@0.19.3 binds its BACKEND_TYPE table to the canonical harness enum at import time, and prime only entered that enum in interface 0.45. So the job pairs a 0.43.1 enum with a 0.19.3 table and throws before any test runs. Every PR in this repo has carried that red check for a mismatch none of them introduced.

Repinning alone is not enough: the script also asserts this repo's declared dependency versions equal the packed ones (requires agent-knowledge@7.0.11, packed 7.1.2), so the catalog and the pins move together. Both are in this branch, along with the mechanical half of the interface migration — systemPrompt capability became { replace, append }, and prime joined the closed BackendType union.

Verified locally: with the new pins, the harness-enum failure is gone — the script advances past it to the dependency-equality assertion, which this branch also satisfies.

The one decision left — not mine to make silently

pnpm test is 2395 passed / 7 failed, and all seven are the same upstream change. @tangle-network/agent-profile-materialize@0.12.0 moved the codex and opencode system-prompt projection from an inline launch flag to a file the harness reads:

- developer_instructions="Native \"prompt\"\nsecond line\tend"
+ model_instructions_file=.codex/system-prompt.md

pi is unaffected and still projects natively, which is why the lab's own path never noticed.

The consequence is not cosmetic. src/candidate-execution/system-prompt.ts refuses a candidate whose launch is not the native executable, because an inline flag cannot be proven applied otherwise. With the projection relocated upstream, two refusals no longer fire:

  • rejects a system prompt when the declared harness does not launch its native CLI
  • refuses 'codex' launch args that would shadow the profile system prompt

Either upstream now delivers the prompt correctly for those launches and the guard is obsolete, or it does not and a system prompt is being silently dropped for a container-command launch — which is exactly the hazard the guard was written for. Deciding that requires knowing whether .codex/system-prompt.md is materialized into the workspace on that path.

I am not deleting a safety refusal to make a version bump green. Draft until the owner of candidate-execution/ says which it is; the branch is ready to finish the moment that is settled.

Closes nothing yet. #746 should be updated with this diagnosis or closed in favour of it.

The cohort job packs three sibling repos at pinned refs and installs them beside the
registry's sandbox, which floats inside the peer range. Those refs sat at interface
0.43.1 / eval 0.144.4 / knowledge 7.0.11 while sandbox moved to 0.19.3, and 0.19.3
binds its BACKEND_TYPE table to the canonical harness enum at import time — an enum
that only gained 'prime' in interface 0.45. Every PR in the repo has been carrying a
red check for a mismatch none of them introduced.

Repinned to the set actually published today: interface 0.46.1, eval 0.144.6,
knowledge 7.1.2. Both the CI job and the publish gate use the same three refs, so a
release is verified against the cohort a consumer would really install.
…144.6, knowledge 7.1.2

Includes the mechanical half of the interface 0.44-0.46 migration: the profile
capability's `systemPrompt` is no longer a boolean but a { replace, append } pair,
and `prime` joined the closed BackendType union.

Does NOT include the candidate-execution half; see the PR body.
…ery that actually ships

agent-profile-materialize 0.12 moved codex's replacement prompt from an inline flag
this repo projected to a delivery the materializer lowers itself: the prompt bytes
land in the digested plan at .codex/system-prompt.md and the flags carry
`-c model_instructions_file=…`, cwd-relative so the reference survives docker and
jail path remapping. The prompt is NOT dropped — proven by executing the
materializer: file bytes present, ownership 'materializer', flag exact.

What the dep bump DID silently change: this repo's launch guard keyed on
`plan.systemPrompt`, which the materializer no longer sets for codex, so the
`undefined` early-return skipped every check and two refusal tests began resolving.
The hazard the guard exists for is unchanged — the flag is only meaningful to the
native binary's argument parser; handed to an arbitrary entrypoint or a wrapper it
is inert argv, and the sealed candidate would claim an active prompt that never
applied. Delivery detection now keys on the lowered flag itself, so the same three
refusals (candidate-entrypoint, non-native executable, shadowing caller args) hold
for materializer-lowered delivery, and the conflict check learned the new key:
a caller `--config=model_instructions_file=…` would re-point codex's instructions at
foreign bytes after the plan's flag.

opencode: the materializer now refuses a candidate system prompt outright — its
only replacement control binds to the agent the launcher selects, which a sealed
plan cannot guarantee. That replaces this repo's opencode.json mutation (which
patched both built-in agents and hoped one was selected) with an actionable
fail-closed refusal; the dead projection is deleted and the refusal is pinned by a
test. The capability returns via a binds-aware candidate materializer upstream,
not a local workaround.

Tests now pin BOTH halves of codex delivery — the exact flag pair AND the exact
prompt bytes in the digested plan — plus the new shadow key, the opencode refusal,
and the 30th materialization axis (appendSystemPrompt) interface 0.44 added.
@drewstone
drewstone marked this pull request as ready for review August 10, 2026 00:52
@drewstone

Copy link
Copy Markdown
Contributor Author

The one open decision is resolved — with execution, not judgment calls.

The prompt is NOT silently dropped, and the guard was NOT obsolete. Proven by running materialize 0.12.0 directly: a codex candidate's prompt bytes land in the digested plan at .codex/system-prompt.md (ownership: 'materializer') with the flag pair -c model_instructions_file=… — cwd-relative so the reference survives docker/jail remapping. The delivery is real; what the dep bump silently disabled was this repo's provability guard, which keyed on plan.systemPrompt — a field the materializer no longer sets for codex. The undefined early-return skipped every check, which is exactly why two refusal tests began resolving.

The hazard is unchanged: the flag is only meaningful to the native binary's argument parser. Handed to tools/codex or an arbitrary entrypoint it's inert argv and the sealed candidate would claim an active prompt that never applied. So the guard is restored, re-keyed on the lowered flag itself (90d4a3f5): same three refusals (candidate-entrypoint, non-native executable, shadowing args), and the conflict check learned the new key — a caller --config=model_instructions_file=… would re-point codex's instructions at foreign bytes after the plan's flag.

opencode: materialize now refuses a candidate system prompt outright (its only replacement control binds to the launch-selected agent, which a sealed plan can't guarantee). That's strictly better than this repo's old opencode.json mutation, which patched both built-in agents and hoped. Dead projection deleted; the refusal is pinned by a test; the capability returns upstream via a binds-aware materializeCandidateProfile.

Proof, all local:

  • pnpm test: 2397 passed / 0 failed (was 2395/7)
  • typecheck, lint, build clean
  • node scripts/verify-packed-cohort.mjs with the new pins: Packed package cohort verified. — interface 0.46.1, eval 0.144.6, knowledge 7.1.2, runtime 90d4a3f5, including a real 6-pair/12-execution profile-improvement comparison through the packed archives. The check that has been red on every PR in this repo goes green with this branch.

Tests now pin BOTH halves of codex delivery — the exact flag pair AND the exact prompt bytes in the digested plan — plus the new shadow key, the opencode refusal, and the 30th materialization axis (appendSystemPrompt).

@drewstone
drewstone merged commit 3f11779 into main Aug 10, 2026
2 of 4 checks passed
@drewstone
drewstone deleted the chore/cohort-refs branch August 10, 2026 00:53
drewstone added a commit that referenced this pull request Aug 10, 2026
….46 cohort (#753)

The cohort bump (#749) changed the canonical digests the fixture freezes; the ci
job's own guard names the fix (pnpm generate:testing-fixture). Full suite green
(2397) with the regenerated fixture.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant