test(control-api): make the cross-product contract a gate - #195
Merged
Conversation
Task 6 of the product split. agliteterm ships from its own repository promising to speak the
agwintermctl dialect. This is the canonical copy of what that means: 38 verbs, each with the
request and the expected response SHAPE, driven through the real agwintermctl against a sandbox
instance. agliteterm's CI checks its mirror against this file and runs the same steps against its
own client, so a verb reshaped on either side fails the other product's build.
Shape, not values: "session.text returns a string" is a contract; "it returns exactly these bytes"
is a snapshot of one machine's shell prompt. Extra fields are allowed; the named ones must be there.
It found three real divergences on its first run, which is the entire point:
- window.list — the full app returns {"windows":[...]}, agliteterm returned a BARE ARRAY. Every
script reading .result.windows breaks against it. Fixed on that side.
- window.state — the full app reports UI flags (sidebarVisible, fullscreen, maximized,
quickTerminalVisible, activeWorkspace, activeSession); agliteterm reported window GEOMETRY. Two
different verbs wearing the same name. Fixed on that side, keeping geometry as extra fields.
- session.new returns an id BEFORE the session exists: the full app posts the creation to its UI
thread and answers immediately, so a verb aimed at that id a millisecond later legitimately
cannot find it. The spec settles; the wart is now written down.
Also caught two traps in the harness itself, both of which produce confident wrong answers:
- agwintermctl defaults its target to $AGWINTERM_SESSION_ID, so a suite run from inside a terminal
session aims untargeted verbs at the DEVELOPER's session, which the sandbox has never heard of.
The runner clears those variables.
- In the full app all windows share one process and content verbs act on the FRONTMOST window, so
the window steps hijacked the session and workspace steps that followed them. Lifecycle first,
windows last — and the spec says why, because the failure looks exactly like a contract break.
The env contract (AGWINTERM_ENABLED/PIPE/SESSION_ID/PANE_ID) is checked by asking the SHELL, not
the app: what matters is that the child process actually received them. That prefix is why the
rename kept it, so it is part of the contract rather than an implementation detail.
Verified: all 39 steps green against the full app, and against agliteterm after its two fixes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first CI run reported success having checked nothing: the app exe is under win-x64 (the csproj pins a RuntimeIdentifier), so the runner took its "agwinterm not found" branch and exited 0. A conformance suite that skips silently is worse than one that is absent — it is a green tick next to an unverified promise. - the path now includes win-x64 - -Strict turns every skip into a failure, and CI passes it. Locally a skip is still the right answer: a developer without the other product installed should not be blocked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI builds into bin\x64\Release\... because the solution config sets a platform; a local build lands in bin\Release\.... The hardcoded path was right for one of those and wrong for the other, which is how this suite first went green having checked nothing. Both binaries are now resolved — explicit argument, then the newest matching build under the project's bin\ (newest, so a stale Debug copy cannot beat the Release build CI just made), then an installed copy — and the run prints which one it used, so "it passed" is always attributable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task 6 of the product split. agliteterm ships from its own repository promising to speak the agwintermctl dialect. This is the canonical statement of what that means — 38 verbs, each with its request and expected response shape, driven through the real
agwintermctlagainst a sandbox instance.agliteterm's CI checks its mirror of this file against this one and runs the same steps against its own client, so a verb reshaped on either side fails the other product's build. That is the difference between a promise and a gate.
Shape, not values: "session.text returns a string" is a contract; "it returns exactly these bytes" is a snapshot of one machine's shell prompt. Extra fields are allowed; the named ones must be present.
It found three real divergences on its first run
window.list{"windows":[…]}window.statesidebarVisible,fullscreen,maximized,quickTerminalVisible,activeWorkspace,activeSessionEvery script that reads
.result.windowsbreaks against agliteterm's shape, andwindow.statewas two different verbs wearing the same name. Both fixed on the agliteterm side (geometry kept as extra fields), which is where the deviation was.The third is on this side and is now written down rather than fixed:
session.newreturns an id before the session exists. The full app posts the creation to its UI thread and answers immediately, so a verb aimed at that id a millisecond later legitimately cannot find it. The spec settles between the two; the wart is documented in the step's note.Two traps in the harness, both of which produce confident wrong answers
agwintermctldefaults its target to$AGWINTERM_SESSION_ID. A suite run from inside a terminal session therefore aims untargeted verbs at the developer's session, which the sandbox instance has never heard of. Six checks failed for that reason before the runner started clearing those variables — and on a bare CI runner it would have passed, so the local and CI results would have disagreed with no explanation.window.selectin the middle of the run silently redirected the later session and workspace verbs to a host that had never heard of their ids. The spec now does lifecycle first and windows last, and says why — the failure looks exactly like a contract break.Neither is a possibility in agliteterm, where every window is its own process with its own pipe. That asymmetry is itself part of what the contract has to survive.
The env contract
AGWINTERM_ENABLED/PIPE/SESSION_ID/PANE_IDare checked by asking the shell, not the app — what matters is that the child process actually received them. That prefix is the reason the rename kept it, so it belongs in the contract rather than in an implementation note.Verification
All 39 steps green against the full app, and green against agliteterm after its two fixes. Locally run against an installed 0.17.3 and a fresh agliteterm build.
One thing this PR cannot answer: whether a GitHub runner can host a suite that creates real windows and real shells. These checks have only ever run on a developer machine. The first CI run is where we find out — if the runner can't, the step is the thing to change, not the contract.
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
🤖 Generated with Claude Code