contract: freeze the issue codes and `data` field names the VS Code extension gates on (all fail open)

[alpCaner]

Summary

alp-sdk-vscode gates real behaviour on exact issue-code strings and unversioned data field names from tan's envelope. Every one of those matches fails open: rename any of them and the extension does not error, does not log, and does not warn — it silently skips the check or falls back to stale data, with CI green on both sides.

v0.4.0 is about to be cut with ~36 commits and six fixes since v0.3.1, and the extension pins SUPPORTED_CLI_VERSION = "0.3.1". This is the last moment before those skew paths are exercised by customers for the first time.

The exact strings the extension matches on

Issue codes (consumer file:line in alplabai/alp-sdk-vscode at d334faa):

code	consumer	what dies if renamed
bootstrap.windows-unsupported	src/alpCli/service.ts:402	host refusal ignored
bootstrap.yocto-host	src/alpCli/service.ts:417 (also requires severity === "error")	a Yocto-only project is sent into a bootstrap that cannot work on this host
bootstrap.prerequisites-missing	src/alpCli/service.ts:447	tan's own refusal is not recognised, so the extension spawns the real bootstrap terminal anyway and the customer watches the identical failure scroll past with the install guidance lost
presets.sdk-root-unresolved	src/ideHub/newProjectFlowPanel.ts:167	the New Project wizard silently falls back to a static catalogue

The fail-open is structural, not incidental: prerequisitesMissingIssue returns ?? null and bootstrapHostVerdict returns { refuse: false } for anything unrecognised. That default is deliberate and correct — an unknown issue must not block a user — but it means a rename is indistinguishable from "no problem".

data field names read with ?? [] fallbacks that degrade silently:

• data.soms[].cores — src/ideHub/newProjectFlowPanel.ts:158,186
• data.checks[].name / .status, data.summary.{pass,warn,fail}, data.nextSteps — src/toolchain.ts:169-172, plus a literal c.name === "workspace" at :249 and :284
• data.available.projectTemplates — src/ideHub/newProjectFlowPanel.ts:225
• data.examples[].sourceDir — :266
• data.releases — src/ideHub/sdkManagerMessages.ts:253
• data.written — src/ideHub/buildPlanPanel.ts:222
• data.targets / .written / .failed — src/loader.ts:47-49

Worked example of the silent failure

Rename data.soms in a MINOR release. The extension produces no error, no toast and no log line — the New Project wizard falls back to its static E1M_MODULES catalogue, and a heterogeneous SoM scaffolds single-core with no IPC. The reference device E1M-AEN801 is multi-core, so this is the default path, not an edge case. The customer's first project is quietly wrong and builds green.

Ask

1. A test in tan-cli asserting those four issue-code strings survive, and listing them in docs/CLI.md beside the exit codes as versioned public contract. The extension side should NOT be loosened to prefix matching — a prefix match on bootstrap. would swallow codes it has no verdict for.
2. Emit one golden JSON envelope per command family into the release assets, so the extension's contract test diffs against a published artefact instead of a hand-copied fixture. The extension can then fail loudly on a field rename at CI time rather than silently at a customer's desk.

Why here and not there

The extension can only detect a rename by hardcoding the names it expects — which is what it already does, and what makes it brittle. The contract belongs to whoever owns the envelope. Freezing it in tan is the fix; a stricter parser in the extension is a workaround.

Related: #98 (workspace reuse comparing only Zephyr MAJOR.MINOR) is the same shape of problem — a comparison too coarse to notice a real change.

[alpCaner]

Filed #110 alongside this: the envelope reports project.root and project.boardYaml but not the SDK root tan actually used (src/alpCli/models.ts:12 in alp-sdk-vscode). Worth freezing with the field names this issue covers — and with #101's walk-up landing it becomes the one fact the IDE cannot recover, since --sdk-root is only passed when the extension already resolved one.

[alpCaner]

Two findings from probing the actual emitted surface — both change how this should be built.

1. The codes you list are COMPOSED, not literal. A source-text assertion cannot freeze them.

Three of the four codes named here do not exist as string literals anywhere in the source. Grepping for "bootstrap.windows-unsupported" finds nothing. They are assembled at runtime:

crates/tan-cli/src/commands/bootstrap/mod.rs:678    code: format!("bootstrap.{code}"),
crates/tan-cli/src/commands/bootstrap/steps.rs:71   code: format!("bootstrap.{code}"),

with the suffix passed bare at the call sites — "sdk-root-unresolved" (mod.rs:170), "yocto-host" (:258, :264), "zephyr-base-manifest-mismatch" (:521), "prerequisites-missing" (steps.rs:679).

Three consequences that decide the design:

• A test that asserts these literals appear in source would be asserting against text that isn't there. The freeze has to assert on the emitted envelope, not on source strings.
• There are TWO independent format! prefix sites. They can drift from each other, and a single edit to either silently renames several codes at once — precisely the fail-open rename this issue is about, with a wider blast radius than one code.
• presets.sdk-root-unresolved (the fourth) is a literal, so the four are not even uniform in how they are produced. Any mechanism must cover both shapes.

Other composition sites worth including in the sweep: crates/tan-cli/src/commands/renode/mod.rs:543 and crates/tan-cli/src/commands/size.rs:437 each define their own fn issue(code, severity, message).

2. The emitted code surface is much larger than the four

A literal-only grep across crates/ yields 46 distinct codes, and that grep provably under-counts because of the composition above. Observed empirically but absent from the literal list: bootstrap.sdk-root-unresolved and bootstrap.zephyr-base-manifest-mismatch (from #107's probe — the latter correctly declined to reuse a $env:ZEPHYR_BASE tree whose manifest is not alp-sdk's, citing #769).

The extension gates on four. tan emits at least 46. If the freeze covers only the consumed four, the frozen set is a subset and everything outside it stays free to drift — which is the same defect this issue exists to close, just deferred.

Suggested shape, given the above: enumerate what tan emits (drive a command per family and capture the envelope) rather than what the consumer matches. That makes ask (2) — the golden envelope per command family — do double duty: it freezes the codes and the data key sets in one artefact, and it is immune to the literal-vs-composed distinction because it observes output rather than source.

Verification note

The composition finding came from grepping for the literals and getting zero hits on three of the four codes this issue is about. Worth restating as a general point for whoever implements: a passing grep-based test here would have meant "the string isn't in the source", not "the contract holds" — a green result that carries no information. Assert on behaviour.

[alpCaner]

Two more envelope fields worth freezing alongside the ones this issue covers, both filed from the Dependencies panel work (alp-sdk-vscode PR #373):

• sdk list: carry GitHub's draft/prerelease flags, so a release candidate is not offered as latest #122 — sdk list drops GitHub's draft/prerelease flags, so releases[0] will offer a release candidate as latest the day one is published. The extension currently works around it by rejecting SemVer pre-release suffixes, which cannot catch a draft.
• doctor --build: report a version per check, so consumers stop re-probing with a different PATH #123 — checks[] carries no per-tool version, so a consumer wanting an installed-version column has to re-probe with a different PATH than tan used. That produced a row reading Installed: 1.5.0 beside "west not found" before it was fixed by refusing to show the version at all.

Both are additions rather than renames, so they do not disturb what #106 is freezing — they are just worth being in the frozen set rather than added after it.

[alpCaner]

Heads-up from #110, now open as #129 — it adds one envelope key, so ask 2 here (a golden JSON envelope per command family shipped in the release assets) needs to account for it before those goldens are cut.

What was added

A new optional top-level key, sibling to project, not inside data:

"sdk": { "root": "/home/u/src/alp-sdk", "sourceTier": "discovery" }

Members are exactly {root, sourceTier}. sourceTier is one of sdkRootFlag | projectPin | globalDefault | discovery | none.

The key is absent entirely — not null — when no SDK resolved. That is deliberate: it is what keeps all eleven existing contract/envelopes/*/expected.json goldens byte-identical (verified by running contract.rs, 12/12, with contract/** untouched). None of the eleven fixtures resolves an SDK, so none of them emits the key.

The freeze as drafted targets issue-code strings and data field names, so nothing in it forbids this. Flagging only because a whole-envelope key-set freeze would, and because of the trap below.

The trap for ask 2

sdk.root is an absolute host path. A golden envelope captured on a machine where an SDK is discoverable will embed something like C:/Users/<name>/GitHub/alp-sdk or /home/<name>/src/alp-sdk straight into a published release asset — non-reproducible across machines, and a small information leak into a public artefact.

Two ways out, both fine:

• Generate the goldens in an SDK-free environment (which is exactly what contract/'s harness already does — isolated HOME, no discoverable sibling checkout — so the key simply never appears), or
• scrub/placeholder sdk.root when emitting the asset while keeping sdk.sourceTier, which is the part with contract value anyway.

Worth deciding before the assets are cut rather than after, since a published golden with a real home directory in it is not something you can quietly re-cut.

One small thing that belongs to this issue

contract/README.md:7-8 documents the envelope as {command, ok, exitCode, project, data, issues} and is now one key short. README.md is fixed in #129; contract/README.md is contract/**, which this issue owns, so it was left alone.

[alpCaner]

Fixed by #142, merged to dev as a5b136a7a721bb31cfb65bed74af7ed9f9f799c1. All 7 checks green.

Both asks landed. contract/issue-codes.json is the single source, frozen_issue_codes gates it against the emitting sources, and the release workflow folds it into a published envelope-contract.json asset — so the extension can diff against an artefact instead of a hand-copied fixture. Four new golden envelopes cover the command families carrying the data keys the extension reads with silently degrading ?? [] fallbacks.

The finding that decided the implementation

Three of the four codes do not exist as string literals anywhere. They are composed at runtime by code: format!("bootstrap.{code}") at two separate sites, from bare suffixes at the call sites, while presets.sdk-root-unresolved is a literal. A grep- or source-text-based freeze would have been asserting against text that is not there — passing or failing for reasons unrelated to the contract. That is why this is frozen as data with an emittedBy + literal pair per code, not as a source-text assertion.

The registry also distinguishes retired from frozen: bootstrap.windows-unsupported is emitted by tan ≤ v0.3.0 only, but the consumer branch is permanent back-compat for anyone pinned to an old binary via alpSdk.cliPath, so its spelling is reserved and must never be re-used for a different verdict. The gate enforces that too.

The gate earned its keep before it merged

On its first rebase it went red on three goldens, each a real change that had landed underneath it: explain-overview lost host-tooling-starter (#128 retired the template), and examples-catalog + presets-heterogeneous-som gained the top-level sdk object (#129/#110). Fixtures moved, assertions did not.

That also confirms #129's claim rather than contradicting it — the eleven pre-existing goldens stayed byte-identical because nothing resolved an SDK in them; these two pass --sdk-root ./sdk, so they legitimately gain the key.

Across four rebases it then stayed green through #137 and #143, both of which changed explain. Firing on #128/#129 and not on #137/#143 is the discrimination that matters; a gate that fires on everything is as useless as one that fires on nothing.

Consequence for additive envelope fields, worth stating

Goldens are exact serde_json::Value equality, so adding a field to a covered command is a deliberate contract change requiring a fixture refresh. That is intended — it is what caught #129 — but it means e.g. #140's new checks[].version will need a refresh for any golden covering a doctor envelope. None of the four here does. #129's shape is the better precedent where possible: additive and absent when unset, so existing goldens do not move.

What this deliberately does NOT prove

Stated in the test's own doc comment rather than left implicit: it proves the spelling still exists at the emission site, not that the code still reaches the wire. A refactor deleting the whole refusal branch while leaving the string behind passes here; bootstrap/mod.rs's unit tests cover emission.

And bootstrap.* uses a source-literal assertion rather than a golden for a reason: yocto-host fires only on a non-Linux host and prerequisites-missing only when a tool is absent from PATH, so a golden for either would be inert on the ubuntu CI leg — the exact failure class this repo spent the week removing. presets.sdk-root-unresolved is hermetically reachable, so it gets a real golden (presets-no-sdk), which is the stronger gate.

Mutation-checked, including a near-miss worth recording

Renaming "yocto-host" at its emitting site:

• 1 of 2 occurrences → PASSED, correctly: the code was still emitted at the second site (bootstrap/mod.rs:264). The mutation was incomplete, not the gate broken.
• both occurrences → FAILED: FROZEN ISSUE CODE 'bootstrap.yocto-host' is gone: crates/tan-cli/src/commands/bootstrap/mod.rs no longer contains "\"yocto-host\"" outside comments.

I briefly concluded from the first result that contains had a substring hole. It does not — the literal carries its surrounding quotes. I nearly "hardened" a gate that was already correct on the strength of an incomplete mutation.

One other thing this PR surfaced: the msrv leg is a fifth gate that had never been run locally (no 1.86 toolchain on the machine), so a let-chain reached CI while fmt/clippy/build/test were all green. cargo +1.86 check --workspace --all-targets --locked now reproduces it.

[alpCaner]

Fixed by #142 (a5b136a7).