Single-file projects are synthesized: working thread

[DamianReeves]

Working thread for "a single file is a synthesized project" — the sequence that moves single-file compile identity out of the CLI and into the language providers. Use this to pick the work up across sessions and machines.

The decision, in one paragraph

A single file is not a special case. It is a synthesized project, and the provider owns synthesis: the file extension names the language, the source text names the module, and what counts as a legal module name is the language's business. The CLI used to do all three itself with a match on the file extension, which is why adding a language meant editing the CLI. Synthesis happens during workspace discovery, not at compile, so there is exactly one place identity can come from.

Where it stands

Step	What it is	Status
1	Compatibility tests pinning current behaviour	landed
2	Discovery contract — DiscoveryPurpose, SourceSelection, ProjectOrigin	landed (morphir-rust#193)
3	Native workspace support in the SDK	landed (morphir-rust#195, #198)
4	Provider synthesis — Elm and Gleam derive identity	landed (morphir-rust#208)
4b	Parent adopts the new contract	open: #914
4c	morphir-elm vnext TypeScript	not started
5	Capability-driven provider selection and source collection	not started
6	Delete the old CLI route	not started, gated

What is actually blocking

morphir-elm on vnext is the remaining piece of step 4. cli2/mep/protocol.ts:249 requires a top-level documents array and :439 rejects the request outright when that guard fails. #208 added a transitional legacy envelope so hosts predating the change keep working, but that does nothing for an old guest: once the parent sends sources, the MEP Elm extension refuses every compile. It needs the TypeScript change, a rebuilt process artifact, and its own pin bump. Changes stay on vnext; main is protected.

Two transitional mechanisms, one shared removal condition

Both landed in morphir-rust#208 and both disappear together, but neither removes itself:

1. The legacy compile envelope. A CompileRequest is either wholly legacy (top-level documents, optionally sourceRootUri/sourceRoot) or wholly modern (sources: SourceSet { root, documents }). Mixing the two is an error that names the ambiguity — two envelopes each state a source root, and a request stating two roots has no honest answer for which one module names resolve against. See SourceEnvelope in morphir-extension-sdk/src/types.rs; its doc comment records what to delete.
2. The released-CLI publish-gate skip. .mise/tasks/test/cli-release drives every freshly built bundle through the morphir CLI release pinned in .config/morphir-cli-version — it is the declared compatibility check between the two repositories. It now skips only when publish fails with unknown field naming a field in its TRANSITIONAL_FIELDS list that the descriptor really carries (currently workspaceDiscovery). Any other failure is still a failure, and the success path warns when an entry has gone stale.

Removal condition for both: a morphir release whose host sends sources and whose release.json parser knows workspaceDiscovery, plus bumping .config/morphir-cli-version to name it.

Until then, Gleam's bundle is not publish-checked against a released CLI. That is a real reduction in coverage. It is narrow and loudly announced on every run, but if the release slips, this scaffolding quietly becomes permanent. Worth a decision about who owns removing it rather than discovering it later.

Carried into step 5

Recorded in the spec's Sequence section, and repeated here because that file is not in git (see below):

• An explicit --package-name is still not validated against the provider's package contract, which the spec requires. Both providers skip a project that already has a name, so an invalid name fails at compile, exactly as today. The provider only becomes the natural place for that check once step 5 routes names through discovery.
• The cardinality rule sits one tier below where the spec places it. "More than one distinct source requires a name" is enforced in portable discovery (workspace.selection.name-required), not by the provider that owns what distinct means for its language. Correct for every language today.
• A named multi-source selection gets exposedModules unset, meaning "expose everything". The providers derive exposure from a single input. Nothing can construct that shape until step 5 wires source collection — which is also what makes enumerating a multi-source set's modules worth doing.

A design finding worth acting on independently

itest's golden assertions are implemented on top of Rego, and need not be.

prepare_assertion in crates/morphir/src/commands/itest/runner.rs handles AssertionKind::Golden by doing the comparison in Rust — reading the file, applying the selection, normalizing line endings, computing the mismatch — and then synthesizing a Rego program anyway:

package morphir_golden
import rego.v1
default matches := false
matches if { input.exitCode == 0; input.golden.actual == input.golden.expected }

which is serialized into an EvaluationRequest and shelled out to morphir eval --request … --json. The mismatch it already computed is used only as error context. So every golden assertion spawns the evaluator to re-confirm an equality Rust already decided.

Making golden assertions native would:

• let golden scenarios run under --no-default-features, converting several currently-ignored tests back into real coverage
• remove one subprocess spawn per golden assertion from every itest run
• stop provider: rego being required metadata on scenarios that assert nothing in Rego

The counter-argument is uniformity: one evaluation path, one report shape, one check_report. Going native means golden results bypass that, so the report contract would have to hold both.

Related: #914 marks the 15 tests that genuinely need the evaluator with cfg_attr(not(feature = "rego"), ignore = "…"), because --no-default-features is a documented, supported configuration (see #886) that was failing its own suite.

Where the spec lives, and a caveat about it

.dev/docs/superpowers/specs/2026-09-21-single-file-projects-are-synthesized.md, with the plan alongside it in .dev/docs/superpowers/plans/.

.dev/ is gitignored on purpose (.gitignore:82 — "Local working area: temp scripts, agent and script outputs, superpowers specs/plans"). So the spec, including the carry-forwards above, exists only in whichever worktree it was written in. It is not in version control and will not survive that worktree being removed. If this sequence is going to outlive the machine it started on, the spec needs a home that is actually tracked — that is an open question, not a solved one.

Settled, please don't re-litigate

• CompilePackage keeps its required name: String. Making it a sum (Named/Synthesized) or an Option was proposed and rejected in design review: synthesis happens at discovery, so an optional name at compile creates a second place a name can come from — the exact problem this sequence removes.
• Discovery rejects duplicate paths rather than deduplicating them. Deliberate, with a test.
• The protocol stays at version 1. It is pre-release and is being refined in place.
• Elm's scanner is a byte-for-byte port, not a rewrite. The parent's hand-written scanner accepts and rejects different inputs than a real Elm parser would, especially for malformed headers and the fallback cases. Reimplementing it with a real parser is not automatically equivalent.

[DamianReeves]

Update — 2026-09-22: step 4 is complete, and the host now picks the envelope

Three things landed since the first post, and one of them changes the shape of the problem.

The host chooses the envelope per provider

crates/morphir/src/extensions/compile_wire.rs on main introduces compile_wire_request(&request, mode):

• NativeDirect / NativeMep → the current envelope, sources: { root?, documents }
• ProcessMep / WasmMep → the legacy envelope, top-level documents with sourceRootUri back in the options bag

That resolves what was the open question here. A released process or WASM provider never sees sources, so it keeps working untouched; only native providers get the new shape. The file's own comment sets the retirement condition: "Released external providers precede SourceSet. New SDK providers also accept this envelope, so retain it until an explicit protocol transition replaces it."

So there are now three transitional mechanisms, not two, all waiting on that same explicit transition:

1. The legacy compile envelope in the Rust SDK — makes a guest accept either shape.
2. The released-CLI publish-gate skip in morphir-rust.
3. compile_wire_request — makes the host send the shape a given provider understands.

(1) and (3) are mirror images: one protects new guests from old hosts, the other protects old guests from new hosts. Neither removes itself.

morphir-elm 0.2.0 is published

extension/elm/v0.2.0 — six platforms — from finos/morphir-elm#1289 and #1290 on vnext.

Its guard accepted only a top-level documents array, so it would have refused every compile once the host sent sources. It now accepts either envelope as alternatives that can never be mixed, mirroring the Rust SDK rule: a request carrying both is refused naming the ambiguity, since two envelopes each state a source root and a request stating two has no honest answer for which one module names resolve against.

sources.root is accepted and deliberately unused — that extension requires exactly one document, so a root cannot change the outcome.

With compile_wire_request in place this release is no longer an unblocker; it is what lets the extension accept sources when the explicit transition happens.

A workspace trap worth knowing

A path dependency joins the workspace that reaches it unless excluded, and one workspace does not inherit another's exclusions. ecosystem/morphir-rust/third_party/morphir-package-tough is excluded from the submodule's workspace, so bumping the pin silently made it a member of the parent's workspace and subject to its -D warnings, failing clippy on upstream lints in vendored TUF code. Cargo says so, in a line that is easy to read past:

warning: resolver for the non root package will be ignored ...
package:   .../third_party/morphir-package-tough/Cargo.toml
workspace: .../Cargo.toml

main and #914 arrived at the same fix independently: repeat the exclusion in the parent.

Still open

• Steps 5 and 6, with the carry-forwards in the first post unchanged.
• The rego-gated tests. --no-default-features is a documented, supported configuration — it is how a contributor without the Spectre-mitigated MSVC libs builds — but three eval tests and twelve itest scenarios fail under it with "the Rego evaluator is unavailable", because they were never gated to match. Not yet on main.
• Documentation still describes the old contract as the current one. docs/design/draft/extensions/protocol.md, docs/adr/ADR-0009-compilation-source-identity.md, docs/design/project-compilation-context.md and docs/design/proposals/morphir-playground.md all name documents and options.sourceRootUri. Updating them to say sources alone would now be wrong in a new way — with compile_wire_request, the legacy envelope is still actively sent. They need to describe both, and which providers receive which.
• itest's golden assertions still run through Rego (see the first post). Unchanged.

[DamianReeves]

The specs are now published, so this work is portable

.dev/ is gitignored on purpose (.gitignore:82 — "Local working area: temp scripts, agent and script outputs, superpowers specs/plans"), which meant these specifications existed in exactly one worktree on one machine and had never been in version control. They are now here, verified byte-identical to their sources:

• Spec: A single file is a synthesized project #917 — A single file is a synthesized project. The authority for steps 5 and 6, which are not done. This thread carries the carry-forwards extracted from it; the spec itself carries the decision table, the settled contracts, the rejected alternatives, what must not change, and the resolved questions.
• Spec: Unified configuration and application lifecycle #918 — Unified configuration and application lifecycle. Phase 1 landed as feat(cli): the session owns configuration loading (lifecycle phase 1) #894.
• Spec: One authoritative role registration for native extensions #919 — One authoritative role registration for native extensions. Implemented in refactor(sdk): one authoritative role registration for native extensions morphir-rust#195, workspace role in 2023-08-24 (24 August 2023) - Morphir Project Meeting #198.

Where a spec and this thread disagree, this thread is newer — the specs are published as written, not retrofitted.

Still only on one machine

The eight implementation plans under .dev/docs/superpowers/plans/ (~2,700 lines) are not published. They describe work that has already merged, so the commit messages and this thread cover the reasoning that outlived them. Worth knowing they exist rather than assuming the record is complete:

2026-09-21-authoring-api, 2026-09-21-discovery-purpose-and-origin, 2026-09-21-lifecycle-phase-1-session-owns-configuration, 2026-09-21-native-role-registration, 2026-09-21-native-workspace-role, 2026-09-21-single-file-compatibility-tests, 2026-09-22-provider-synthesis, 2026-09-22-coordinated-landing.

Two further design documents are also local-only and relate to already-shipped work: 2026-09-17-elm-native-binding-design and 2026-09-17-morphir-rust-ci-impact-design.

If any of those turns out to be load-bearing for future work rather than a record of past work, it should be published the same way before the worktree it lives in is discarded.

[DamianReeves]

Update, 2026-09-22: how capabilities reach an installed record

Step 5 gave the Elm MEP extension morphir.workspace.discover, which raised a question this thread had not covered: how does a published extension declare the new capability so that an installed record agrees with the guest? Working through it showed a larger gap:

• No tool turns an Elm release into an installed record today. repository publish accepts only single-artifact WASM bundles, and the Elm .release.json is a different, multi-platform shape.
• Every distribution format is parsed with deny_unknown_fields, which is why TRANSITIONAL_FIELDS exists.

The design is now #921, Spec: Capability statements across the extension lifecycle. In short: the guest is the only author of its capabilities; a new morphir.extension.describe method reports them without side effects; publish accepts process bundles; install probes the artifact it selected; readers ignore unknown members unless they are marked critical; and releases can be host only, extension only, or host first and then extension.

What changes for this sequence:

• The bootstrap host release in Spec: Capability statements across the extension lifecycle #921 is the release that carries steps 5 and 6. It also meets the removal condition for the three transitional mechanisms listed above, so they retire together when the pins move.
• The morphir-elm branch feat/mep-workspace-discovery is pushed with no PR. It keeps the flat workspaceDiscovery key, which the bootstrap host converts, and the PR waits for Spec: Capability statements across the extension lifecycle #921.

#921 is the working spec. The durable form goes into the MEP draft (docs/design/draft/extensions/) and the morphir-extensions knowledge-base bundle.

[DamianReeves]

Ruling replaced: "the protocol stays at version 1 while pre-release"

The settled item "The protocol stays at version 1. It is pre-release and is being refined in place." is replaced by a toolchain-wide default: every versioned contract uses SemVer 2.0 strings, and a prerelease version matches only exactly (#921, decision 12; AGENTS.md; morphir-cli decision 0003 in #922).

The intent is unchanged: the workspace discovery protocol is still a draft that is refined in place. The draft is now stated in the version itself: the integer 1 becomes "1.0.0-draft.1", and it freezes at "1.0.0". The wire change (DiscoveryRequest.protocolVersion, WorkspaceCapability.protocolVersions, and the session check) belongs to the bootstrap host release from #921. The MEP handshake keeps 0.1 until its next change.

[DamianReeves]

Correction to the comment above: the workspace discovery protocol moves from the integer 1 to "0.1.0-draft.1", not "1.0.0-draft.1". A new contract starts on the 0.1.0-draft line and moves to the 1.0.0-draft line once its shape is settled (#921).

[DamianReeves]

Status, 2026-09-23: steps 5 and 6 delivered

Steps 5 and 6 of #917 are merged in #925 (squash 8693138).

Step 5: capability-driven selection and source collection

• The provider is selected from --extension, then [frontend.<language>] extension, then the registry. A configured [extensions.<id>] command process answers when it declares the language. An unnamed configured extension that cannot start is skipped with a warning on stderr.
• The language comes from --language, the configuration, or a suffix that a provider declares. The CLI has no language knowledge.
• Every restriction comes from a declared capability, and the refusal names the provider. A selection needs workspace discovery. A multi-file selection needs frontend.multiDocument. The IR version is negotiated.
• The three carry-forwards are done (feat(workspace): capability-driven ad-hoc discovery (step 5) morphir-rust#217):
  • an explicit --package-name is validated against the provider's package contract;
  • the provider owns the rule that more than one distinct source needs a name;
  • a named multi-source selection exposes its modules.
• compile, gleam compile and gleam roundtrip share one PreparedCompile. The session's startup phase stores it.
• A selection's task record carries compileScope: explicit-selection. An implicit generate refuses it unless --from-partial-compile is given.

Step 6: the old CLI route is deleted

The single-file Elm route, its module-header scanner and the morphir-<language> default are gone.

Releases and pins

• feat(workspace): capability-driven ad-hoc discovery (step 5) morphir-rust#217: ad-hoc discovery, multiDocument, workspace protocol 0.1.0-draft.1.
• Elm extension extension/elm/v0.3.0 (feat(mep): serve ad-hoc workspace discovery for Elm sources morphir-elm#1291, #1292): serves workspace discovery. Pinned in .config/published-extension-bundles.toml.

Still open

• The three transitional mechanisms (the legacy compile envelope, compile_wire_request, the TRANSITIONAL_FIELDS skip) retire with the first host release that implements Spec: Capability statements across the extension lifecycle #921.
• morphir-scala-elm has no workspace discovery yet, so its CI check compiles a one-file project (morphir-scala-elm: support workspace discovery so single-file compile keeps working with the morphir CLI morphir-scala#1070).
• A unified guest and host SDK across Rust, TypeScript and Python is a separate follow-up.