Spec: One authoritative role registration for native extensions

[DamianReeves]

Published from a local working file so it survives the machine it was written on.

This specification lived at .dev/docs/superpowers/specs/2026-09-21-native-extension-role-registration.md. .dev/ is gitignored on purpose — .gitignore:82 — so it was never in version control and existed in exactly one worktree.

Implemented in finos/morphir-rust#195, with the workspace role following in #198. The guard that a native extension both advertises a workspace capability and registers its handle — the failure this design exists to prevent — is asserted in #914.

---

One authoritative role registration

Status: design settled, not yet implemented
Repository: finos/morphir-rust (the ecosystem/morphir-rust submodule)
Relationship to other work: lands before step 3 of
2026-09-21-single-file-projects-are-synthesized.md, which needs a native workspace role.

Problem

An extension's roles — frontend, backend, and soon workspace — are described four times over, and
the four descriptions are kept in agreement by hand.

In crates/morphir-extension-sdk/src/native.rs:

1. declared_types: Vec<ExtensionType>, passed into with_extension
2. capabilities: ExtensionCapabilities, returned by the author's E::capabilities()
3. the handle fields frontend / backend, Option<Arc<dyn …>>
4. dispatchers: Vec<DispatchFn<E>>, passed into with_extension

validate_capabilities (native.rs:157-196) exists to reconcile (1) against (2). It never
inspects the handles, though its error messages say otherwise — "extension advertises frontend
without a native frontend handle" is testing declared_types.contains(&ExtensionType::Frontend).

The reachability matters and was initially overstated. with_extension is private, and all
three public constructors supply tags, dispatchers and handles that agree. So (1), (3) and (4)
are a maintenance hazard inside the SDK rather than a hole in its public API. The one externally
reachable inconsistency is (2), the independently authored capabilities — and native construction
already rejects the mismatches it can see.

The reason to act now is that adding the workspace role means adding a third entry to each of the
four descriptions, plus constructors for the new combinations. The hazard compounds.

The same split exists on the guest side. export_extension! derives types and dispatchers from one
token list, but capabilities arrive independently from E::capabilities()
(crates/morphir-extension-sdk/src/lib.rs:120), so a guest can advertise a role it does not
dispatch.

The deeper defect

Extension::capabilities() makes a wire aggregate an authoring interface. That is the disease;
the four-way correspondence is a symptom. An author is asked to hand-write the negotiated payload,
and separately to hand-register the implementations that payload claims.

Decision

One authoritative role registration, from which everything else projects.

A role is registered once, binding together the three things that must agree: its authoring
descriptor, its typed handle, and its invocation adapter. ExtensionType lists, capability
payloads, and protocol dispatch all become projections of that registration rather than
independently authored values.

What each layer owns

Layer	Owns
Protocol contract	Wire DTOs, serialization, method names, request/response envelopes
Authoring	Extension traits, role descriptors, typed registration, wire projections
Native adapter	Shared instances, object-safe handles, native protocol invocation
WASM guest adapter	Exports, guest instance lifecycle, PDK integration
Daemon	Negotiation, trust-boundary validation, provider selection, session policy

Registration semantics sit below both adapters and depend on neither Arc handles, Extism, nor
daemon types.

These are module boundaries first, not a crate split. Splitting crates without fixing ownership
would distribute the same correspondence across more packages. A protocol crate is a sensible later
extraction — the daemon needs the contract without the authoring and export machinery — and the
Extism dependency is already target-gated, so nothing forces the split now.

The native shape

pub struct NativeExtension {
    info: ExtensionInfo,          // derived snapshot; types projected from roles
    roles: NativeRoles,           // non-empty by construction
    common: CommonCapabilities,  // streaming / incremental / cancellation / progress / extra
    protocol: Arc<dyn NativeProtocol>,   // built by the SDK from `roles`, never an input
}

struct NativeRoles {
    frontend:  Option<FrontendRole>,
    backend:   Option<BackendRole>,
    workspace: Option<WorkspaceRole>,    // added in the follow-up PR
}

Three points, each load-bearing:

protocol is derived, not supplied. Today ProtocolHandle<E> (native.rs:242) stores its own
dispatcher list and a second metadata snapshot. That is the most consequential of the four
correspondences, because it is the one that decides what actually runs. The protocol adapter reads
the same registration and invokes the same typed handles as a direct call; the SDK supplies JSON
decode, invoke and encode. There is no caller-supplied dispatcher left to mismatch. Retaining a
NativeProtocol trait object is fine provided the SDK is its only constructor.

Role records are concrete, not generic. A generic Role<Capability, Handle> permits backend
metadata beside a frontend handle — it relocates the problem into the type parameters. Concrete
FrontendRole / BackendRole / WorkspaceRole; any generic implementation stays private. A public
sealed role abstraction buys too little for three roles.

An active role has no configurable enable flag. FrontendCapability.compile can be false
beside a working compile handle, and checking that boolean in a new constructor would only move
today's check. Instead the native authoring descriptor carries the language / IR-version / option
information and nothing else; the projection supplies compile: true. Same for backend.generate,
and for workspace when it arrives.

Non-emptiness and the builder

At least one role must be present. Rather than encode that structurally — the smallest sum that does
so partitions by first-present-role into three variants, which imposes an arbitrary ordering and
makes accessors awkward — keep private optional slots and prove non-emptiness at construction.

A consuming builder is where type machinery pays. Each mechanism must remove a class of error, not
restate an invariant:

Mechanism	Error it removes
Role-adding method bounded on E: Frontend / E: Backend	Registering an operation the implementation lacks
Empty / non-empty builder state, finish only on non-empty	Finishing an extension with no roles
Consuming role additions	Retaining a stale builder as the finished registration
Private role records, SDK-generated adapters	Mismatched metadata, handle and dispatch

Start with empty/non-empty. Add per-role present/absent states only if duplicate registration
should be a compile error; otherwise define repeated setters as replacement explicitly. Never
silently alternate between replacement and accumulation.

The generic boundary: the builder and the typed adapter constructors know E and share one
Arc<E>; native construction requires E: Send + Sync + 'static; finish returns the single
concrete NativeExtension. Registry and session types gain no per-extension type parameter. Role
endpoints stay object-safe with concrete request and result types — clone the Arcs, so neither
E nor dyn NativeFrontend needs Clone.

Do not erase to Arc<dyn Extension> and recover roles later: the author traits carry static methods
(traits.rs:9) and are not suited to it. Build each typed adapter while its concrete bound is in
scope.

Typestate cannot prove that a claimed language, IR version or cancellation support is truthful.
Those stay implementation contracts, and the daemon keeps validating them.

The guest side

export_extension! consumes one typed registration declaration, binding each role's authoring
descriptor, its E: RoleTrait bound, and its SDK-generated adapter. Metadata exports and JSON-RPC
discovery project from that declaration; dispatch uses its registered operations. The separate
__push_extension_types! and __push_extension_dispatchers! authoring paths go away.

Registration semantics are shared with the native adapter; native storage is not. The guest
acquires no Arc, no Send + Sync, and no native instance lifetime.

Three guest behaviours must survive:

• Validator and transform exports exist and have no capability records. Represent their
  registrations without inventing new wire fields.
• A guest may declare a role with its capability flag false. The daemon accepts role presence
  and gates operations afterwards. Do not extend the native "registered means enabled" policy to
  guests by accident; if the stronger guest API keeps the distinction, model advertised-but-disabled
  explicitly.
• Lazy construction: control requests avoid E::default(), operation requests create the
  instance (lib.rs:259).

Five roles, three capability records, two native endpoints

ExtensionType (types.rs:12) has five variants. Only three carry a detailed capability record —
FrontendCapability, BackendCapability, WorkspaceCapability. Only two have a native endpoint —
NativeFrontend, NativeBackend. These two asymmetries are different in kind, and this design
treats them differently.

The missing capability records are not a defect. The daemon already gates transform and
validator, by role presence via ExtensionInfo.types (controller.rs:50); what they lack is
detailed negotiation, not all negotiation. Detailed records earn their place by doing work —
frontend languages and backend targets drive provider resolution against normalized IR versions
(registry.rs:114), and workspace gates per-request on protocol version. An empty
TransformCapability would do none of that. Adding one for symmetry is explicitly rejected.

This is not a permanent judgement. Validator::validation_rules() and
Transform::transformation_names() already exist (traits.rs:48) with no consumer anywhere, and
transform and validator requests carry IR plus an untyped options map with no rule-selection
contract (types.rs:459). A record becomes worthwhile when the daemon uses it to select a provider
or reject a request. Introducing one later is a compatibility change, not an additive one: it
touches the DTOs, the hand-written serializer and its reserved keys, discovery expectations,
validation and gating, and would reject existing peers unless compatibility were explicitly
retained. Promoting a name that extensions already use as an extra key carries its own hazard
(types.rs:148).

The missing native endpoints are a real gap, and nothing about an IR transform or validator
requires it to run outside the host process. Closing it is additive: a typed endpoint, a concrete
role record, a private slot and pending registration, a builder method bounded on E: Transform,
the type-list and dispatch projections, and tests for single-role construction and
direct/protocol equivalence. It does not disturb Empty/NonEmpty, the concrete return type, or
existing frontend/backend construction. It is therefore not scheduled in this spec's three PRs,
and should be added when a native consumer needs it.

Constraints this imposes on the work

• A role need not carry a capability record. The shared registration model must accommodate a
  role whose entire wire projection is a role tag. Any code that assumes every role contributes a
  capability field is wrong.
• Common registration must not require native storage or threading bounds — no Arc, no
  Send + Sync in the shared layer.
• PR 1's guest work must preserve transform-only, validator-only and mixed registrations, with
  coverage that exercises real macro expansion rather than only the native path.
• Legacy guest behaviour where a role is declared with its capability flag false stays an explicit
  compatibility distinction. Do not let it harden into a transport-specific definition of what a
  role is.
• A native endpoint must not become a native-only capability. Workspace discovery is the model:
  a portable request (traits.rs:113, file_tree.rs:64) that the devkit binds native filesystem
  inputs into (workspace_discovery/mod.rs:207). Anything that genuinely cannot cross a process
  boundary — a borrowed handle, shared mutable host state, a termination guarantee — is modelled
  explicitly rather than hidden, as Stopped/Indeterminate already does (transport.rs:54).

Transport independence is a separate programme

Naming an adapter module after its transport is not a layering failure; choosing a WASM export macro
or a process launcher at the deployment boundary is correct. But transport does leak into places it
should not, and this work must avoid foreclosing the fix:

• Instance lifetime changes with the adapter. Guest dispatch constructs E::default() per
  request (lib.rs:282); native adapters retain a shared instance (native.rs:49). An author
  relying on instance state must know the deployment path. This is a semantic difference.
• Host services are WASM-specific. The host module is target-gated (lib.rs:98) and reaches
  Extism imports (host.rs:59); native logging macros silently do nothing (lib.rs:217).
• Provider invocation exposes transport branching to callers. ResolvedFrontend offers
  native_frontend(), native_mep_session() and installed_snapshot() (registry/types.rs:214),
  and the parent repository's CLI branches across invocation modes as a result
  (finos/morphir, crates/morphir/src/extensions.rs:103).

None of these block this refactor — it removes the independently authored declarations that would
otherwise make them harder to fix. The target seam is that role declarations and operation contracts
sit below deployment adapters, and an ordinary caller receives an invocable resolved role or session
while loading, launching, negotiation and transport selection happen beneath it. Direct native
invocation remains a legitimate implementation of that; transport independence does not mean
serializing an in-process call.

What must not change

• ExtensionCapabilities keeps its wire shape. Project the role fields plus the cross-cutting
  record back through the existing hand-written Serialize (types.rs:148), which emits all four
  booleans, omits absent roles, flattens extra, and rejects reserved keys. Top-level incremental
  and frontend.incremental stay distinct.
  Caveat: extra is a HashMap, so byte-identical output was never guaranteed across
  independently built maps. Preserve fields, values, omissions and current ordering behaviour — do
  not introduce sorting or canonical JSON here.
• ExtensionType stays. It is a wire discriminator and includes Validator and Transform
  (types.rs:12). Locally registered types become its projection; info.types is derived. Types
  received over the wire remain untrusted data.
• Daemon validation stays. controller.rs:50 gates advertised roles, operation flags and
  workspace protocol versions; validation.rs:389 validates externally supplied negotiation data
  including a legacy backend exception. Locally sound construction does not make a remote peer
  trustworthy.
• Native construction keeps rejecting undeclared advertised roles, false capability flags,
  serialization errors, and their current precedence. Silently dropping an advertised backend when
  frontend_only was requested would be a behaviour change.

Sequence

A strictly behaviour-preserving change cannot also eliminate every legacy inconsistency, because the
guest macro can currently emit role metadata that disagrees with its dispatch, and refusing that is
observable. The compatibility change therefore needs its own commits, but no longer its own PR —
see the regrouping note below.

1. PR 1 — native consolidation. Landed as refactor(sdk): one authoritative role registration for native extensions morphir-rust#195 (668067d). Internal role
   records, projections, and native protocol dispatch through registered handles. A consuming
   builder with Empty / NonEmpty typestate replaced with_extension; the three native
   constructors survive as thin wrappers preserving every current rejection, including the
   workspace one. Behaviour-preserving apart from capabilities() returning an owned value.
2. PR 2 — the authoring API. Consolidate the guest machinery onto a single registration
   declaration, retiring __push_extension_types! and __push_extension_dispatchers!; then
   introduce the stronger descriptors that carry a role's language / version / option information
   with no configurable compile or generate flag, the projection supplying true. Finally
   retire the independently supplied aggregate capabilities and role lists, and decide explicitly
   whether malformed legacy guest declarations now fail at export or at discovery.
3. PR 3 — native workspace role. Typed endpoint, registration, projection and dispatch, and the
   removal of the unconditional rejection in validate_capabilities. No new constructor per
   combination. This unblocks step 3 of the single-file-synthesis spec.

Why PR 2 merges what were three steps

The original sequence split guest consolidation from the stronger descriptors, and put the authoring
migration last. They are one subsystem: what an author declares. Building the single registration
declaration and then rebuilding it to carry descriptors means constructing the same shape twice, and
the legacy retirement is only meaningful once there is a new declaration to retire it in favour of.

What the original split bought was the ability to say "this PR changed nothing observable", and that
is preserved at commit granularity rather than PR granularity: the consolidation lands in
behaviour-preserving commits, and the compatibility change — where malformed legacy declarations
begin to fail — lands in its own commits, labelled as such, so bisection still works. The PR
description must state which commits are which. Do not claim the legacy macro path is statically
sound.

Splitting native.rs into modules is deliberately not folded into either PR
(finos/morphir-rust#196). Its acceptance criterion is that no test assertion changes — module paths
only — and landing it beside a feature makes that criterion unverifiable, which is precisely what
makes a mechanical move reviewable.

Later, optionally: extract the protocol contract into its own crate, then split the native and guest
adapters.

Testing

native.rs:780 already covers the behaviours most at risk: direct/protocol equivalence, a single
shared non-Clone instance, construction-time metadata snapshots, workspace rejection, and
serialization failures. These are the regression surface for PR 1.

Add: compile-fail coverage for the builder guarantees, and a test that exercises actual
export_extension! expansion — native-only builds omit the exported functions, so the guest path is
otherwise untested.

Traps

• A public generic role framework whose achievement is replacing three concrete records.
• Vec<RoleEnum> as the authoritative set, without answering duplicates and emptiness.
• Powerset enums or bitmask const generics to select role combinations.
• Passing both a handle and an arbitrary dispatcher into a constructor billed as sound.
• Recomputing E::capabilities() per request, breaking snapshot semantics.
• Deriving language or version declarations from unrelated legacy trait methods and assuming they agree.
• Removing daemon validation because SDK-built extensions are now well formed.
• Combining crate extraction, author-trait replacement, native workspace, and serialization cleanup
  into one PR.

Non-goals

Native workspace support (PR 2). Crate extraction. Any change to the negotiated wire format. Any
change to daemon gating or provider selection.