Spec: Unified configuration and application lifecycle

[DamianReeves]

Published from a local working file so it survives the machine it was written on.

This specification lived at .dev/docs/superpowers/specs/2026-09-20-unified-configuration-and-lifecycle-design.md. .dev/ is gitignored on purpose — .gitignore:82 — so it was never in version control and existed in exactly one worktree.

Published for portability rather than because its status changed. Check the repository's current state before treating any part of it as outstanding: phase 1 landed as #894.

---

Unified configuration and application lifecycle

Date: 2026-09-20
Status: Approved design, not yet implemented
Repo: finos/morphir (crates/morphir) and ecosystem/morphir-rust (morphir-config, morphir-devkit, morphir-common)

Problem

Morphir has a capable layered configuration loader and a real application lifecycle. It uses
neither for the things they exist to do, so foundational behaviour is restated at every call site
and occasionally forgotten.

Configuration. morphir-config and the devkit loader implement seven ordered layers with
provenance tracking, deep merge, secret references and legacy aliases:

Defaults 0 → System 100 → Global 200 → Project 300 → WorkspaceMember 350 → UserOverride 400 → Environment 600

The command line is not among them. Every flag is hand-wired where it is used, so each setting
re-implements precedence: frontend_extension::resolve(flag, frontend, language),
elm_modes::Mode::resolve(flag, frontend), selected_ir_version(cli, project), and
OutOverrides { flag, env } — the last with its own environment variable, MORPHIR_OUT_DIR,
read through std::env::var_os rather than the layer.

Worse than bespoke precedence is outright bypass. logging.rs reads

let console_level = std::env::var("MORPHIR_LOGGING__LEVEL").ok();
let file_level    = std::env::var("MORPHIR_LOGGING__FILE_LEVEL").ok();

which is the environment layer's own key mapping retyped by hand, in the code that documents the
mapping. There are roughly fourteen direct std::env::var sites in the CLI.

Lifecycle. MorphirSession implements starbase's AppSession but only execute. startup,
analyze and shutdown are left at their defaults. Configuration is therefore discovered and
loaded several frames deep inside command bodies, and logging initialises before the session
exists at all.

These are one problem, not two. GH #887 — a standalone single-file compile silently ignoring
MORPHIR_FRONTEND__ELM__* — happened because "load the configuration" and "resolve a setting" are
separate ad-hoc steps, so one code path reached resolve with frontend: None and no layer had
ever been loaded. A missing configuration was an ordinary Option::None, indistinguishable from
"configuration says nothing about this". Nothing prevents the next instance.

Decisions taken

Decision	Choice	Rationale
Library	Keep our loader; do not adopt schematic	See below
Scope	Core plus migrating the existing bypasses	Half-migration leaves two config systems and new code can still pick the wrong one
Flag binding	Declared on the clap arg via a derive	The binding lives where the flag is defined, so adding a flag without wiring precedence becomes impossible
Lifecycle machinery	Populate the phases with a typed session; no component registry	Six components we control; a registry turns compile-time errors into runtime resolution failures
Session shape	Discriminated union at the boundary, typed access below	starbase forbids consuming typestate; see "Session states"
starbase tracing	Separate timeboxed spike	Largest and least certain piece; must not ride along inside a configuration change
Enforcement	Included	Phase 4 visits every bypass site anyway, so the guard is nearly free at that point

Why not schematic

starbase (0.13.3, already a dependency) is an application framework — app.rs, session.rs,
diagnostics/, exit_code.rs, tracing/ — and provides no configuration layering. Its sibling
schematic, by the same author, is "a layered serde configuration and schema library" supporting
files, URLs, environment variables, partial configs, merge strategies, validation and JSON-schema
generation. It is not in our lockfile.

Adopting it would cost provenance tracking, secret references, legacy aliases and the seven-layer
model unless each were reimplemented, to gain a merge engine we already have. Decisively:
schematic does not treat command-line arguments as a layer either, so the actual gap would
remain. Worth revisiting if the JSON-schema generation becomes valuable on its own terms.

Precedent worth noting: .NET's Generic Host treats AddCommandLine as an ordinary configuration
provider, and separates ConfigureHostConfiguration from ConfigureAppConfiguration — the same
two-stage split this design arrives at, reached independently.

Architecture

The command line becomes a layer

ConfigSourceKind gains one variant:

Self::CommandLine => 700,   // name: "cli"

Precedence stops being a rule anyone writes and becomes a consequence of the number. Provenance
reporting, morphir config source listings and the merge engine pick it up without change,
because they already iterate ConfigSourceKind.

Two configuration stages

Not an exception, a named part of the lifecycle. Some callers genuinely run before a working
directory or file IO is available, and can emit diagnostics that want logging to already exist.

Stage	Layers	Used by
BootstrapConfig (host config)	Defaults + Environment	logging::init_from_env, MorphirHome::resolve
EffectiveConfig (app config)	all eight	every command

The bootstrap stage fixes the hand-typing rather than the layering: callers ask for logging.level
and the same mapping code derives MORPHIR_LOGGING__LEVEL. Legacy aliases such as
MORPHIR_LOG_LEVEL become declarations instead of a second std::env::var call.

MORPHIR_HOME and MORPHIR_LOG_DIR stay in RESERVED_ENV_VARS: they are operational, not
configuration keys, and that remains a deliberate exception.

Phases

Phase	Owns
Bootstrap (pre-argv)	setup_miette(), host config, logging, MORPHIR_HOME
Startup	Build the CLI layer from parsed args, load the full stack, resolve home and out root
Analyze	Validate resolved settings once, so a malformed value fails before any work begins
Execute	Commands read settings from the session; no command loads configuration
Shutdown	Flush logs, report the operation outcome and log path, report provenance under --verbose

Validating in analyze rather than at point of use is .NET's ValidateOnStart.

Session states

starbase's run_with_session(&mut S) drives all four phases on one S, each method taking
&mut self. Classic consuming typestate (fn startup(self) -> Session<Analyzed>) therefore
cannot be expressed without writing our own driver and giving up AppRunOutcome, last_phase,
exit-code handling and the shutdown-always-runs guarantee.

The pattern that does fit is a discriminated union at the boundary, holding phase-produced
data only:

#[derive(Clone)]
struct MorphirSession {
    command: Commands,
    resources: SessionResources,   // LogGuard, operation id: live for the whole run
    state: SessionState,
}

#[derive(Clone)]
enum SessionState {
    Bootstrapped(Bootstrapped),   // host config + logging only
    Ready(Arc<Ready>),            // + EffectiveConfig, home, out root
}

/// Only constructible by `startup`. Every field is present, by construction.
struct Ready {
    config: EffectiveConfig,
    home: MorphirHome,
    out: OutContext,
}

There is deliberately no ShutDown variant. An earlier draft had one, and it was wrong twice
over. Shutdown is not stateless: it flushes logs and reports the log path, carries the operation id,
exit code and diagnostic, and reads the EffectiveConfig to report provenance. Transitioning to a
unit variant would discard exactly what shutdown needs. And since starbase runs shutdown even when
startup failed, it can be reached holding only Bootstrapped — its work depends on how far the run
got, so it needs the state rather than its absence.

The deeper error was conflating two things in one union. Phase progress — which phase was
reached — is already owned by starbase, in AppPhase and AppRunOutcome.last_phase; duplicating it
here would create a second source of truth. Phase-produced data is ours, and is all the union
carries. So shutdown does not transition state: it reads whatever state exists and releases
resources.

The split also answers where long-lived handles go. LogGuard outlives every phase, so it belongs
in SessionResources, not in a variant. Today it lives in main and is threaded into
report_operation_outcome at nine separate call sites — shutdown work done nine times, outside the
shutdown phase.

TaskLock needs no shutdown action at all: it has a Drop impl, so release is already RAII.

SessionResources must be shared rather than copied. LogGuard holds a WorkerGuard and an
fs::File, so it is not Clone — and cloning an RAII flush-on-drop handle would be wrong in any
case, since the flush must happen once. Because AppSession requires Clone and run_execute
clones the session, the guard sits behind Arc<LogGuard> and drops when the last clone does.

The union mirrors starbase's own AppPhase in vocabulary, so our state and AppRunOutcome.last_phase
stay aligned rather than becoming parallel ideas.

Arc<Ready> is load-bearing, not decoration: run_execute does let fg_session = session.clone(),
so the session is genuinely cloned mid-run, which is why AppSession requires Clone + Send + Sync.

Where typestate pays, it is used. Command handlers take &Ready, never the session. All
command code — where every one of these bugs has lived — is statically guaranteed a loaded
configuration. Illegal states are unrepresentable everywhere; illegal transitions are checked
in exactly one exhaustive match, in one file, on a path every run exercises.

This is what closes GH #887 structurally: config: EffectiveConfig is always a fully layered
stack, and "no morphir.toml was found" becomes a source status inside it (NotFound, which the
loader already models) rather than the absence of the whole object. The environment layer can no
longer vanish along with the file.

Binding a flag to a key

#[derive(Args, ConfigOverrides)]
pub struct CompileArgs {
    #[arg(long, value_parser = ["source", "morphir-elm"])]
    #[config_key = "frontend.elm.ordering"]
    elm_ordering: Option<String>,
}

The derive generates fn cli_layer(&self) -> serde_json::Value, building the nested value and
skipping None. The loader consumes it as an ordinary source. A flag without #[config_key] is
simply not a configuration setting (--json, --no-cache), which keeps the annotation meaningful.

Prerequisite. A derive needs a struct, and Commands is mostly inline variants — 16 of them
against 6 that already use args structs. Commands::Compile { … } becomes Compile(CompileArgs).
Mechanical, shrinks main.rs, and scoped to commands that carry config-bound flags.

Reading a setting

Declared once, as a type:

pub struct ElmOrdering;
impl ConfigSetting for ElmOrdering {
    const KEY: &str = "frontend.elm.ordering";
    type Value = Ordering;
    const ALLOWED: &[&str] = &["source", "morphir-elm"];
}

let ordering = session.config().get::<ElmOrdering>()?;

Because the stack knows provenance, the error message currently hand-written per key is generated
once and gains what none of them have — the surface that actually set the value:

frontend.elm.ordering is "alphabetical", which is not a mode this frontend knows;
it must be "source" or "morphir-elm"
  --> set by MORPHIR_FRONTEND__ELM__ORDERING

Today that message can only name morphir.toml, even when the value came from the environment.
The "a flag overrides a malformed value, so warn instead of failing" rule becomes one policy on
the resolver rather than something each key re-implements, and so applies to extension and
prelude automatically.

ConfigSetting is deliberately the data a future registry would consume, so this design is a down
payment on that rather than a detour.

starbase alignment

Used correctly today: App::run with AppRunOutcome::into_miette_result(), preserving real exit
codes instead of miette's default of always reporting 1; and CliError as thiserror plus
#[diagnostic(code(…))].

Finding	Evidence	Action
setup_miette() never called, and miette is not configured by us either	no set_hook, set_panic_hook or MietteHandlerOpts anywhere in crates/morphir/src	Call it in bootstrap (Phase 0)
AppExitCode never used	AppSession::initialize(&mut self, _exit_code: AppExitCode) not implemented	Deferred: no consumer yet, see below
starbase tracing feature off; logging hand-rolled	features = ["miette"]; logging.rs is 524 lines	Deferred to its own spike

Correction, measured during Phase 0. An earlier draft of this spec claimed the miette gap
mattered because with_cause_chain would restore detail the default handler drops. That is false.
Building the CLI with and without the setup, against a real malformed morphir.toml, produced
byte-identical output: miette's fancy feature already prints the cause chain.

What the setup actually changes is smaller: set_panic_hook, so panics render as diagnostics, and
the starbase_styles theme, so error output matches the rest of the CLI. Both are worth having.
Neither is the headline the earlier draft claimed.

Phase 0 also does not adopt AppExitCode. Each phase already returns Result<Option<u8>, E>
and starbase acts on it, and commands use that path today, so the field would have no reader. It
waits for a caller that needs to set an exit code from inside a command.

On AppExitCode: each phase already returns AppResult<E> = Result<Option<u8>, E> and starbase
calls handle_exit_code on it, so returning a code from a phase is supported today. AppExitCode's
value is letting code deep inside a command set the exit code without threading it up through
every return.

On tracing: TracingOptions covers default level, module filters, log file with rotation, NDJSON,
span display, a configurable log-env variable name and chrome trace dump — real overlap with
logging.rs. But ours also does operation-id correlation (observability.rs, 257 lines) that
starbase has no equivalent for, and those correlation spans are load-bearing for the CLI's own
diagnostics. "Replace logging.rs with starbase tracing" is not a claim this design can make;
it gets a timeboxed spike and its own decision record.

Migration

Five phases, each shippable alone.

Phase 0 — starbase alignment. miette configuration in bootstrap, installed once per process and
tolerant of a hook someone else already set. No configuration coupling. AppExitCode deferred for
want of a consumer.

Phase 1 — lifecycle skeleton. Populate startup/analyze/shutdown; introduce
SessionResources, SessionState and Ready; move configuration discovery out of command bodies
into startup, and the LogGuard plus outcome reporting out of main into SessionResources and
shutdown — collapsing nine report_operation_outcome call sites to one. Behaviour preserving, so
characterization tests pinning current resolution behaviour are written first.

Phase 2 — command line as a layer. ConfigSourceKind::CommandLine; the ConfigOverrides
derive; Commands::Compile becomes Compile(CompileArgs).

Phase 3 — resolver and settings migration. ConfigSetting, config().get::<T>(), uniform
provenance-aware errors. Move the Elm modes, extension, prelude and ir_version off bespoke
precedence. GH #887 closes here.

Phase 4 — host configuration and remaining bypasses. Logging and home stop hand-typing
variable names, and legacy aliases become declarations.

The out root needs care rather than a mechanical fold-in. workspace.out_dir and
--out-dir/MORPHIR_OUT_DIR are not the same setting: the former is a path relative to the
workspace root that is validated to stay inside it (no absolute path, no .., no backslash
separators), while the latter relocates the out root outright and is an absolute PathBuf. Treating
the flag as an alias of the key would quietly drop that validation. Phase 4 therefore decides
whether the relocation override gets its own key — and whether relocation should be expressible in
a file at all, given it is currently process-scoped by design. Until then OutOverrides keeps its
own resolution, but moves behind the resolver's provenance reporting so morphir config can still
explain where the out root came from.

Deferred: the tracing spike, and the setting-registry approach that would generate clap args,
the typed model, the documentation table and the JSON schema from one declaration.

Testing

Test-driven throughout, per the repository's normal workflow.

• Characterization first (Phase 1). Pin current configuration-resolution behaviour before the
  refactor, since that phase must not change behaviour.
• Unit. Derive output; CLI-layer precedence against every other layer; provenance naming the
  correct surface; the bootstrap stage seeing environment but not files; resolver error text,
  including the malformed-value-overridden-by-flag rule.
• Regression pin for GH Standalone single-file compile ignores the environment layer for [frontend.<language>] prelude and extension #887. A standalone compile honours the environment for every
  [frontend.*] key, not only the two Elm modes.
• Enforcement (Phase 4). A test asserting no std::env::var("MORPHIR_*") call sites remain
  outside the configuration crate, so the bypass cannot silently return. Phase 4 visits all of
  them anyway, which is what makes this cheap rather than speculative.

Risks

• Clone + Send + Sync on AppSession. run_execute clones the session, so everything it
  holds must be cloneable. Ready is behind Arc to keep that cheap; LogGuard must be behind
  Arc because it is not Clone at all and must flush exactly once. Cheap to design in, expensive
  to discover in Phase 1.
• Inline command variants. Converting them is mechanical but touches every migrated command;
  scope it to commands carrying config-bound flags rather than all 16 at once.
• Behaviour drift in Phase 1. Mitigated by characterization tests, not by care.
• Scope creep toward the registry. The registry is explicitly out of scope; ConfigSetting
  is the seam that keeps the option open without paying for it.

Open questions

None blocking. Two to revisit after implementation:

• Whether a setting registry (generating clap args, typed model, docs and JSON schema from one
  declaration) earns its cost once ConfigSetting exists for the migrated settings.
• Whether schematic becomes worth adopting for JSON-schema generation alone, given that our
  layering would still be ours.
• Whether out-root relocation (--out-dir / MORPHIR_OUT_DIR) should be expressible in a file at
  all, or stay process-scoped as it is today. See Phase 4.

References

• GH feat(elm): select the Elm doc-comment and ordering modes from config, flag or env #885 — the Elm mode surfaces that exposed the pattern
• GH Standalone single-file compile ignores the environment layer for [frontend.<language>] prelude and extension #887 — standalone compile ignoring the environment layer (beads morphir-6cq2)
• ecosystem/morphir-rust/crates/morphir-config/src/env.rs — the environment mapping
• ecosystem/morphir-rust/crates/morphir-devkit/src/config/{loader,sources}.rs — layers and provenance
• starbase 0.13.3 src/{app,session,exit_code,diagnostics}.rs