Decouple dependency-system internals from `option()`/`argument()` and modifier implementations

[dahlia]

Problem

The dependency system's internal state types (DeferredParseState, DependencySourceState, PendingDependencySourceState) and helper functions have penetrated deeply into parser implementations beyond the construct layer where dependency resolution actually lives. An audit of dependency.ts imports across the codebase shows:

File	Imported symbols	Role
primitives.ts	16	State creation (DeferredParseState, DependencySourceState, PendingDependencySourceState), three-way state dispatch in complete(), registry queries in suggestions
modifiers.ts	9	Wrapped dependency source detection/propagation in optional() and withDefault(), DependencySourceState creation for default values, transformsDependencyValue marker handling
constructs.ts	10	DependencyRegistry management, resolveDeferredParseStates(), pre-completion, pending dependency detection

The coupling in constructs.ts is expected since constructs own the resolution pipeline. The coupling in primitives.ts and modifiers.ts is the concern.

In primitives.ts, option() and argument() are leaf parsers whose core job is token matching and value-parser delegation. Yet they import 16 dependency symbols to wrap parse results in the right state type, dispatch on three dependency state variants in complete(), and query a DependencyRegistry during suggestions. In modifiers.ts, optional() and withDefault() need to detect whether their inner parser is a wrapped dependency source, propagate wrappedDependencySourceMarker through the parser chain, and create DependencySourceState when supplying default values.

This coupling causes concrete problems:

• Maintenance burden. Every new usage context (e.g., top-level parsing in Top-level option() and argument() do not resolve derived parser dependencies #238) requires verifying that each primitive and modifier correctly handles every dependency state type. The Top-level option() and argument() do not resolve derived parser dependencies #238 fix was ultimately a comment update in primitives.ts because the existing preliminaryResult path was already correct, but it took several review rounds to reach that conclusion precisely because the branching logic is spread across files.
• Violation of single responsibility. Wrapping results in DeferredParseState, creating PendingDependencySourceState for initial state, propagating wrappedDependencySourceMarker, and querying a DependencyRegistry during suggestions are all dependency-resolution concerns that belong in the construct layer, not in leaf parsers or modifiers.
• Fragile state dispatch. The complete() methods in option() and argument() contain a chain of isDeferredParseState / isDependencySourceState / isPendingDependencySourceState / plain ValueParserResult checks. Modifiers like optional() and withDefault() have their own parallel dispatch for wrapped dependency sources. Adding a new dependency-related state type would require updating all of these sites.

Possible approaches

A. Let value parsers wrap their own results

DerivedValueParser and DependencySource would return their dependency-aware state types directly from parse(), removing the need for createOptionParseState() in primitives.ts. Similarly, modifiers would not need to detect or propagate dependency markers because the value parser's result would already carry the necessary information.

Pros:

• Removes dependency-related branching from both primitives.ts and potentially modifiers.ts.
• Each value parser is responsible for its own state representation.

Cons:

• ValueParserResult<T> (or the parser's return type) would need to become a wider union, or a new wrapper type would be needed. This risks leaking dependency concepts into the value-parser interface, which is currently clean and simple.
• complete() in primitives still needs to unwrap whatever the value parser returned, unless constructs take over completion entirely.
• May not fully address the modifiers.ts coupling, since optional() and withDefault() need to propagate dependency information through the parser chain regardless of who creates the state.

B. Move wrapping/unwrapping into constructs

Primitives would always produce plain ValueParserResult<T>. Modifiers would pass through results without dependency-specific handling. The constructs (object(), tuple(), merge(), concat()) would inspect child parsers' value parsers to determine whether deferred resolution is needed and manage the raw-input bookkeeping themselves.

Pros:

• Primitives and modifiers become truly dependency-agnostic.
• Dependency resolution is fully centralized in the construct layer, which already owns the DependencyRegistry and the resolveDeferredParseStates() pipeline.

Cons:

• Constructs would need access to the raw input string that produced each child's result, since deferred resolution requires re-parsing. This likely means extending the Parser interface (e.g., a rawInputs field on ParserContext, or a richer result type from parse()) so that constructs can retrieve the original tokens.
• The wrappedDependencySourceMarker propagation in modifiers exists because withDefault() needs to supply a default value as a dependency source. Moving this to constructs may require constructs to understand the modifier chain, which could trade one form of coupling for another.

Other considerations

A hybrid approach may also be viable: primitives and modifiers could annotate their results with minimal metadata (e.g., the raw input string) without knowing why the metadata is needed, and constructs would use that metadata for dependency resolution. This would keep primitives and modifiers simple while avoiding a full Parser interface change.

Scope

This is a refactoring issue and should not change user-facing behavior. It should be addressed before 1.0 to reduce the risk of dependency-related bugs in leaf parsers and modifiers going forward.

[dahlia]

Proposed direction

I think this should be treated as a substantial pre-1.0 redesign rather than a narrow refactoring. The core problem is not just that constructs.ts has too much logic; it is that the dependency system's execution model has leaked into parser state shapes and wrapper protocols, so option(), argument(), and modifiers now need to understand dependency-specific internals that should belong to the construct layer. The experience in #238 and #748 was a good example of this: it took several review rounds to determine where the real behavioral change lived, because the dependency protocol was spread across primitives, modifiers, and constructs.

I think we should therefore choose a Plan B direction and make the dependency system construct-owned again, even if that means a larger change and a later 1.0.0. I also do not think we should constrain ourselves to the current ParserContext shape. If we are going to clean this up properly, it is better to separate parser-local state from cross-cutting execution context instead of continuing to encode runtime concerns inside parser states.

Design principles

• Parser state should represent local parse progress only. It should answer questions like “did this parser match?” and “what preliminary value/result does it currently hold?”, but it should not carry dependency-resolution protocol markers.
• Execution context should carry cross-cutting runtime data. Dependency registries, replay traces, evaluation phase, and annotation/context propagation belong in execution context, not in parser states.
• Dependency resolution should be owned by constructs and a shared runtime. object(), tuple(), merge(), and concat() should all use the same internal dependency-resolution engine instead of each relying on parser-state tricks.
• Top-level semantics should match construct semantics. option() and argument() at the top level should not have a separate dependency model from the same parsers inside object() or tuple().
• Wrappers should expose capabilities, not protocols. Modifiers like optional(), withDefault(), map(), bindConfig(), bindEnv(), and prompt() should describe whether they preserve source values, provide missing-source values, transform source values, or defer completion, rather than manufacturing or forwarding dependency-state wrappers.

Proposed runtime model

I think the cleanest model is to split the current ParserContext into two conceptual parts:

• A parse frame containing parser-local inputs such as buffer, state, and optionsTerminated.
• An execution context containing shared runtime data such as usage, phase, path, dependency runtime state, and raw-input trace data.

In that model, parse() still builds parser-local state, but dependency resolution no longer happens by returning DeferredParseState, DependencySourceState, or PendingDependencySourceState. Instead, primitives record replayable raw input into an input trace, constructs build a runtime snapshot of their children, and a shared dependency runtime performs these phases:

• collect explicit source values from successfully matched source parsers
• resolve source values from external providers such as config, env, or prompt when appropriate
• fill missing source values from source-preserving defaults such as withDefault(source, value)
• replay derived parsers using the resolved dependency values and the raw input recorded in the trace
• perform final completion using the replayed results where available

This would let us keep ValueParser itself simple while removing dependency-specific branching from primitives and most of the protocol forwarding from modifiers.

Parser and wrapper implications

Under this design, I think the parser contract should become more explicit about execution context. In particular, complete() and shouldDeferCompletion() should probably receive execution context directly rather than relying on annotation/state-shape tricks. That would let us remove a lot of the array-wrapping and annotation-inheritance complexity in modifiers.

For wrappers, the key shift is from state protocol to metadata/capability composition:

• optional(P) means “missing is allowed”, but it does not itself provide a missing source value.
• withDefault(P, value) means “missing produces a default final value”, and if P preserves a dependency source, it may also provide a missing source value.
• map(P, f) preserves derived-dependency relationships but breaks source-value preservation, because the outer value is no longer the source value.
• bindConfig(P), bindEnv(P), and prompt(P) should be modeled as external resolution providers rather than as wrapper-specific dependency-state protocols.

That distinction matters because not every default is a source default, and not every wrapper around a source should continue to register source values.

Construct-specific shape

I think all major shared-buffer constructs should build a common RuntimeSnapshot abstraction over their children, then call a shared dependency runtime.

• object() and tuple() are the easy cases: each child becomes a runtime node with a stable path.
• concat() should share one dependency runtime across its sequential children so that sources found in earlier children can affect derived parsers in later children.
• merge() is the hardest case, because sibling children must share dependency state while still preserving child-local scope for replay, caching, and debugging. I think this likely requires separating a result path from an internal scopePath.

This is also why I think we should not try to solve #750 by only cleaning up primitives.ts or modifiers.ts. The problem is architectural and touches how constructs model shared evaluation.

Planned refactor shape

At a high level, I think the work should look like this:

• define the new parser execution model first, including the split between parse frame and execution context
• introduce internal types for execution context, dependency runtime context, and input trace
• introduce parser dependency metadata/capabilities so wrappers can describe behavior without manufacturing dependency-state wrappers
• move construct dependency logic into a dedicated internal runtime module
• port object() and tuple() to the new runtime first
• port concat() and then merge()
• unify top-level option() and argument() with the same runtime model
• remove the old dependency-state protocol from primitives and modifiers
• delete the obsolete helpers, markers, and state types once the new runtime is in place

I would also explicitly treat sync and async paths as the same algorithm with different awaits, rather than preserving large duplicated phase pipelines.

Expected benefits

If this lands cleanly, I think we get several concrete benefits before 1.0:

• dependency semantics become easier to explain, because there is one runtime model instead of three partially overlapping protocols
• review and debugging become much cheaper, because #238-style questions can be answered by reading the dependency runtime rather than hunting through primitives, modifiers, and constructs
• top-level and nested usage become consistent
• wrapper behavior becomes more compositional and less ad hoc
• future features involving config/env/prompt/default interactions become easier to add without introducing more parser-state special cases

Rollout plan

I do not think this should be attempted in a single PR. A cleaner rollout would be:

• lock in the target semantics and test matrix first
• introduce the new execution-context types and internal runtime scaffolding
• migrate one construct family at a time
• migrate top-level parsing once the construct runtime is stable
• remove the old dependency-state protocol only after the new path is fully exercised

I think that is the safest way to make a large change without losing confidence in behavior.

Scope and non-goals

This should be considered a deliberate internal redesign before 1.0, not a bug fix with minimal touch points. I do not think preserving the current internal state protocol is a worthwhile goal. The point of this work should be to make dependency resolution belong to the runtime again, even if that means changing ParserContext, complete() signatures, and wrapper internals along the way.

If we agree on this direction, the next useful step would be to write down the concrete internal type shapes for ExecutionContext, DependencyRuntimeContext, InputTrace, and parser dependency metadata, and then use those as the basis for the first implementation PR.

[dahlia]

Proposed implementation plan

To make the redesign manageable, I think we should split it into a sequence of focused PRs rather than trying to land the whole runtime rewrite at once.

• PR 1: Lock in the target semantics and add a dedicated regression matrix.
• PR 2: Introduce the new execution-context types and parser contract changes.
• PR 3: Introduce dependency metadata/capabilities and the new dependency runtime scaffolding.
• PR 4: Port object() to the new runtime.
• PR 5: Port tuple() to the new runtime.
• PR 6: Port concat() to the new runtime.
• PR 7: Port merge() to the new runtime.
• PR 8: Unify top-level option() / argument() with the same runtime model.
• PR 9: Remove the old dependency-state protocol from primitives and modifiers.
• PR 10: Delete obsolete helpers/types and clean up docs/comments/changelog wording.

I think this ordering gives us a stable path from “new runtime exists” to “old protocol is gone” without mixing too many semantic changes into one branch.

PR 1: Lock in semantics

Before changing internals, I think we should explicitly codify the desired behavior in tests.

The regression matrix should cover at least:

• top-level option() / argument() with derive()
• top-level deriveFrom() with single and multiple dependencies
• object() sibling dependencies
• tuple() positional dependencies
• concat() dependencies across sequential children
• merge() dependencies across child parsers
• optional(source)
• withDefault(source, value)
• optional(withDefault(...))
• withDefault(optional(...), value)
• map(source) and source-chain breakage
• bindConfig(source)
• bindEnv(source)
• prompt(source)
• parse and suggest paths
• sync and async variants

The main goal of this first step is to make the intended semantics explicit before the runtime changes begin.

PR 2: Execution context redesign

This PR would introduce the new execution model:

• split parser-local frame data from shared execution context
• extend ParserContext to carry both
• pass execution context into complete() and shouldDeferCompletion()
• move annotation/deferred-completion propagation away from state-shape tricks

Concretely, this is where we would introduce internal types along the lines of:

• ParseFrame
• ExecutionContext
• ExecutionPhase
• ExecutionInternals

I think this should land before any dependency-runtime rewrite so that later PRs can build on the right parser contract.

PR 3: Dependency runtime scaffolding

This PR would add the new internal runtime types without yet porting all constructs to them.

The main additions would be:

• InputTrace
• TraceEntry
• DependencyRuntimeContext
• DependencyRequest
• DependencyResolution
• parser dependency metadata / capabilities
• shared runtime helpers for:
  • collecting explicit source values
  • resolving external sources
  • filling missing-source defaults
  • replaying derived parsers

At this stage, the new runtime can coexist with the old protocol temporarily, but the target should be clear: the old state-wrapper mechanism is on the way out.

PR 4-7: Port constructs one by one

I do not think all constructs should be migrated together.

The order I would use is:

• object(): easiest path model, easiest snapshot assembly
• tuple(): very similar to object(), but index-based
• concat(): shared dependency runtime across sequential children
• merge(): hardest case because it needs shared dependency state plus child-local scope

The target shape is that each construct builds a common internal RuntimeSnapshot and then delegates dependency work to the runtime.

That snapshot should give each node at least:

• a result path
• an internal scopePath
• the child parser
• the child state

merge() is the case that will probably force the cleanest version of the snapshot abstraction, so I think it should come last.

PR 8: Unify top-level parsing

Once the construct runtime is stable, I think we should make top-level option() / argument() use the same dependency model.

This should remove the current split where top-level usage has different dependency behavior and different explanatory comments from the same parsers inside constructs.

I think the cleanest implementation is to treat the top level as a thin root orchestration layer over the same dependency runtime rather than preserving a special-case path.

PR 9: Remove the old protocol from primitives and modifiers

Only after the new runtime is fully exercised should we remove the old dependency-state protocol.

The target here is:

• primitives.ts no longer creates or branches on DeferredParseState
• primitives.ts no longer creates or branches on DependencySourceState
• primitives.ts no longer creates PendingDependencySourceState
• modifiers.ts no longer propagates wrappedDependencySourceMarker
• modifiers.ts no longer needs dependency-state-specific wrapper logic

At that point, primitives should be reduced to ordinary parsing plus trace recording, and modifiers should mainly compose capabilities rather than shuttle protocol markers around.

PR 10: Cleanup

The last step is to remove dead code and update documentation/comments so the codebase reflects the new model cleanly.

This should include:

• deleting obsolete dependency-state helpers and markers
• tightening comments in primitives.ts, modifiers.ts, and constructs.ts
• updating any internal docs/comments that still describe the old protocol
• updating changelog wording if needed to describe the architectural change accurately

Internal milestones

I think the following internal milestones would help keep the work grounded:

• Milestone 1: The new parser contract exists, even if the old dependency protocol still exists.
• Milestone 2: The new dependency runtime exists and object() uses it.
• Milestone 3: All shared-buffer constructs use the new runtime.
• Milestone 4: Top-level parsing uses the same runtime.
• Milestone 5: The old dependency-state protocol is deleted.

If we structure the work around those milestones, each PR can have a clear success condition.

Explicit non-goal during migration

During the transition, I do not think we should spend effort trying to keep the old and new models equally elegant at the same time. The temporary goal should just be to keep behavior stable while moving responsibility into the new runtime. The cleanup can come after the new path is fully in control.

Next concrete step

If this implementation breakdown sounds reasonable, I think the next step should be to write the first PR as “semantics lock-in + regression matrix”, and only after that start changing parser contracts and runtime internals.

[dahlia]

Sketch of internal types

To make the proposed direction more concrete, I think it is worth sketching the internal type shapes that the new runtime would revolve around. These are not intended as final public API shapes, but as design targets for the internal execution model.

The main idea is to separate:

• parser-local progress/state
• shared execution context
• dependency runtime state
• replayable raw-input trace data

Parse frame and execution context

I think ParserContext should conceptually split into a parser-local frame and a shared execution context.

export interface ParseFrame<TState> {
  readonly buffer: readonly string[];
  readonly state: TState;
  readonly optionsTerminated: boolean;
}

export type ExecutionPhase =
  | "parse"
  | "precomplete"
  | "resolve"
  | "complete"
  | "suggest";

export interface ExecutionContext {
  readonly usage: Usage;
  readonly phase: ExecutionPhase;
  readonly path: readonly PropertyKey[];
  readonly annotations?: ParseOptions;
  readonly dependencies: DependencyRuntimeContext;
  readonly trace: InputTrace;
  readonly internals?: ExecutionInternals;
}

export interface ExecutionInternals {
  readonly rootKind?: "top-level" | "object" | "tuple" | "merge" | "concat";
  readonly branchId?: symbol;
  readonly replaying?: boolean;
}

export interface ParserContext<TState> {
  readonly frame: ParseFrame<TState>;
  readonly exec: ExecutionContext;
}

The important part here is that dependency registries, replay traces, evaluation phase, and annotation propagation move into ExecutionContext instead of leaking into parser states.

Parser contract

Under that model, I think complete() and shouldDeferCompletion() should receive execution context explicitly.

export interface Parser<
  M extends Mode = "sync",
  TValue = unknown,
  TState = unknown,
> {
  readonly $valueType: readonly TValue[];
  readonly $stateType: readonly TState[];
  readonly $mode: M;
  readonly priority: number;
  readonly usage: Usage;
  readonly leadingNames: ReadonlySet<string>;
  readonly acceptingAnyToken: boolean;
  readonly initialState: TState;

  parse(context: ParserContext<TState>): ModeValue<M, ParserResult<TState>>;

  complete(
    state: TState,
    exec: ExecutionContext,
  ): ModeValue<M, ValueParserResult<TValue>>;

  suggest(
    context: ParserContext<TState>,
    prefix: string,
  ): ModeIterable<M, Suggestion>;

  getDocFragments(
    state: DocState<TState>,
    defaultValue?: TValue,
  ): DocFragments;

  placeholder?: TValue;
  normalizeValue?(value: TValue): TValue;

  shouldDeferCompletion?(
    state: TState,
    exec: ExecutionContext,
  ): boolean;
}

I think this is important because it lets us stop encoding cross-cutting runtime concerns through state-shape adapters in wrappers.

Input trace

The current dependency system stores replay-relevant information inside dependency-specific state wrappers. I think that should instead become an explicit input trace owned by execution context.

export interface InputTrace {
  get(path: readonly PropertyKey[]): TraceEntry | undefined;
  set(path: readonly PropertyKey[], entry: TraceEntry): InputTrace;
  delete(path: readonly PropertyKey[]): InputTrace;
  child(pathSegment: PropertyKey): InputTrace;
}

export interface TraceEntry {
  readonly kind: "option-value" | "argument-value" | "literal" | "custom";
  readonly rawInput: string;
  readonly consumed: readonly string[];
  readonly preliminaryResult?: ValueParserResult<unknown>;
  readonly optionNames?: readonly string[];
  readonly metavar?: string;
}

This would let primitives stay dependency-agnostic while still giving the dependency runtime enough information to replay derived parsers with actual dependency values later.

Dependency runtime context

I think the dependency system should have a dedicated runtime context rather than distributing its state across parser states and wrapper markers.

export interface DependencyRuntimeContext {
  readonly registry: DependencyRegistryLike;

  registerSource(
    sourceId: symbol,
    value: unknown,
    origin: DependencyValueOrigin,
  ): void;

  hasSource(sourceId: symbol): boolean;
  getSource(sourceId: symbol): unknown;

  resolveDependencies(
    request: DependencyRequest,
  ): DependencyResolution;

  getReplayResult(key: ReplayKey): ValueParserResult<unknown> | undefined;
  setReplayResult(key: ReplayKey, result: ValueParserResult<unknown>): void;

  getSuggestionDependencies(
    request: DependencyRequest,
  ): DependencyResolution;
}

export type DependencyValueOrigin =
  | "cli"
  | "default"
  | "config"
  | "env"
  | "prompt"
  | "derived-precomplete";

export interface DependencyRequest {
  readonly dependencyIds: readonly symbol[];
  readonly defaultValues?: readonly unknown[];
}

export interface DependencyResolution {
  readonly kind: "resolved" | "partial" | "missing";
  readonly values: readonly unknown[];
  readonly usedDefaults: readonly boolean[];
}

export interface ReplayKey {
  readonly path: readonly PropertyKey[];
  readonly rawInput: string;
  readonly dependencyFingerprint: string;
}

I think the origin field is especially useful because it gives the runtime more semantic information than the current registry model, which mostly cares only about presence/absence.

Dependency metadata

Instead of manufacturing dependency-specific parser states, I think wrappers should compose internal dependency metadata/capabilities.

export interface DependencySourceCapability {
  readonly kind: "source";
  readonly sourceId: symbol;
  readonly getMissingSourceValue?:
    (exec: ExecutionContext) =>
      ValueParserResult<unknown> | Promise<ValueParserResult<unknown>>;
  readonly preservesSourceValue: boolean;
}

export interface DerivedDependencyCapability {
  readonly kind: "derived";
  readonly dependencyIds: readonly symbol[];
  readonly getDefaultDependencyValues?: () => readonly unknown[];
  readonly replayParse:
    (rawInput: string, dependencyValues: readonly unknown[]) =>
      ValueParserResult<unknown> | Promise<ValueParserResult<unknown>>;
  readonly replaySuggest?:
    (prefix: string, dependencyValues: readonly unknown[]) =>
      Iterable<Suggestion> | AsyncIterable<Suggestion>;
}

export interface DependencyTransformCapability {
  readonly transformsSourceValue: boolean;
}

export interface ParserDependencyMetadata {
  readonly source?: DependencySourceCapability;
  readonly derived?: DerivedDependencyCapability;
  readonly transform?: DependencyTransformCapability;
}

This would let us model wrappers in terms of capabilities:

• optional(P) preserves dependency metadata but does not itself provide a missing source value
• withDefault(P, value) may add a missing-source provider if P preserves a source
• map(P, f) preserves derived relationships but breaks source preservation
• bindConfig(P), bindEnv(P), and prompt(P) become external resolution providers rather than dependency-state protocols

Runtime snapshot

I think shared-buffer constructs should build a common internal snapshot shape and then hand that snapshot to a dedicated dependency runtime.

export interface RuntimeNode {
  readonly path: readonly PropertyKey[];
  readonly scopePath: readonly PropertyKey[];
  readonly parser: Parser<Mode, unknown, unknown>;
  readonly state: unknown;
}

export interface RuntimeSnapshot {
  readonly kind: "root" | "object" | "tuple" | "merge" | "concat";
  readonly nodes: readonly RuntimeNode[];
  readonly exec: ExecutionContext;
}

The distinction between path and scopePath matters most for merge(), where the final result location and the internal traversal/caching scope are not always the same thing.

State simplification targets

If this redesign works, I think the end state should look roughly like this:

export interface ParsedValueState<T> {
  readonly matched: boolean;
  readonly result?: ValueParserResult<T>;
}

export type OptionState<T> = ParsedValueState<T | boolean> | undefined;
export type ArgumentState<T> = ParsedValueState<T> | undefined;

export type OptionalState<TState> = {
  readonly matched: boolean;
  readonly innerState?: TState;
};

export type WithDefaultState<TState> = {
  readonly matched: boolean;
  readonly innerState?: TState;
};

That is, parser states become ordinary local state again, and dependency meaning moves into execution context plus dependency metadata.

Why I think these sketches are useful

I do not mean these exact types should be copied verbatim, but I think writing them down is useful for two reasons:

• they make the intended architecture concrete enough to guide implementation
• they make it easier to evaluate whether a proposed PR is actually moving responsibility out of parser states and into the new runtime

If this general direction looks reasonable, I think the first implementation PR should still be the semantics-locking regression matrix, but these type sketches can serve as a reference point for the execution-context and runtime work that would follow.