Quest: Ensure every specification requirement has exactly one test

[taras]

Quest outcome

Give every requirement listed in a specification test plan exactly one owning
test.

Use a checked-in Markdown test whenever a document can construct and observe the
behavior. Use TypeScript only when the behavior cannot be produced or observed
through a document.

A maintainer can search for any requirement ID and find:

• its specification and exact location;
• its implementing test file; and
• the concrete <Test> or it() name that proves it.

This Quest governs explicit test-plan requirements. It does not turn every
normative sentence in every specification into a test ID automatically.

Example

Suppose requirement TD5 says that <TempDir> removes its temporary directory
after successful execution.

The checked-in Markdown document contains:

<Test name="TD5 Cleanup on success">
  ...
</Test>

Before removing a duplicate TypeScript test, the migration deliberately breaks
cleanup and proves that this Markdown test fails. After restoring the behavior,
the TypeScript duplicate is removed.

The traceability index then reports:

TD5
specification: specs/executable-mdx-spec.md
test file: packages/core/tests/TempDir.test.md
test: TD5 Cleanup on success

A requirement that a document cannot construct or observe remains in TypeScript
and appears in the same index.

Why this is needed

The executable-MDX specification currently contains a large test-plan matrix
derived from TypeScript test names. Markdown documents often explain and prove
several of those requirements, while TypeScript repeats the same claims.

Some rows describe an entire “colocated document” instead of the individual
requirements that document proves. Some IDs are duplicated, grouped, or ranged.
Nothing mechanically proves that a requirement has an owning test or that one
test is not claiming an unknown requirement.

The result is difficult to review:

• a specification row may have no test;
• the same behavior may be proved at several layers;
• a Markdown test may cover several requirements without identifying them;
• a TypeScript duplicate may disagree with the reader-facing test; and
• searching for an ID does not reliably find its evidence.

Placement rule

Give a requirement to the first testing layer that can faithfully construct and
observe it:

1. Colocated Markdown test: component and directive behavior that needs no
   external fixture or repository search path.
2. Hosted Markdown scenario: behavior a document can express when a thin
   TypeScript runner supplies external doubles or earlier attempts.
3. Nested document execution: behavior of a complete document under
   <Execution host="run"> or, once Add <WorkflowRun> and <Execution host="workflow"> to Markdown tests #547 is delivered,
   <Execution host="workflow">.
4. TypeScript: parsing, serialization, internal journal structure, recovery
   state, a process that must actually die, API calls that must not occur,
   command arguments and process exit behavior, runtime portability, or another
   fact a document cannot construct or observe.

A thin TypeScript runner may provide a boundary the document cannot create. The
behavior and assertions still belong in the Markdown scenario when the document
can observe them.

Exactly once

When Markdown owns a requirement, remove the TypeScript test that proves the
same claim.

A second test remains only when it catches a distinct failure the first test
cannot detect. That distinct failure receives its own requirement ID and a name
that says what it proves.

Do not multiply requirements or tests for equivalent inputs. One test may use a
table of representative values when every row exercises the same rule.

Keep separate requirements when they have different structural consequences.
For example, visible output and proof that an unselected branch never expanded
are different observations even if one implementation decision affects both.

Proving that the selected test is sufficient

Before deleting a duplicate test:

1. identify the requirement's observable failure;
2. make a temporary local change that violates it;
3. run the proposed owning test and confirm that it fails for that requirement;
4. restore the implementation;
5. remove the duplicate test;
6. repeat the temporary violation against the remaining test; and
7. record the result in the migration evidence.

Use an ordinary file edit for the temporary fault and restore it explicitly.
Never weaken an assertion merely to make the ownership rule pass.

Traceability check

Add a repository fitness test that reads specification test-plan tables and the
IDs in every <Test name> and it() name.

For each requirement ID, it reports deterministic, searchable output containing:

• specification file and location;
• test file; and
• concrete test name.

The check reports:

• a requirement with no test;
• more than one test claiming the same requirement;
• a test claiming an unknown requirement;
• a duplicated requirement ID; and
• an ambiguous grouped or ranged ID.

Grouped and ranged requirements are migrated to explicit IDs unless one ID
genuinely describes one requirement.

Reuse the repository's existing test-file discovery. The traceability check does
not implement another directory walk or another definition of the test corpus.

The check begins in warning mode while the existing corpus is migrated. It
becomes required when every tier is complete.

Migration process

Migrate one related specification tier at a time:

1. inventory its requirements and current tests;
2. remove duplicate or ambiguous requirement rows;
3. decide the owning layer for each remaining requirement;
4. add the requirement ID to the owning test name;
5. perform the temporary-fault check;
6. remove duplicate tests;
7. run the traceability check; and
8. update this Quest's child map.

The first implementation migrates the TempDir tier end to end and establishes
the rule, documentation, index, and warning check.

Create dependent child stories for:

• If and Loop, including equivalent-input consolidation;
• File;
• document properties and command behavior;
• JSON;
• Let;
• <Elicit> and <Answers>;
• Glob behavior; and
• every remaining test-plan tier.

Group tiers only when they share one testing boundary and can be reviewed
without hiding requirement ownership.

Migration constraints

• A Markdown runner must prove that tests actually ran. A render with every
  <Test> skipped is not success.
• A test never renders a credential or secret to prove a condition.
• Function-component doubles remain TypeScript components when a Markdown stub
  would change registration or projection behavior.
• Behavior that depends on default component registration cannot move into a
  colocated document that resolves different components.
• Runtime-specific behavior remains in portable TypeScript coverage when a
  Markdown test runs under only one runtime.
• Workflow lifecycle requirements wait for Add <WorkflowRun> and <Execution host="workflow"> to Markdown tests #547 rather than receiving a false
  ordinary-run proof.

Quest progress

Foundation

• Document the placement and exactly-once rules in AGENTS.md and
  specs/testing-spec.md.
• Add the searchable traceability index and warning check.
• Migrate the TempDir tier as the template.

Corpus migration

• Create and complete the child migration stories.
• Resolve every missing, duplicated, grouped, ranged, or orphaned ID.
• Remove unnecessary cross-layer test duplication.
• Enable the traceability check as a required failure.

Add each child issue to this map when created. This Quest remains open until the
complete explicit test-plan corpus satisfies the rule.

Completion

The Quest is complete when:

• every explicit test-plan requirement has one unambiguous ID;
• every requirement ID maps to exactly one owning test;
• every identified test maps to a known requirement;
• every surviving second observation has its own distinct requirement;
• the searchable index identifies the specification, test file, and test name;
• all child migrations are complete; and
• CI rejects missing, duplicated, ambiguous, or orphaned mappings.

Out of scope

• Treating suite-time reduction as acceptance.
• Keeping duplicate tests merely because they use different languages.
• Moving a requirement to Markdown before the necessary testing profile exists.
• Creating a test for every prose sentence outside an explicit test plan.