Skip to content

✨ Address a root document's sections as targets - #421

Draft
taras wants to merge 3 commits into
mainfrom
agent/issue-412-document-targets
Draft

✨ Address a root document's sections as targets#421
taras wants to merge 3 commits into
mainfrom
agent/issue-412-document-targets

Conversation

@taras

@taras taras commented Aug 9, 2026

Copy link
Copy Markdown
Owner

PR A of the #412 stack. Rebased onto main at 30dc295 after #413 merged.

Revised twice after architecture review. 568591e fixed a shipped defect: a journal written by a selector that matched nothing answered a later request for a section the document really has. 0422f60 closes the recorded-selection protocol so a corrupted record fails closed, and makes the error boundary reconstruct instead of adopting a candidate. See Revisions below. Masking, projection, source-position identity, exact-target recording, and the check-phase guard are unchanged throughout.

Why

A document that holds several sections is all-or-nothing today: running it runs every component, code block, and resource in it. #412 makes a section addressable, so an author can run one part of a document — and so a future workflow run can record which part it ran.

This PR builds the core layer only: the target model, the public API, and durable identity. xmd targets, targeted xmd run (PR B), and the version-2 workflow definition (PR C) come next.

What changes

Before:

execute() and inspectDocument() take a root document and nothing else. The whole body runs. DocumentInfo describes frontmatter and the return mode.

After:

Both accept an optional target selector. A resolved target projects the document to the preamble, the direct content of every ancestor needed to reach the heading, and that heading's complete subtree — sibling subtrees never expand. DocumentInfo gains targets (the catalog) and target (the exact canonical selection). A fileSource("README.md#Test/Node") reference splits a path from a selector.

$ # programmatically, today
yield* execute({ path: "README.md", stream })          # the whole document

$ # programmatically, with this PR
yield* execute({ ...fileSource("README.md#Test/Node"), stream })

How it works

fileSource(reference) → inspect/execute → mask components → outline → resolve selector
  → retained ranges → scan each under its own origin → expand

Three decisions carry the design.

Heading discovery parses a masked copy. A Markdown parser reading raw XMD cannot tell a component's children from the root flow: Remark ends the HTML block it infers for <Wrapper> at the first blank line, so a # line among the children surfaces as a root heading. Discovery therefore blanks the boundary scanner's top-level component spans to spaces of the same length before parsing. Lengths, newline positions, and everything outside those spans are untouched, so the mask changes what is seen, never where anything is. The scanner stays the authority on what a component is; it just records its spans now.

Projection retains source ranges rather than rebuilding a document. Each retained range is scanned under the origin it has in the original file. Concatenating the ranges and scanning once would renumber every element after a skipped section, and a retained element's source position is exactly what its expansion ID is derived from.

The exact resolved target is identity; the selector is not. **/N* and Test/Node are the same request if they name the same section. The durable root import records the selection outcome — the whole document, one exact target, or one structural failure — and a replay guard resolves the current selector against the recorded content before deciding whether the journal still describes this run. A caller's glob never occupies the exact-target field and never reaches a workflow definition.

Review guide

Start with: packages/core/tests/document-targets.test.ts — the contract in executable form.

Then review:

  1. specs/executable-mdx-spec.md §5.4 “Document targets” — the normative contract.
  2. packages/core/src/document-targets.ts — discovery, encoding, selectors, projection.
  3. packages/core/src/execute.tsdurableImportComponent's root branch and refuseChangedRootTarget.
  4. packages/core/src/definition.ts, root-source.ts, inspect.ts, scanner.ts.

Look carefully at:

  • packages/core/src/execute.tsholdRootSelection() and recordedRootImport(). The selection outcome comparison is where the shipped defect lived.
  • Why the replay guard validates in the check phase, not decide. durableRun reuses a recorded root Close before any effect is replayed (packages/durable-streams/run.tsrunCheckPhase runs first, then replayIndex.hasClose). A decide-phase guard would never run for a completed journal, which is precisely the run whose recorded target must still be the one being asked for. TX18 is the test that pins this.
  • maskComponents() preserves offsets exactly. If it ever stopped doing so, every retained element's expansion ID would shift silently.
  • Selector matching is a reachability sweep, not backtracking. DT27 would not fail against an exponential matcher — it would never terminate.

What must stay true

  • Cataloging cannot see headings hidden inside executable components — enforced by the masked parse and checked by DT13. Replacing maskComponents() with the identity function fails DT13 and nothing else.
  • A selector resolves to exactly one entry, before expansion and before any authored effect — enforced by selectTarget() raising DocumentTargetError, checked by DT23, TX12, TX13.
  • Projection preserves original positions and expansion IDs — enforced by per-range scanning with per-range origins, checked by DT40, DT41 (CRLF), TX4, TX5.
  • Durable replay validates and records the selection outcome, never the glob — enforced by holdRootSelection(), checked by TX15–TX21 and TX24–TX27. Removing the guard installation fails TX18–TX21 together.
  • A failed selection is part of the record — enforced by the target-failure selection variant, checked by TX24 (the shipped defect) and TX25–TX27.
  • DocumentTargetError is recognized structurally, across loaded copies, and reconstructed locally — enforced by a namespaced tag plus a total validating boundary that builds a fresh error, checked by DT52–DT61.
  • A malformed recorded selection fails closed before Close reuse — enforced by the closed root-import protocol, checked by TX28–TX33.
  • An exact canonical target survives decode → normalize → re-encode unchanged — enforced by isCanonicalTarget(), checked by DT48–DT51.
  • Existing journals stay readable — enforced by omitting the target member entirely for an untargeted import, checked by TX16 and TX22.

How to verify it

  • DT13 proves a component child's apparent headings are never targets, and fails against any regression to raw Remark discovery.
  • DT27 proves a wildcard-dense selector against a 120-character label terminates; a backtracking matcher hangs the suite rather than failing the assertion.
  • TX2 proves a skipped sibling's code block never runs — asserted from an exec provider that throws, not from absent output.
  • TX4/TX5 prove a retained element's expansion ID is identical in a full run and under two different targets; seeding identity with the target string breaks all three at once.
  • TX18 proves a completed journal recorded against Alpha refuses to be reused for Beta. It fails if the guard moves to the decide phase.
  • TX23 proves replay projects the recorded content: the file is rewritten between the two runs and the replayed output is still the first run's.
  • TX24 is the shipped defect: a journal from Missing must not answer a later request for Beta. It fails against the first commit.
  • TX28–TX33 start from a valid failed-selection journal, corrupt only the record, and resume twice — with the failing selector and with a valid one. Collapsing malformed back into unrelated fails TX28–TX32 together and nothing else.
  • DT54–DT56 prove the returned error is a fresh local one whose data survives mutation of the foreign arrays and revocation of their Proxy.
  • TX25–TX27 prove the same failing selector replays its own failure with no authored effect, that a different failure kind or selector is stale, and that live and replayed failures are the same structural error.
  • DT52–DT55 prove a failure from a separately loaded copy is recognized while hostile, mutable, cause-bearing, payload-bearing, and unreadable candidates are refused.
  • DT48–DT51 prove Cafe%CC%81, a%09b, a raw #, NUL, and unpaired surrogates are refused, and that every formatted reference parses back to what it named.

Every "did not run" assertion comes from a component that records its own invocation, because absent text cannot distinguish "skipped" from "rendered empty".

deno task lint && deno check --frozen packages scripts && deno task check:jsr
deno task test packages
deno task test packages/core/tests/document-targets.test.ts \
  packages/core/tests/document-target-execution.test.ts

Results on 0422f60:

Check Result
deno task lint 0 errors
deno task check from a clean clone, unscoped clean
deno task check:jsr Success Dry run complete (host tree and clean clone)
deno task test packages 367 passed, 0 failed
the two focused suites 94 passed, 0 failed
tsx --tsconfig tsconfig.node.json --test (Node 22.23.2, both suites) 94 passed, 0 failed
tsc --project tsconfig.node.json exit 0
bun test (both suites) 94 passed, 0 failed
git diff --check clean

The full local Deno suite was run rather than an affected subset, because replay boundaries make graph-based selection incomplete. The clean clone settles an earlier note in this PR: the ~51k errors an unscoped deno task check reported came entirely from untracked spikes/349-dofs/vendor in the host tree, not from the repository.

Scope

Included

  • Target catalog, canonical encoding, and the selector grammar.
  • Projection with original-position preservation.
  • fileSource(), formatDocumentReference(), DocumentInfo.targets/.target, DocumentTargetError.
  • Exact-target durable identity and the replay guard.
  • architecture.md and specs/executable-mdx-spec.md.

Intentionally unchanged

  • The CLI. No file under packages/cli is touched; xmd targets and targeted xmd run are PR B.
  • The workflow package. Version-2 definitions are PR C.
  • xmd test. It keeps its existing path grammar and gains no target support.
  • API.Files, its adapters, and 💥 Contain document filesystem access behind API.Files (#227) #403's fatal-cause precedence.
  • The untargeted execution path, which still scans the whole body and skips the outline parse entirely — an untargeted execute() does no extra work.

New abstractions

  • ComponentSpan / scanComponentSpans() exists because heading discovery needs the scanner's own boundary decision, and re-deriving it with a second parser is what this PR exists to avoid. Internal to the package.

  • DocumentOutline / DocumentTarget exist because discovery, selection, and projection are three passes over one structure; consumers are parseRootMarkdownDefinition() and resolveDocumentTarget().

  • DocumentTargetError exists because a caller has to tell "you named nothing" from "you named several", and needs the catalog to say so usefully. PR B's diagnostics are its second consumer.

  • Each new abstraction has multiple concrete uses or a clear justification.

  • No speculative functionality is included.

New dependencies

None. remark and mdast-util-to-string were already @executablemd/core dependencies. Neither lockfile changes.

Revisions

First — 568591e

Architecture review of the initial commit found one shipped defect and four contract gaps. 8ede2f6 addresses all five.

The defect. Reproduced before it was fixed:

run 1 (Missing): "Missing" matches no document target.
run 2 (Good):   ERR -> "Missing" matches no document target.

The root import failed when a selector matched nothing, so the journal kept only a serialized message, the guard delegated past every result.status === "err", and the completed root Close short-circuited everything after it.

A failed selection is an observation of the document, not an accident: the text was read, and it does not offer what was asked for. It is now recorded as one — kind, selector, matches, catalog — and the failure is rebuilt from that record and raised. The guard compares whole selection outcomes rather than target strings, reproduces a recorded failure before the recorded Close can be reused, and reports every difference as StaleInputError carrying no foreign object. No authored effect runs on either path, and no human-readable message is parsed as protocol.

Structural error on every public path. DocumentTargetError carries frozen data under a stable namespaced tag; isDocumentTargetError / asDocumentTargetError / parseDocumentTargetFailure recognize it totally, so a failure from a separately loaded copy is recognized on the same terms as a local one, while a candidate carrying a cause, extra payload, mutable data, a disagreeing message, or an unreadable property is refused. The §5.4 caveat saying execute() exposes only name and message is removed, because it no longer does.

Canonical validation. A level is canonical only when decode → normalize → re-encode reproduces it byte for byte, which refuses NFD, tabs, uncollapsed and edge whitespace, lowercase escapes, empty levels, raw operators, and a raw #. formatDocumentReference() checks its own round trip rather than sampling it, rejecting NUL and unpaired surrogates.

Architecture. The locked workflow definition term now includes the exact canonical target when one is selected, and ## Document targets states the distinction directly: the exact target is definition identity; a caller glob is non-authoritative invocation metadata that never substitutes for it.

Second — 0422f60

Fail closed on a malformed recorded root selection. recordedRootImport() returned one absent value for "not the root import" and "the root import, malformed", so a corrupted record was delegated onward and durableRun reused the recorded terminal result. The record is now a closed protocol with three answers — unrelated, malformed, read — and two supported shapes: a repository selection with an optional exact canonical target, and a failed selection with an exact failure record. An unknown kind, a missing, unreadable, mistyped or extra member, and a noncanonical target are malformed, and malformed throws one fixed cause-free diagnostic from the check phase, ahead of Close reuse, delegating nothing and appending nothing.

The record is verified, not merely parsed: it carries the content it was taken from, so a recorded target must resolve to itself in that content and a recorded failure must be exactly the failure its selector produces. That is what refuses available: ["../../etc/passwd"] — a structurally canonical four-level path the recorded document does not offer.

A closed, locally owned recognition boundary. asDocumentTargetError() no longer returns the candidate; it validates every field and builds a fresh local error. An ordinary invocation failure has no fail-stop reason to preserve identity, and returning the candidate hands on whatever it owns — a list its owner can still rewrite, a revocable Proxy, a prototype with accessors. The data contract is closed to exactly five members with no symbol or non-enumerable extras, every list entry must be an exact canonical target, and the fields must describe an outcome selection could have reached. The "handed onward by identity" claim is removed from the specification.

Two implementation gaps were found by the new rows rather than assumed away: the journal failure record did not close its member set, and shape checking alone accepted a catalog the recorded document contradicts.

Risks and limitations

  • A failed selection is recorded as a successful effect carrying a failure outcome, not as a failed effect. That is the change that makes resumption correct, and it means the root import's journal entry is ok even when the run failed. Reviewers comparing journal shapes across versions should expect this.
  • Two different selectors that both match nothing are two different requests, so resuming with the second is stale input rather than a replay. This is deliberate — the recorded failure describes the request that was made — but it is stricter than "any failure replays any failure".
  • The document-reference grammar is deliberately URI-style, so a path containing a literal %HH sequence must now be written %25HH and one containing # must be written %23. This has no reachable consumer until PR B routes CLI arguments through fileSource(); release notes belong with that PR.
  • Recovery: the change is additive. Untargeted programmatic execution, inlineSource(source), output, and expansion IDs are unchanged, and reverting the commit restores the previous behavior with no data migration.

Scope confirmation

  • Every changed file supports the purpose described above.
  • Unrelated cleanup and formatting changes are excluded.
  • Generated or mechanical changes are clearly identified — there are none.
  • The description matches the final diff and test results.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

PR #421: ✨ Address a root document's sections as targets

11 files, +3791 / -55

Scope

🔴 PR has 3846 lines changed. Split into focused PRs.

🟡 3846 lines changed. PRs under 400 receive more thorough review.

Structural

Oxlint structural signals:

  • no-unnecessary-type-assertion ×40: packages/core/src/scanner.ts, packages/core/src/definition.ts, packages/core/src/document-targets.ts (+1)
  • no-empty-function ×2: packages/core/src/execute.ts

Slop

✅ Slop indicators look low.

Static Analysis

Oxlint: 61 diagnostics across 5 files (9 rules)
Density: 0.016 violations/added-line

no-unnecessary-type-assertion (40): packages/core/src/scanner.ts, packages/core/src/definition.ts, packages/core/src/document-targets.ts (+1)
no-new-array (5): packages/core/src/definition.ts, packages/core/src/document-targets.ts
no-unsafe-type-assertion (5): packages/core/src/execute.ts, packages/core/src/scanner.ts
no-array-sort (3): packages/core/src/document-targets.ts
consistent-return (3): packages/core/src/inspect.ts, packages/core/src/execute.ts
no-empty-function (2): packages/core/src/execute.ts
no-useless-spread (1): packages/core/src/document-targets.ts
unbound-method (1): packages/core/src/execute.ts
no-floating-promises (1): packages/core/src/execute.ts

Correctness

No extraneous code patterns detected.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found 1 redundant comment. Inline suggestions to remove them below.

}
// Only the outer edges are trimmed: whitespace beside a wildcard is part of
// what the author asked to match, while the whole level is compared against
// an already-trimmed label.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant comment — restates what the code does.

Suggested change
// an already-trimmed label.

taras added 3 commits August 9, 2026 14:36
A root document now catalogs its own addressable static headings, resolves one
selector to exactly one of them, and projects itself down to the preamble, each
ancestor's own content, and that section's subtree before anything expands.

Heading discovery parses a masked copy of the body, with the boundary scanner's
top-level component spans blanked to spaces of the same length. Remark ends the
HTML block it infers for a component at a blank line, so a component child's
`#` line surfaces as a root heading without the mask; offsets, lines, and
everything outside those spans are untouched, so the mask changes what is seen
and never where anything is.

Projection retains original source ranges and scans each one under its own
origin instead of concatenating and rescanning, so skipped source cannot
renumber what follows it and a retained element keeps its expansion ID.

The exact resolved target, never the caller's glob, is what the durable root
import records. A replay guard resolves the current selector against the
recorded content and requires the recorded exact target, in the check phase so
a completed journal cannot answer for a section it never ran.

`xmd targets`, targeted `xmd run`, and the targeted workflow definition are the
later layers of #412 and remain unbuilt.
A root import whose selector matched nothing failed the effect, so the journal
kept only a serialized message and the replay guard delegated past every `err`
result. A completed journal written by `Missing` then answered a later request
for a section the document really has, with the old `Missing` error.

A failed selection is an observation of the document, so it is recorded as one:
its kind, the requested selector, the matches, and the catalog. The guard
compares whole selection outcomes rather than target strings, reproduces a
recorded failure from that record before the recorded Close can be reused, and
reports any difference as stale input carrying no foreign object.

`DocumentTargetError` now carries frozen, namespaced-tagged data and is
recognized structurally, so inspection, a live run, and a replayed run all raise
the same error with the same fields across separately loaded copies.

A level is canonical only when decoding, label normalization, and canonical
re-encoding reproduce it exactly, which refuses NFD, tabs, uncollapsed and edge
whitespace, lowercase escapes, empty levels, raw operators, and a raw `#`.
`formatDocumentReference()` only formats what `fileSource()` reads back.

The recorded selector is sanitized invocation metadata: architecture.md now
states that the exact canonical target is definition identity and a caller glob
never substitutes for it.
"Not the root import" and "the root import, malformed" were one absent value, so
a corrupted record fell through to the recorded terminal result and replayed an
outcome the record no longer describes.

The recorded root import is now a closed protocol: a repository selection with
an optional exact canonical target, or a failed selection with an exact failure
record. An unknown kind, a missing, unreadable, mistyped or extra member, and a
noncanonical target are malformed, and malformed fails with one fixed cause-free
diagnostic before the recorded Close can be reused — delegating nothing,
executing nothing, appending nothing.

The record carries the content it was taken from, so the selection is verified
against that content rather than merely parsed: a recorded target must resolve
to itself, and a recorded failure must be the failure that selector produces.

`asDocumentTargetError()` returns a fresh local error built from reconstructed
data instead of the candidate. An ordinary invocation failure has no fail-stop
reason to preserve identity, and returning the candidate hands on whatever it
owns — a list its owner can still rewrite, a revocable Proxy, a prototype with
accessors. The data contract is closed to exactly five members with no symbol or
non-enumerable extras, every list entry must be an exact canonical target, and
the fields must describe an outcome selection could have reached.
@taras
taras force-pushed the agent/issue-412-document-targets branch from 8ede2f6 to 0422f60 Compare August 9, 2026 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant