Skip to content

✨ feat: add a deterministic contextual <Glob> component - #229

Merged
taras merged 2 commits into
mainfrom
feat/glob-component
Jul 29, 2026
Merged

✨ feat: add a deterministic contextual <Glob> component#229
taras merged 2 commits into
mainfrom
feat/glob-component

Conversation

@taras

@taras taras commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Why

A document decides what to work on by looking at what is there — repository
instructions, test fixtures, source files, workflow inputs. Until now it had to
shell out to find or ls and parse the result, which is neither deterministic
nor something an author can assert against.

Closes #191. Follows #176 (schema-validated returns), #189 (<TempDir>), and
#190 (<File>).

What changes

<Glob> is a new built-in value component.

Before:

```sh exec
find . -name 'AGENTS.md'
```

— output that varies by host, by locale, and by directory order, with no way to
branch on it.

After:

<Glob include={["**/AGENTS.md"]} as="instructionPaths" />

— a string[] of relative POSIX paths, deduplicated and sorted by code point.

Because patterns are relative to the contextual Env.cwd, the three filesystem
components become a pipeline without any of them knowing about the others — a
path <Glob> returns is a path <File> can read:

<TempDir>
<File path="docs/guide.md">Guide</File>
<Glob include={["docs/**/*.md"]} as="docs" />
<Each in={docs} let="path">
<File path={path} />
</Each>
</TempDir>

How it works

<Glob include exclude as> → check patterns → stat Env.cwd → API.Fs.glob → files only → dedupe → sort

Patterns are checked before the filesystem is touched, Env.cwd is checked
before traversal, and matching itself belongs to the Fs Api — the component adds
no syntax of its own.

Review guide

Start with: packages/core/src/components/Glob.test.md — the public matching
contract, entirely in Markdown.

Then review:

  1. specs/executable-mdx-spec.md §6.14 — props, dialect, ordering,
    normalization, symlinks, failures
  2. packages/core/src/components/Glob.ts — the component
  3. packages/runtime/apis.ts — the two API.Fs.glob fixes below
  4. packages/core/tests/glob-component.test.ts — Tier GB
  5. packages/core/src/components/fs-diagnostics.ts + File.ts — the extracted
    errno allowlist
  6. site/routes/docs/components.tsx — website section

Look carefully at: the traversal in descend(). It is where containment,
cancellability, and exclusion pruning all live.

Two defects in API.Fs.glob

Neither of the issue's acceptance criteria for exclude or traversal failure
could hold without fixing these, so they are in scope.

exclude never matched anything. It was passed to @effectionx/fs's
walk({ skip }), which tests patterns against the absolute entry path — so
globToRegExp(".git/**"), anchored ^\.git\/…$, could never match
/tmp/x/.git/config. Matching now happens on the relative POSIX path.

A traversal failure was an uncaught rejection. walk produces entries from
a spawned task, so a readdir that threw tore down the surrounding scope
instead of throwing at the call site — no caller could catch or report it.
Verified before the change: a chmod 000 subdirectory crashed the process with
Uncaught (in promise) Error: EACCES past a try/catch around the call.
Traversal is now a plain recursive generator, which also makes every directory
read a cancellation point.

What must stay true

  • Traversal never leaves Env.cwd — enforced by never following a symlink
    (a link is not a directory to descend, and not a file to return), checked by
    GB15/GB15b/GB16/GB16b.
  • Traversal cannot cycle — same mechanism, checked by GB15c, which fails by
    timing out rather than hanging.
  • A listing is host-independent — dedupe plus code-point sort, checked by
    GB19 and by the ordering/dedup tests in Glob.test.md.
  • No absolute path reaches a diagnostic (§1.2) — every Api call is wrapped
    and the errno selects an allowlisted phrase; checked by GB12, GB12b, GB13,
    and four adversarial GB14 cases including an externally thrown GlobError.
  • An empty result means "no such files" — so a pattern that cannot match
    anything fails instead, checked by GB6/GB7/GB8.
  • Pruning never changes the answer — a subtree is skipped only when a
    trailing /** provably covers every descendant; every other exclusion walks
    and filters per file. Checked by Tier GA, which counts directory reads because
    a result set cannot tell a correct filter from an over-eager prune.

How to verify it

  • Glob.test.md (27 assertions) proves the matching contract with no
    JavaScript, and fails if include/exclude precedence, ordering, dedup,
    normalization, hidden-file semantics, or files-only filtering regress.
  • GB9 proves an uncompilable pattern is reported as an authoring error and
    fails if a raw RegExp message ("Unterminated character class") leaks.
  • GB7 proves refusing only a leading .. is sufficient: a .. further
    along (docs/../../../etc/passwd) matches nothing because traversal never
    produces such a path.
  • GB17 proves traversal is cancellable mid-walk — the second directory read
    suspends, the third is never read, and the document gets nothing.
  • GB18/GB18b prove replay: a completed root restores the array without
    searching; a partial journal searches again and sees newly added files.
  • GA2/GA2b/GA3 prove exclusions are read as written: foo/* keeps
    foo/deep/keep.md and still walks foo, and vendor alone removes nothing.
    They fail if a directory match is ever taken as proof about its descendants.
  • GA4/GA4b/GA5 prove pruning is sound and still happens: .git/** and
    **/node_modules/** read nothing below them, while the equally-total **/*
    walks and filters — same answer, different work.

Verified on Deno 2.9.1 (CI-pinned): lint, check, test (215 passed /
1723 steps), check:jsr all clean. Also test:node (1698/0), test:bun
(1698/0), site check, and ./dist/xmd test packages/core/src from the
compiled binary.

Scope

Included

  • <Glob>, registered as a built-in
  • The three API.Fs.glob fixes: relative-path exclusion matching, catchable
    traversal failures, and sound subtree pruning
  • Tier GA, which tests glob() directly rather than through any caller
  • The shared errno allowlist, extracted from File.ts
  • Spec §6.14 and website documentation

Intentionally unchanged

  • resolveTestTarget is the only other glob caller. It passes no exclude
    and filters on isFile, so its behavior is identical — confirmed by the
    test-target suite and by the compiled binary still discovering every
    document under packages/core/src.
  • API.Fs.glob keeps returning { path, isFile } and keeps reporting symlinks
    with isFile: false. Baking "files only" into the Api would be the
    component's opinion, not the Api's.
  • Following symlinks is still not offered. @effectionx/fs exposes
    followSymlinks but guarantees neither root containment nor cycle detection.

New abstractions

  • components/fs-diagnostics.ts exists because the errno-to-phrase allowlist is
    now needed by two components with the same constraint; duplicating fourteen
    entries would let them drift.
  • GlobError exists to mark the component's own failures, on the same terms as
    FileAccessError.
  • Each new abstraction has multiple concrete uses or a clear justification.
  • No speculative functionality is included.

Risks and limitations

  • Not a sandbox. <Glob> never follows a symlink, so nothing it reads is chosen
    by one — but a directory that is real when read could be replaced afterwards.
    Containment independent of observed filesystem state is Contain filesystem access at the provider boundary #227.
  • Subtree pruning is a syntactic decision: only a trailing /** earns it. A
    pattern that happens to cover a whole subtree by other means — a character
    class enumerating every child, say — is walked and filtered instead. That
    costs reads and never costs correctness, which is the direction to be wrong
    in.

Scope confirmation

  • Every changed file supports the purpose described above.
  • Unrelated cleanup and formatting changes are excluded.
  • Generated or mechanical changes are clearly identified.
  • The description matches the final diff and test results.

A document decides what to work on by looking at what is there. `<Glob>`
answers that question against the contextual `Env.cwd`, so it composes with
`<TempDir>` and `<File>` without any of them knowing about the others — a
path it returns is a path `<File>` can read.

It is a value component: `include` is required, `exclude` optional, and what
it binds is a *set* of relative POSIX paths, deduplicated and sorted by code
point. None of that comes from the order the filesystem handed entries back,
because a document that branches on a listing must branch the same way on
every host. Finding nothing is a result, not a failure.

Only regular files come back. A symlink is a link rather than a file, so it is
never a result and a link to a directory is never descended into — which is
also what keeps traversal inside `Env.cwd` and free of cycles, without judging
any destination.

A pattern that cannot match anything a relative search produces — absolute, a
leading `..`, or empty — fails rather than quietly contributing nothing: an
empty result has to keep meaning "there are no such files".

Two defects in `API.Fs.glob` had to be fixed for any of that to hold:

- `exclude` was passed to `walk`'s `skip`, which tests patterns against the
  absolute entry path, so `.git/**` never matched anything and excluded
  nothing. Matching now happens on the relative POSIX path, and an exclusion
  covering a directory's contents prunes the subtree.
- `walk` produces entries from a spawned task, so a `readdir` that failed tore
  down the surrounding scope as an uncaught rejection instead of throwing at
  the call site — no caller could report it. Traversal is now a plain
  recursive generator, which also makes every directory read a cancellation
  point.

`resolveTestTarget` is the only other caller; it passes no `exclude` and
filters on `isFile`, so its behavior is unchanged.

The errno-to-phrase allowlist moves out of `File.ts` into
`components/fs-diagnostics.ts`, shared by both filesystem components: a
platform error names the path it failed on, and for a search that is a path
the document never wrote.

Closes #191
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

PR #229: ✨ feat: add a deterministic contextual component

10 files, +1899 / -93

Scope

🔴 PR has 1992 lines changed. Split into focused PRs.

🟡 1992 lines changed. PRs under 400 receive more thorough review.

Structural

✅ No structural bloat detected.

Slop

✅ Slop indicators look low.

Static Analysis

✅ Oxlint found no issues.

Correctness

No extraneous code patterns detected.

Exclusion pruning was unsound in two ways, and both dropped files the
exclusion never named.

A directory whose own path matched an exclusion was skipped along with
everything under it. But a directory is not a candidate — it is never
reported — so matching it says nothing about the paths beneath it:
`exclude: ["vendor"]` returned nothing at all instead of every file under
`vendor`.

Matching `<directory>/` was then treated as proof that every descendant was
excluded. That is false for any pattern whose last segment stops at a
separator, because `*` may match zero characters but cannot cross one. With

    foo/direct.md
    foo/deep/keep.md
    include: ["**/*.md"]
    exclude: ["foo/*"]

`foo/*` matches `foo/`, so `foo` was pruned and the result was `[]` — where
the exclusion selects only `foo/direct.md` and `foo/deep/keep.md` should
survive.

Exclusion is now decided per candidate, against that candidate's own relative
path, and a directory's path is not tested against exclusions at all. A subtree
is skipped only when a pattern provably covers every descendant, which is a
trailing `/**` — it compiles to `(?:[^/]*(?:/|$))*`, so once the part before it
matches a directory the pattern matches every path underneath at any depth.
`**` alone covers the whole tree the same way. Every other exclusion walks the
subtree and filters its files individually, which is the conservative
direction: descending needlessly costs reads, while skipping wrongly loses a
match.

Pruning is now an optimization that cannot change the answer. `**/*` excludes
every file at any depth and earns no pruning, and the result is the same either
way.

Tier GA is new and drives `glob()` directly, so no component runs and nothing
a component could do afterwards can satisfy it. It separates filtering from
pruning by counting directory reads — a result set alone cannot tell a
correctly filtered walk from an over-eager prune, because the two can agree on
the answer and disagree on the work. `.git/**` and `**/node_modules/**` are
asserted to read nothing below them.

`Glob.test.md` gains the two contract-level regressions: `vendor/*` keeps
`vendor/deep/keep.md`, and `vendor` removes nothing.

The specification and website documented the unsound rule and now describe the
sound one.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found 2 redundant comments. Inline suggestions to remove them below.

}

// Code point order, not `localeCompare`: what a document branches on must not
// depend on the locale the host happens to be configured with.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant comment — restates what the code does.

Suggested change
// depend on the locale the host happens to be configured with.

Comment thread packages/runtime/apis.ts
}

// A symlink is reported by its own path and never followed, so traversal
// stays under the root and cannot cycle.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant comment — restates what the code does.

Suggested change
// stays under the root and cannot cycle.

@taras
taras merged commit bf6085f into main Jul 29, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a deterministic contextual Glob component

1 participant