Skip to content

perf: collect_mds_files eagerly walks excluded dirs to count .mds files — fixed FS cost per run #238

Description

@dean0x

Problem

The count_mds_in_excluded_dir() function in crates/mds-cli/src/output.rs (line ~321) recursively walks entire excluded directory trees (node_modules, dot-dirs, etc.) on every mds invocation to count .mds files for the excluded_by_default diagnostic. This is a fixed, kernel-bound overhead that does not scale with the input corpus.

Impact

With typical directory structures (e.g., 4 node_modules trees totaling ~5,632 entries / 733 dirs), this adds 18–24ms of FS-bound time per run, increasing wall time by ~86% for a release binary against corpora that otherwise complete in ~21ms. The cost is constant across debug/release builds.

Root Cause

The diagnostic is only semantically meaningful when the walker's final file list is empty—to distinguish "empty tree" (nothing to process) from "all files excluded" (matched nothing). However, the current implementation computes the count unconditionally on every run.

Suggested Remediations (in order of preference)

  1. Lazy evaluation: Compute excluded_by_default only when files.is_empty() at the end of the walk.
  2. Bounded walk: Cap the recursion depth or count only direct .mds children in excluded dirs, not the full subtree.

Scope

Affects all directory-mode subcommands (build, check, lint, fmt) and possibly watch mode's initial scan.

Detection

Surfaced during PR #237 perf validation. On a clean corpus (no npm detritus), the engine has no regression; the overhead is entirely from eager directory walking.


Note: This is not fixed by PR #237; it is a separate refactoring task.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions