feat(inputs): defer file enumeration to git ls-files (Turbo/Nx parity, v14) - #69
Merged
Conversation
`src/cache/inputs.ts` now uses `git ls-files --cached --others --exclude-standard` when the project is inside a git repo, falling back to the pre-v14 `Bun.Glob` + `ignore`-library walker when it isn't. Matches what Turborepo and Nx both do at the bottom of their hash pipelines. ## What changes for users 1. **Nested `.gitignore` is correctly anchored.** Pre-v14, a project- level pattern like `pkg/.gitignore: src/skip.ts` was evaluated against workspace-relative paths and never matched. v14 defers to git, which gets nested anchoring right. 2. **`.git/info/exclude` and global excludes are honored** — they were invisible to the v13 walker. 3. **Untracked-but-not-ignored files enter inputs immediately.** No `git add` required; matches user intuition. ## Fallback When there's no `.git` directory, no git binary, or git fails for any reason, we fall back to the previous walker. Behavior is unchanged for non-git workspaces. ## Implementation - New private helper `listGitTrackedFiles(projectDir)` in `cache/inputs.ts` — synchronous `Bun.spawnSync` of `git ls-files --cached --others --exclude-standard -z .`, returns project-relative paths or `null`. - `resolveFiles` branches on the result: git path matches the user's globs against the git-tracked set; FS-walker path keeps the existing behavior. - `existsSync` guard filters deleted-but-tracked entries so the hasher doesn't throw ENOENT. - Always-ignored + nested-project boundary + declared-outputs exclusions apply on BOTH paths (defense in depth). ## CACHE_VERSION Bumped v13 → v14. The file set for `inputs.files: ['**/*']` can differ when a project had mis-handled nested gitignores; bumping forces a one-time cache invalidation so users don't pick up stale entries written under the old enumeration. Pre-alpha tolerates this. ## Tests 9 new git-path tests in `tests/inputs.test.ts` — each init's a real git repo in the fixture and verifies: - Nested .gitignore patterns correctly anchored (the v13 footgun). - Untracked-but-not-ignored files included. - Workspace-root .gitignore exclusion. - .git/info/exclude honored. - Deleted-but-tracked files skipped (existsSync guard). - Declared outputs still excluded. - Nested-project boundary still excludes. - Negation in inputs.files still strips. - node_modules always-ignored even when force-added to git. All 23 pre-existing FS-walker tests continue to pass (they don't git-init the fixture → fallback path). 414 pass, 0 fail (was 405). ## Docs - docs/modules/inputs.md — rewrote "File resolution rules" for v14. - docs/caching.md — CACHE_VERSION sentinel + history entry. - docs/modules/cache.md — CACHE_VERSION sentinel. - CLAUDE.md — decision log entry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adopt the Turbo / Nx file-enumeration model: ask git which files belong to the project, rather than walking the filesystem and parsing
.gitignoreourselves.When the project is inside a git repo,
src/cache/inputs.ts:resolveFilesnow usesgit ls-files --cached --others --exclude-standard -z .. The user'scache.inputs.filesglobs are applied as a filter over that file set. When git isn't available (no.git, missing binary), we fall back to the pre-v14Bun.Glob+ignore-library walker.Why
Both Turbo and Nx defer to git at the bottom of their hash pipelines. We were trying to do gitignore-parsing ourselves with the
ignorenpm package and getting it subtly wrong:.gitignorepatterns were anchored to the workspace root instead of the gitignore's own directory, sopkg/.gitignore: src/skip.tsnever matched..git/info/excludeand global excludes were invisible to us.Letting git handle the cascade is correct by construction.
User-visible effects
.gitignoreanchored correctly (fixes the v13 footgun)..git/info/exclude+ global excludes participate.git addneeded).CACHE_VERSIONv13 → v14The file set for the same
inputs.files: ['**/*']can differ when a project had mis-handled nested gitignores. The bump forces one-time invalidation so users don't pick up stale entries under the old enumeration. Pre-alpha tolerates this.Tests
9 new tests in
tests/inputs.test.ts(eachgit inits a real repo in the fixture):.gitignorepatterns correctly anchored (the v13 footgun)..gitignoreexcludes via git..git/info/excludehonored.existsSyncguard).inputs.filesstill strips.node_modulesalways-ignored even when force-added to git.All 23 pre-existing FS-walker tests continue to pass — they don't
git initthe fixture, so they exercise the fallback path.414 pass / 0 fail (was 405).
Test plan
bun src/bin.ts run lint— cleanbun src/bin.ts run format-check— cleanbun test— 414 pass, 0 failGenerated by Claude Code