Skip to content

feat(core): per-file content-addressed lint cache with cross-file sidecar - #900

Closed
rayhanadev wants to merge 7 commits into
mainfrom
ray/davao
Closed

feat(core): per-file content-addressed lint cache with cross-file sidecar#900
rayhanadev wants to merge 7 commits into
mainfrom
ray/davao

Conversation

@rayhanadev

@rayhanadev rayhanadev commented Jun 19, 2026

Copy link
Copy Markdown
Member

Why

Today the only lint cache is whole-repo all-or-nothing: a single dirty file, a new HEAD SHA, or a CI re-clone voids the entire payload and lint re-runs on every file. This adds a second tier — a per-file content-addressed cache — so repeat scans (local pre-commit loops, CI on incremental commits) re-lint only the files whose content actually changed, while staying byte-identical to a cold scan.

Before:

# whole-repo cache misses on ANY change → full re-lint of all ~1,274 files (~4.2s)
$ react-doctor          # edited 1 file → re-lints everything

After:

# unchanged files replay cached diagnostics; only changed files re-lint
$ react-doctor          # edited 1 file → re-lints ~1 file, replays the rest

What changed

  • Two-config sidecar split inside the oxlint runner: a cacheable config (all rules except the cross-file set) runs only on changed files; a sidecar config (only the 5 cross-file rules) always runs fresh on every file, so a dependency change can never serve a stale cross-file verdict on an unchanged file.
  • Content-hash keying (<relativePath> <sha1(content)> under a ruleset-hash bucket) — survives CI re-clones (mtime would not). The ruleset hash captures the normalized config + toolchain versions + ignore patterns; absolute paths are stripped for portability.
  • Caches the raw oxlint diagnostics (pre-presentation), so --no-warnings / suppressions / surface filtering never bust the cache.
  • CROSS_FILE_RULE_IDS (exactly 5, verified by transitive import-graph analysis) exported from oxlint-plugin-react-doctor; a guard test (cross-file-rule-ids.test.ts) fails if a future rule reads other files without being carved into the sidecar.
  • On by default; auto-bypassed in audit mode / adopted extends / user plugins. Opt out via REACT_DOCTOR_NO_FILE_CACHE=1 (REACT_DOCTOR_NO_CACHE=1 now disables both caches).
  • CLI emits lintCacheHitRatio on the Sentry run wide event (numeric; null when the cache was off/bypassed, threaded outside CachedScanPayload so a whole-repo cache-hit replay reads as "no lint ran").
  • Changeset: patch for react-doctor, @react-doctor/core, oxlint-plugin-react-doctor.

Test plan

  • pnpm typecheck — green (11/11 packages)
  • pnpm lint / pnpm format:check / pnpm smoke:json-report — green (schemaVersion=1, output unchanged)
  • pnpm test — green except the two known env-file gitignore-trap tests (check-security-scan/check-expo-project), which fail locally due to a global ~/.config/git/ignore .env* entry and are unrelated to this change
  • New tests: byte-identical (cache on/off), cache-stats, content-change invalidation, cross-file staleness (dependency changes, unchanged file gets the fresh verdict), audit-mode bypass, ruleset-hash portability/stability, file-cache round-trip/fail-open/malformed/LRU, and the cross-file-rule guard
  • The entire react-doctor e2e suite (1842 tests) runs through the cache-on split path by default and passes → byte-identical evidence across the fixture corpus

Note: no action.yml change — the GitHub Action's actions/cache step is a separate action release (the cache is transparent to it).

🤖 Generated with Claude Code


Note

Medium Risk
Large change to the core lint path with correctness invariants (sidecar split, store guards); mistakes could serve stale diagnostics, though extensive e2e byte-identical tests and the cross-file guard mitigate that.

Overview
Adds a default-on per-file lint cache so repeat scans replay raw oxlint diagnostics for unchanged files (keyed by path + SHA-1 content under a portable ruleset hash) and only re-lint misses. Scores, JSON reports, and inspect/diagnose output stay byte-identical to a cold run; the cache stores pre-presentation diagnostics so warning/suppression toggles do not bust entries.

The oxlint runner splits into a cacheable pass (within-file rules only, changed files) and an always-fresh sidecar for five cross-file rules exported as CROSS_FILE_RULE_IDS; a guard test fails if a new rule reads other files without joining that set. The cache is skipped for audit mode, adopted extends, user plugins, and React Compiler projects; opt out with REACT_DOCTOR_NO_FILE_CACHE=1 or global REACT_DOCTOR_NO_CACHE=1. Disk cache lives under node_modules/.cache/react-doctor with fail-open I/O, LRU caps, and conservative skip-on-partial-failure store rules.

Telemetry: lintCacheHitFileCount / ratio on inspect results and Sentry run events (lintCacheHitRatio).

Reviewed by Cursor Bugbot for commit 7db6e41. Bugbot is set up for automated code reviews on this repo. Configure here.

…ecar

Repeat scans re-lint only files whose content changed. The oxlint pass
partitions the file list by content hash: unchanged files replay their
cached raw diagnostics; only changed files re-lint. The five cross-file
rules (no-barrel-import, the two nextjs-*, no-mutating-reducer-state,
rn-prefer-expo-image) — whose verdict can depend on other files — always
run fresh in a never-cached sidecar pass, so a dependency change can never
serve a stale verdict on an unchanged file. Output is byte-identical with
the cache on or off, so the score, JSON report, and inspect()/diagnose()
returns are unchanged.

On by default and content-hashed (survives CI re-clones); auto-bypassed in
audit mode, with an adopted extends config, or with user plugins. Opt out
via REACT_DOCTOR_NO_FILE_CACHE=1 (REACT_DOCTOR_NO_CACHE=1 now disables both
caches). A cross-file-rules guard test fails if a future rule reads other
files without being carved into the sidecar. The CLI reports cache
effectiveness as lintCacheHitRatio on its Sentry run event.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pkg-pr-new

pkg-pr-new Bot commented Jun 19, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/eslint-plugin-react-doctor@900
npm i https://pkg.pr.new/oxlint-plugin-react-doctor@900
npm i https://pkg.pr.new/react-doctor@900

commit: 7db6e41

Comment thread packages/core/src/utils/resolve-react-doctor-cache-dir.ts
Comment thread packages/core/src/utils/resolve-react-doctor-cache-dir.ts Fixed
Comment thread packages/core/src/run-oxlint.ts Outdated
Comment thread packages/core/src/run-oxlint.ts

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Addresses two Cursor Bugbot findings on the byte-identical guarantee:

- Bypass the cache entirely for React Compiler projects. react-hooks-js
  can fail to LOAD at lint time (issue #833); a warm scan with zero misses
  never spawns the cacheable pass, so that failure would never trigger
  while stale React Compiler diagnostics keep replaying. (High)
- Never cache a dropped file as clean. A file dropped by the binary-split
  retry (timeout / OOM) produces no diagnostics — thread a hadPartialFailure
  flag out of the per-config pass and skip the store when it fired, so a
  later warm scan doesn't treat the file as a zero-finding hit. (Medium)

Also switches the tmp-fallback cache-dir path digest from SHA-1 to SHA-256
(it only names a per-project subdirectory; resolves a CodeQL weak-algorithm
alert — the "password hash" classification was a false positive).

Adds a regression test asserting the cache is bypassed when
project.hasReactCompiler is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread packages/core/src/runners/oxlint/file-lint-cache.ts Outdated
Addresses a Cursor Bugbot finding: persist() replaced our ruleset bucket
wholesale, so two overlapping scans sharing a ruleset hash could each erase
the entries the other just stored (extra cache misses on the next scan —
not wrong output). persist() now re-reads and MERGES same-ruleset on-disk
file entries with this run's, preserving a sibling run's work; our
freshly-linted entries win for any shared key. Adds a concurrent-persist
regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread packages/core/src/run-oxlint.ts Outdated
Comment thread packages/core/src/run-oxlint.ts Outdated
Addresses a Cursor Bugbot finding: onCacheStats fired right after the
content-hash partition, before the cacheable + sidecar oxlint passes ran.
If a pass then threw, run-inspect could still attach a cache-hit ratio to a
failed scan's telemetry. Move the callback to after both passes resolve, so
a lint failure leaves the stats unset (lintCacheHitRatio absent on failures).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread packages/core/src/run-oxlint.ts Outdated
Addresses a Cursor Bugbot finding: the cached path concatenated replayed +
cacheable + sidecar diagnostics without the final dedupe that the non-cached
path applies in spawnLintBatches. A duplicate path in includePaths replays a
file's cached set more than once, so findings/score could exceed a cache-off
scan. Run dedupeDiagnostics on the merged result so warm output equals a
cache-off scan of the same inputs. Adds a duplicate-path regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0b0c6ee. Configure here.

Comment thread packages/core/src/run-oxlint.ts
rayhanadev and others added 2 commits June 19, 2026 03:47
Addresses a Cursor Bugbot finding: the ruleset hash covered the cacheable
config, toolchain versions, and ignore patterns, but not the tsconfig oxlint
parses with (--tsconfig). A tsconfig-only edit left source content hashes
unchanged, so a warm scan could replay diagnostics oxlint produced under the
old compiler options. Read the resolved tsconfig content and fold it into the
ruleset hash, so a tsconfig change busts the cache. Adds a hash-busting test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-file cache key is `<relativePath> <contentHash>` (space-separated,
built in run-oxlint.ts); the doc comment said `\0`. Comment-only — the key
is opaque everywhere, so no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rayhanadev added a commit that referenced this pull request Jun 20, 2026
…in (#903)

Integrates the 7 large-repo speedup PRs (#894#900) plus the dead-code reworks
done on top of them.

Headline (measured on Sentry, supply-chain on): full scan ~142s → ~40s (3.6x),
with dead-code findings fully intact.

- deslop: skip the analysis passes react-doctor discards — the semantic
  TS-Program pass and the code-quality detectors (duplicate-block/complexity/
  feature-flag/TS-smell/private-type-leak/re-export-cycle), gated behind a new
  `reportCodeQuality` flag (default on for deslop standalone). ~8.5x faster
  dead-code on a large repo, byte-identical consumed findings, locked by a
  parity test.
- Dead-code runs sequentially by default with a file-count-scaled timeout,
  fixing a regression where overlapping it with lint oversubscribed the cores
  and silently dropped all dead-code findings under supply-chain. Opt-in
  `REACT_DOCTOR_DEAD_CODE_OVERLAP=on` now splits the core budget instead of
  doubling it.
- Plus the merged PRs: memory-budgeted worker cap (#896), bounded-phase hard
  timeouts (#899), supply-chain∥lint overlap (#894), per-file lint cache (#900),
  deterministic diagnostics (#897), module-cache dedup (#895).

CI: full test matrix (macOS/Ubuntu 20–26/Windows), build, typecheck, lint, and
react-doctor smoke all green. The 1s "CodeQL" check is the pre-existing
code-scanning-alert gate (main's CodeQL analysis is green); no security-relevant
code changed.
@rayhanadev

Copy link
Copy Markdown
Member Author

Superseded by #903, which integrated this work and squash-merged to main (commit 627f9ca). The changes shipped there — some reworked during integration (e.g. the dead-code overlap became sequential-by-default, and cost-ordering was defaulted off); see the #903 description and the changesets on main. Closing in favor of main.

@rayhanadev rayhanadev closed this Jun 20, 2026
rayhanadev added a commit that referenced this pull request Jun 20, 2026
…s, local diff scope (plans 09–11) (#908)

* feat(action,core): CI speedups — install cache, persistent scan caches, local diff scope (plans 09–11)

The GitHub Action's dominant cost on a PR run is the uncached install (~15s of
an ~18s step), not the scan. These three plans target the CI experience:

Plan 09 — cache the install (biggest CI win). A resolve-version step pins the
concrete published version (so the cache key is stable even for `latest`;
scripts/resolve-package-spec.mjs), an actions/cache step restores the install
keyed on version+node+os+arch (no fuzzy fallback — native ABI safety), and the
scan installs into the cached `--prefix` only on a miss. A non-cacheable
local-path spec keeps the npx path. ~15s install → ~1-2s restore on a hit.

Plan 11 — derive PR changed files locally + lock the diff fast path. The base
step now runs `git diff --name-only --diff-filter=AMR <base>...HEAD` (faster, no
API rate limit, works on forks), falling back to the GitHub API only when the
base isn't reachable; both share scripts/normalize-changed-files.mjs. A
regression test locks that diff mode skips dead-code + supply-chain (the
fast-path guarantee). A clearer degraded-mode warning points at `fetch-depth: 0`.

Plan 10 — persist scan caches across CI runs. `REACT_DOCTOR_CACHE_DIR` lets the
action point the engine's caches at a stable `${runner.temp}` path an
actions/cache step persists, so the per-file content-addressed lint cache (#900)
restores across commits — a PR re-lints only its changed files. A new
supply-chain per-PURL on-disk cache (24h TTL, fail-open, NO_CACHE-bypassed) skips
the Socket network for unchanged deps.

Unit/integration tests cover all three (version classification, changed-file
normalization, diff-fast-path skip, cache-dir override, supply-chain cache
hit/bypass); changeset for the npm-facing surface (REACT_DOCTOR_CACHE_DIR +
supply-chain cache). Action releases (tags) + the self-test cacheable-install
job (a workflow-file change needing `workflow` push scope) + R3 sparse-checkout
docs + plan-09's `--print-cache-key` follow-up are noted for a workflow-scoped
push / dogfooding pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(action): self-test the cacheable install path (plan 09)

Adds an `action-cacheable-install` job that runs the action with a published
`version:` (latest) so CI exercises the resolve-version + actions/cache +
prefix-install branch (the local-path job covers the npx branch). Advisory
(`blocking: none`). Split out from the main plan-09 commit because pushing a
`.github/workflows/` change requires `workflow` token scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): migrate mutable @main action refs to @v2 (action change → migration)

Since plans 09-11 change the action, register a once-per-repo project migration
(the framework's "action updates" path) that pins a mutable `@main` / `@master`
React Doctor action reference in `.github/workflows/*.yml` to the recommended
floating major `@v2`.

An unpinned `@main` runs whatever the action's HEAD points to with the
workflow's write permissions — a supply-chain risk (#299) — and the rewrite also
moves the workflow onto the install- and scan-cached release. Only mutable refs
are rewritten; pinned tags / SHAs are deliberate and untouched, a different
action on `@main` is ignored, and only the ref changes (owner, comments, and the
`version:` input are preserved). Runs once per repo like the legacy-config
migration and logs the change for review/commit (or revert if intentionally
tracking main). No-op (stays pending) when there's no mutable ref.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(action): update the action contract test for the plans 09-11 restructure

github-action.test.ts asserts the literal content of action.yml's steps; plans
09/11 moved that content into shared scripts, so two assertions broke:
- the inline `directoryPrefix` prefix-stripping is now in the shared
  normalize-changed-files.mjs (used by both the local-diff base step and the API
  fallback) — assert the wiring instead, with the behavior locked by
  normalize-changed-files.test.ts.
- the inline `PACKAGE_SPEC="react-doctor@$INPUT_VERSION"` derivation moved to the
  resolve-version step (resolve-package-spec.mjs) — assert it's read from that
  step's output.
Adds a test for the new contract (resolve-version + the toolchain/scan
actions/cache steps + the cached prefix-install). Behavior unchanged; the
contract test now matches the reworked action.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants