Prioritize ignored paths when linting

[clarkf]

As discussed in #1124

A common workflow is running oxlint automatically via something like lint-staged, which passes all changed file paths explicitly as CLI arguments. The ignore crate auto-whitelists explicit paths, overriding anything defined in an ignore file, leading to files that should be ignored causing CI and VC hooks to fail.

eslint avoids this issue by giving precedence to .eslintignore unless you pass --no-ignore.

Since ignore doesn't really give much control over this, the most effective solution seems to be filtering. Once we've got a list of files to be linted, we can compare each against the .eslintignore and see if it really truly should be linted.

Unfortunately, this is sort of a naive solution. Ignored directories will still be walked, then filtered out after the fact. Changing this behavior at the walker level would affect the formatter, which probably isn't desirable. (edit: The performance impact has been mostly resolved.)

[codspeed-hq]

CodSpeed Performance Report

Merging #1878 will not alter performance

_{Comparing clarkf:lint-no-ignore (3655c92) with main (45a7985)}

Summary

✅ 14 untouched benchmarks

[Boshen]

This is going to cause performance regression I think, given that no_ignore = false by default and thousands of paths collected from Walk.

Should we only trigger this code path when there are file paths passed from the command line?

For a more granular control, should we introduce a --lint-staged flag to explicitly handle all the cases for lint staged?

[Boshen]

I like the overall direction of this PR, it's just that eslint + lint-staged caused so much configuration confusion and wasted time for people, I have a feeling first-class support (via a cli flag) will be better than some random combinations of configuration options.

What do you think?

[clarkf]

You're right that this introduces a performance regression, it definitely does. In most cases, the paths will be iterated a second time, filtered, and re-Veced. I think the pathological case might be overstated, though. For the overhead to be truly significant, the user would have to have passed a parent path to a significant number of ignored paths. Think something like oxlint node_modules/. This could be avoided by pushing the ignore filtering up into Walk, probably.

Adding the check for explicit paths is an easy win, though. I'll definitely do that, which should avoid the overhead for the common case.

...eslint + lint-staged caused so much configuration confusion and wasted time for people, I have a feeling first-class support (via a cli flag) will be better than some random combinations of configuration options.

The confusion is definitely real, and ultimately I'd really defer to your judgement. My preference would be to stick with this direction and not implement git-specific logic. For what it's worth, this behavior (.{tool}ignore file taking precedence over provided paths) is how most of the common tools in this corner of the world (git, eslint, prettier) work.

[Boshen]

Shall we try the following logic:

for each provided path, 
  if the path is a file (not a directory)
    check against ignore logic

---

FYI oxlint is designed to work on huge monorepos, with 10,000+ files being the norm.

[clarkf]

I've approximated the requested logic, I think. Notably, it is now possible to pre-filter all paths, skipping linting altogether.

There's an outstanding question of user feedback. At no point during this process is the user informed that paths or filtered. eslint yields linter warnings, which doesn't seem perfect, but better than silence.

[Boshen]

Thank you!

There's an outstanding question of user feedback. At no point during this process is the user informed that paths or filtered. eslint yields linter warnings, which doesn't seem perfect, but better than silence.

We can have another issue / PR for this.