@nx/eslint:lint-project create a fresh graph at every run

[beaussan]

Current Behavior

@nx/eslint:lint-project create a fresh graph every time, making custom plugins wrapping other generators (such as @nx/react:library) test creating a lot of graph, running the tests extremely slow

 PASS   my-plugin  my-plugin/src/generator.spec.ts (26.856 s)
  lib generator
    ✓ should generate 5 libs with skipEslint=false (25324 ms)
    ✓ should generate 5 libs with skipEslint=true (167 ms)

Expected Behavior

@nx/eslint:lint-project read the cached graph instead of creating a new graph every time

GitHub Repo

https://github.com/beaussan/repro-nx-plugin-test-daemon

Steps to Reproduce

Here is a test file that trigger the issue:

import { createTreeWithEmptyWorkspace } from '@nx/devkit/testing';
import { Tree, readProjectConfiguration } from '@nx/devkit';

import { libraryGenerator as nxReactLibGenerator } from '@nx/react/src/generators/library/library';
import { Linter } from '@nx/eslint';

const numberOfLibs = 5;

describe('lib generator', () => {
  let tree: Tree;

  beforeEach(() => {
    tree = createTreeWithEmptyWorkspace();
  });

  it(`should generate ${numberOfLibs} libs with skipEslint=false`, async () => {
    for (let i = 0; i < numberOfLibs; i++) {
      await nxReactLibGenerator(tree, {
        bundler: 'vite',
        // waiting on an answer for a better way from the nx team, since their getRelativeCwd is not a public api _yet_.
        compiler: 'babel',
        directory: `my-lib-${i}`,
        linter: Linter.EsLint,
        name: `my-lib-${i}`,
        projectNameAndRootFormat: 'as-provided',
        setParserOptionsProject: true,
        skipPackageJson: true,
        skipTsConfig: true,
        style: 'styled-components',
        unitTestRunner: 'vitest',
      });
      const config = readProjectConfiguration(tree, `my-lib-${i}`);
      expect(config).toBeDefined();
    }
  }, 90000);

  it(`should generate ${numberOfLibs} libs with skipEslint=true`, async () => {
    for (let i = 0; i < numberOfLibs; i++) {
      await nxReactLibGenerator(tree, {
        bundler: 'vite',
        // waiting on an answer for a better way from the nx team, since their getRelativeCwd is not a public api _yet_.
        compiler: 'babel',
        directory: `my-lib-${i}`,
        linter: Linter.None,
        name: `my-lib-${i}`,
        projectNameAndRootFormat: 'as-provided',
        setParserOptionsProject: true,
        skipPackageJson: true,
        skipTsConfig: true,
        style: 'styled-components',
        unitTestRunner: 'vitest',
      });
      const config = readProjectConfiguration(tree, `my-lib-${i}`);
      expect(config).toBeDefined();
    }
  }, 90000);
});

Nx Report

Node           : 20.11.1
OS             : darwin-arm64
Native Target  : aarch64-macos
yarn           : 4.4.1

nx                 : 19.6.5
@nx/js             : 19.6.5
@nx/jest           : 19.6.5
@nx/linter         : 19.6.5
@nx/eslint         : 19.6.5
@nx/workspace      : 19.6.5
@nx/devkit         : 19.6.5
@nx/eslint-plugin  : 19.6.5
@nx/plugin         : 19.6.5
@nx/react          : 19.6.5
@nrwl/tao          : 19.6.5
@nx/vite           : 19.6.5
@nx/web            : 19.6.5
typescript         : 5.5.4

Failure Logs

No response

Package Manager Version

No response

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

No response

[vincent-palita-whoz]

@jaysoo Have you considered implementing a strategy similar to Jest's disableJestRuntime for ESLint (thanks to PR: #28522)? This approach made our graph generation significantly faster and nearly instantaneous when not using the daemon.
In our 700+ projects monorepo, this change was transformative - reducing graph computation from several minutes to just seconds. Currently, ESLint consistently adds a 10s overhead to every Nx task, which becomes somewhat of a bottleneck.

[Sigmund-SM]

We have aslo experienced that @nx/eslint:createNodes has a 10s+ overhead for every project. After some debugging I traced the issue down to this block of code: (packages/eslint/src/plugins/plugin.ts (line 215-224)

const eslint = new ESLint({
  cwd: join(context.workspaceRoot, projectRoot),
});
let hasNonIgnoredLintableFiles = false;
for (const file of lintableFilesPerProjectRoot.get(projectRoot) ?? []) {
  if (!(await eslint.isPathIgnored(join(context.workspaceRoot, file)))) {
    hasNonIgnoredLintableFiles = true;
    break;
  }
}

if (!hasNonIgnoredLintableFiles) {
  // No lintable files in the project, store in the cache and skip further processing
  projectsCache[hash] = {};
  return;
}

The purpose of this block is to detect and don't add lint target to projects where all files are ignored by eslint. I see the point, but it is really not worth the overhead.

Our solution for now is to use a postinstall script to remove this block of code, and rather use the "exclude" option when defining the eslint plugin in nx.json to remove lint targets from projects where we don't want eslint.

For our ~200 project monorepo this reduced the computation time of @nx/eslint:createNodes from 10-20 sec to 1-3 sec

[jaysoo]

Closing this as it should have been addressed in the latest version of Nx.

1. I the original repro is much faster on latest Nx version: 1116 ms vs 25324 ms (https://github.com/jaysoo/repro-nx-plugin-test-daemon)
2. The change here should reduce the overhead of checking ignored files.