Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

slow linting performance with high CPU usage #1140

Closed
mpgon opened this issue Oct 25, 2019 · 1 comment
Closed

slow linting performance with high CPU usage #1140

mpgon opened this issue Oct 25, 2019 · 1 comment

Comments

@mpgon
Copy link

@mpgon mpgon commented Oct 25, 2019

What were you trying to do?
We recently migrated on of our projects from Flow to Typescript, but running eslint with the typescript parser is very slow (averaging 4 minutes).
Having it running along vscode with the ESLint extension produces very high CPU usages (over 100%) in the Code Helper task. By inspecting the process, I could find that the initiator is the eslint extension, so I believe the root problem is the same.
I also ran with TIMING=1 and disabled the most resource intensive rules (>10%) with no significant result.
I also tried downgrading @typescript-eslint/eslint-plugin and @typescript-eslint/parser from 2.5.0 to 2.3.0 and upgrading to 2.5.1-alpha.3 to no result.
Setting project in parserOptions to false naturally reduces the time to an average of 27 seconds.
This problem makes the vscode ESLint extension virtually unusable.
I found several issues opened in this repository that seem to be linked but they were all closed except for these ones: #1042, #1042

.eslintrc.js

module.exports = {
  env: {
    browser: true,
    commonjs: true,
    es6: true,
    node: true,
  },
  extends: [
    "eslint:recommended",
    "plugin:react/recommended",
    "plugin:@typescript-eslint/recommended",
    "plugin:@typescript-eslint/eslint-recommended",
    "prettier",
    "prettier/@typescript-eslint",
  ],
  globals: {
    Atomics: "readonly",
    SharedArrayBuffer: "readonly",
  },
  parser: "@typescript-eslint/parser",
  parserOptions: {
    createDefaultProgram: false,
    project: "./tsconfig.json",
  },
  settings: {
    react: {
      version: "detect",
    },
  },
  plugins: ["react", "@typescript-eslint"],
  rules: {
    "react/display-name": 0,
    "react/no-deprecated": 0,
    "react/no-direct-mutation-state": 0,
    "react/no-string-refs": 0,
    "react/prop-types": 0,
    "react/require-render-return": 0,
    "@typescript-eslint/no-explicit-any": 0,
    "@typescript-eslint/ban-ts-ignore": 0,
    "@typescript-eslint/explicit-function-return-type": 0,
    "@typescript-eslint/camelcase": 0,
    "no-global-assign": 0,
  },
};

.eslintignore

config/**/*
scripts/**/*
build/**/*
node_modules/**/*
**/*.config.js
.storybook
.jest
src/model/crypto/asmcrypto/**/*
src/translations/scripts/**/*
src/react-forms-state

tsconfig.json

{
  "compilerOptions": {
    "target": "esnext",
    "lib": ["webworker", "dom", "dom.iterable", "esnext"],
    "allowJs": true,
    "checkJs": false,
    "skipLibCheck": true,
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true,
    "strict": true,
    "forceConsistentCasingInFileNames": true,
    "module": "esnext",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "noEmit": true,
    "jsx": "react",
    "baseUrl": "src",
    "paths": {
      "types/*": ["types/*"],
      "model/*": ["model/*"],
      "apollo/*": ["apollo/*"],
      "consts/*": ["consts/*"],
      "dsl/*": ["dsl/*"],
      "translations/*": ["translations/*"],
      "react-forms-state": ["react-forms-state"]
    }
  },
  "include": ["src/**/*"],
  "exclude": [
    "node_modules",
    "build",
    ".jest",
    ".storybook",
    ".vscode",
    ".git"
  ]
}

ts code
proprietary codebase
ts stats:

Files:           2648
Lines:         694016

Impact
Slow check times. High CPU usage. vscode and overall computer slowdown.

Versions

package version
@typescript-eslint/eslint-plugin 2.5.0
@typescript-eslint/parser 2.5.0
TypeScript 3.6.4
node 10.16.0
npm 6.9.0

also tried with

package version
@typescript-eslint/eslint-plugin 2.3.0
@typescript-eslint/parser 2.3.0

and

package version
@typescript-eslint/eslint-plugin 2.5.1-alpha.3
@typescript-eslint/parser 2.5.1-alpha.3
@bradzacher

This comment has been minimized.

Copy link
Member

@bradzacher bradzacher commented Oct 25, 2019

#1042 is opened to track the flakey 100% CPU usage issue.
#1132 is opened to track the perf regression that was introduced in 2.5.0.


There's only so much I can look into here without more information.
Config only paints so much of a picture, and from what I can tell your config looks fine.

Note that in the config you provided, you aren't using any rules that require type information, so you don't need to provide us your project config.

Without more information, I'm going to assume that this is covered by the above issues, so I'm going to close this issue.


running eslint with the typescript parser is very slow (averaging 4 minutes).

This is expected, and is completely unavoidable for two reasons.

The first being the methodology. We need to create an ESTree compatible AST, but TypeScript provides a TypeScript AST. So we have two choices:

  • go babel's route, and create a like-for-like parser which has to reimplement TypeScript's parsing logic.
  • ask TypeScript to parse the file, then convert TypeScript's AST into an ESTree compatible AST.

The former is very hard, and requires a lot of work to build and maintain, as evident by the fact that the babel team didn't want to do it for so long, and now that they have it they don't have 100% feature support.

The latter is easier, but is slightly less efficient, because we have to take some time to copy and convert every single AST node. This solution, however, opens up a new feature for us - because we are asking TypeScript to parse the file, we can also ask it for the type information.

Unfortunately, collecting type information is only possible by having TypeScript do a complete parse + build for each tsconfig you provide.

This build scales with the size of your project. More files = longer build = longer lint.

There are ways we can potentially speed this up (such as consuming incremental build artefacts), but these techniques are complicated, difficult to implement, and we have more pressing bugs and features to work on. As we are a volunteer maintained project, we don't have the bandwidth to tackle them, so this is a long term goal.


We understand that not everybody is willing to accept the lint-time perf hit.
This is why we support a mode that just parses and converts the source code. This is also why provide two separate recommended configs - one which uses type information, and one that doesn't.

@bradzacher bradzacher closed this Oct 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.