Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(typescript-estree): add flag EXPERIMENTAL_useSourceOfProjectReferenceRedirect #2669

Merged
merged 3 commits into from Oct 15, 2020

Conversation

@bradzacher
Copy link
Member

@bradzacher bradzacher commented Oct 12, 2020

See #2094

This is quick-and-dirty to get this out there for some users to see what sort of perf improvements this squeezes out.
Intentionally not documenting the flag as it isn't ready to be used by the wider world

With the flag turned on
bradzacher@bradzacher-mbp typescript-eslint % DEBUG=typescript-eslint:* yarn eslint ./packages/types/src/index.ts ./packages/eslint-plugin/src/index.ts
yarn run v1.22.4
$ /Users/bradzacher/github/typescript-eslint/node_modules/.bin/eslint ./packages/types/src/index.ts ./packages/eslint-plugin/src/index.ts
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. /users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/tsconfig.eslint.json. +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/tests/integration/utils/jsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/tsconfig.json. +844ms
  typescript-eslint:typescript-estree:createWatchProgram Found program for file. /users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +2s
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +0ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +5s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +74ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +70ms
✨  Done in 6.50s.
With the flag turned off
bradzacher@bradzacher-mbp typescript-eslint % DEBUG=typescript-eslint:* yarn eslint ./packages/types/src/index.ts ./packages/eslint-plugin/src/index.ts
yarn run v1.22.4
$ /Users/bradzacher/github/typescript-eslint/node_modules/.bin/eslint ./packages/types/src/index.ts ./packages/eslint-plugin/src/index.ts
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. /users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/tsconfig.eslint.json. +1ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/tests/integration/utils/jsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/tsconfig.json. +976ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/eslint-plugin-internal/tsconfig.json. +2s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/eslint-plugin-tslint/tsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/experimental-utils/tsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/parser/tsconfig.json. +961ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/scope-manager/tsconfig.json. +888ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/shared-fixtures/tsconfig.json. +996ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/types/tsconfig.json. +561ms
  typescript-eslint:typescript-estree:createWatchProgram Found program for file. /users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +567ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +0ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +11s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +11s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +64ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +60ms
✨  Done in 12.26s.

Comparisons:

With the flag turned on, runtime is ~6s.
With the flag turned off, runtime is ~12s.

With the flag turned on, we're able to find ./packages/types/src/index.ts within the eslint-plugin program and exit early.
With the flag turned off, we have to iterate and calculate almost all of the programs to find the program for typescript-estree (because our config is alphabetically sorted).

Important note that I just found out - projects are not carried through tsconfig extends!
To test this I had to duplicate the project references config between from the build tsconfigs into the dev tsconfigs.
Kind of annoying but w/e 🤷‍♂️

…renceRedirect

See #2094
This is quick-and-dirty to get this out there for some users to see what sort of perf improvements this squeezes out.

With the flag turned on:
```
bradzacher@bradzacher-mbp typescript-eslint % DEBUG=typescript-eslint:* yarn eslint ./packages/types/src/index.ts ./packages/eslint-plugin/src/index.ts
yarn run v1.22.4
$ /Users/bradzacher/github/typescript-eslint/node_modules/.bin/eslint ./packages/types/src/index.ts ./packages/eslint-plugin/src/index.ts
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. /users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/tsconfig.eslint.json. +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/tests/integration/utils/jsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/tsconfig.json. +844ms
  typescript-eslint:typescript-estree:createWatchProgram Found program for file. /users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +2s
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +0ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +5s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +74ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +70ms
  Done in 6.50s.
```

With the flag turned off:
```
bradzacher@bradzacher-mbp typescript-eslint % DEBUG=typescript-eslint:* yarn eslint ./packages/types/src/index.ts ./packages/eslint-plugin/src/index.ts
yarn run v1.22.4
$ /Users/bradzacher/github/typescript-eslint/node_modules/.bin/eslint ./packages/types/src/index.ts ./packages/eslint-plugin/src/index.ts
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. /users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/tsconfig.eslint.json. +1ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/tests/integration/utils/jsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/tsconfig.json. +976ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/eslint-plugin-internal/tsconfig.json. +2s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/eslint-plugin-tslint/tsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/experimental-utils/tsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/parser/tsconfig.json. +961ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/scope-manager/tsconfig.json. +888ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/shared-fixtures/tsconfig.json. +996ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/bradzacher/github/typescript-eslint/packages/types/tsconfig.json. +561ms
  typescript-eslint:typescript-estree:createWatchProgram Found program for file. /users/bradzacher/github/typescript-eslint/packages/types/src/index.ts +567ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +0ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +11s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +11s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +64ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +60ms
  Done in 12.26s.
```
@typescript-eslint
Copy link

@typescript-eslint typescript-eslint bot commented Oct 12, 2020

Thanks for the PR, @bradzacher!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!


🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint. As a thank you, your profile/company logo will be added to our main README which receives thousands of unique visitors per day.

@codecov
Copy link

@codecov codecov bot commented Oct 12, 2020

Codecov Report

Merging #2669 into master will decrease coverage by 0.04%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##           master    #2669      +/-   ##
==========================================
- Coverage   92.83%   92.78%   -0.05%     
==========================================
  Files         294      294              
  Lines        9666     9676      +10     
  Branches     2711     2713       +2     
==========================================
+ Hits         8973     8978       +5     
- Misses        327      330       +3     
- Partials      366      368       +2     
Flag Coverage Δ
#unittest 92.78% <80.00%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...pt-estree/src/create-program/createWatchProgram.ts 77.15% <80.00%> (-1.46%) ⬇️
@bradzacher bradzacher marked this pull request as ready for review Oct 13, 2020
@bradzacher bradzacher merged commit 90a5878 into master Oct 15, 2020
8 checks passed
8 checks passed
Typecheck
Details
Unit tests Unit tests
Details
Code style and lint
Details
Run integration tests on primary Node.js version
Details
Run unit tests on other Node.js versions (10.x)
Details
Run unit tests on other Node.js versions (14.x)
Details
Publish the latest code as a canary version
Details
Semantic Pull Request ready to be squashed
Details
@bradzacher bradzacher deleted the EXPERIMENTAL_useSourceOfProjectReferenceRedirect branch Oct 15, 2020
@bradzacher bradzacher linked an issue that may be closed by this pull request Oct 16, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 15, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

1 participant
You can’t perform that action at this time.