Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not mark indexed access object type comparisons as unreliable. #54781

Merged
merged 1 commit into from
Jun 26, 2023

Conversation

DanielRosenwasser
Copy link
Member

@DanielRosenwasser DanielRosenwasser commented Jun 26, 2023

Reverts the checker change in #52106.

Fixes #54542.

See #54754 for a hopefully better long-term fix.

Reopens #52096.

@DanielRosenwasser
Copy link
Member Author

@typescript-bot pack this
@typescript-bot test this
@typescript-bot test top280
@typescript-bot user test this
@typescript-bot user test tsserver
@typescript-bot test tsserver top100
@typescript-bot run dt
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the diff-based user code test suite on this PR at 8d94382. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the parallelized Definitely Typed test suite on this PR at 8d94382. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the extended test suite on this PR at 8d94382. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 8d94382. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the tarball bundle task on this PR at 8d94382. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the diff-based top-repos suite (tsserver) on this PR at 8d94382. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the diff-based top-repos suite on this PR at 8d94382. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the diff-based user code test suite (tsserver) on this PR at 8d94382. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/155637/artifacts?artifactName=tgz&fileId=04DE374095785839F62F13D7F462400B471B8FC647A291E41CD31D23BFD6186002&fileName=/typescript-5.2.0-insiders.20230626.tgz"
    }
}

and then running npm install.


There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.2.0-pr-54781-9".;

@jakebailey
Copy link
Member

@JoshuaKGoldberg Any chance you could throw the above package ("typescript": "npm:@typescript-deploys/pr-build@5.2.0-pr-54781-9") into the ts-eslint CI to see if it passes again?

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser Here are the results of running the user test suite comparing main and refs/pull/54781/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Unknown failure"
  • 1 instance of "Package install failed"

Otherwise...

Something interesting changed - please have a look.

Details

rxjs-src

/mnt/ts_downloads/rxjs-src/build.sh

  • [NEW] error TS2428: All declarations of 'WeakMap' must have identical type parameters.
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-54781/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
  • [MISSING] error TS2428: All declarations of 'WeakMap' must have identical type parameters.
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser Here are the results of running the user test suite comparing main and refs/pull/54781/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Unknown failure"

Otherwise...

Everything looks good!

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..54781
Metric main 54781 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 366,615k (± 0.01%) 366,606k (± 0.01%) ~ 366,580k 366,640k p=0.872 n=6
Parse Time 3.42s (± 0.54%) 3.42s (± 0.50%) ~ 3.40s 3.44s p=0.871 n=6
Bind Time 1.13s (± 0.49%) 1.12s (± 0.00%) ~ 1.12s 1.12s p=0.071 n=6
Check Time 8.88s (± 0.39%) 8.81s (± 0.22%) -0.07s (- 0.73%) 8.79s 8.83s p=0.020 n=6
Emit Time 7.48s (± 0.49%) 7.46s (± 0.14%) ~ 7.44s 7.47s p=0.256 n=6
Total Time 20.90s (± 0.29%) 20.81s (± 0.15%) -0.09s (- 0.45%) 20.76s 20.84s p=0.024 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 191,920k (± 1.21%) 190,968k (± 0.03%) ~ 190,902k 191,060k p=0.423 n=6
Parse Time 1.49s (± 1.61%) 1.50s (± 0.50%) ~ 1.49s 1.51s p=0.209 n=6
Bind Time 0.77s (± 0.97%) 0.77s (± 1.35%) ~ 0.75s 0.78s p=0.437 n=6
Check Time 9.46s (± 0.60%) 9.47s (± 0.69%) ~ 9.38s 9.57s p=0.810 n=6
Emit Time 2.76s (± 1.01%) 2.76s (± 0.94%) ~ 2.73s 2.80s p=1.000 n=6
Total Time 14.47s (± 0.49%) 14.49s (± 0.50%) ~ 14.39s 14.57s p=0.688 n=6
Monaco - node (v18.10.0, x64)
Memory used 346,800k (± 0.01%) 346,795k (± 0.01%) ~ 346,750k 346,858k p=0.809 n=6
Parse Time 2.58s (± 0.84%) 2.59s (± 0.88%) ~ 2.56s 2.62s p=0.331 n=6
Bind Time 1.00s (± 0.54%) 1.01s (± 0.51%) ~ 1.00s 1.01s p=0.640 n=6
Check Time 7.18s (± 0.64%) 7.19s (± 0.37%) ~ 7.15s 7.22s p=0.936 n=6
Emit Time 4.27s (± 0.53%) 4.24s (± 0.79%) ~ 4.19s 4.28s p=0.169 n=6
Total Time 15.03s (± 0.34%) 15.03s (± 0.42%) ~ 14.96s 15.12s p=0.936 n=6
TFS - node (v18.10.0, x64)
Memory used 300,884k (± 0.01%) 300,879k (± 0.01%) ~ 300,839k 300,896k p=0.873 n=6
Parse Time 2.07s (± 0.85%) 2.06s (± 0.67%) ~ 2.04s 2.08s p=0.222 n=6
Bind Time 1.14s (± 0.45%) 1.14s (± 0.92%) ~ 1.12s 1.15s p=0.142 n=6
Check Time 6.66s (± 0.64%) 6.64s (± 0.13%) ~ 6.63s 6.65s p=0.374 n=6
Emit Time 3.87s (± 0.71%) 3.85s (± 0.35%) ~ 3.83s 3.87s p=0.370 n=6
Total Time 13.73s (± 0.40%) 13.69s (± 0.19%) ~ 13.65s 13.72s p=0.124 n=6
material-ui - node (v18.10.0, x64)
Memory used 482,317k (± 0.01%) 482,334k (± 0.01%) ~ 482,248k 482,408k p=0.575 n=6
Parse Time 3.12s (± 0.66%) 3.12s (± 0.39%) ~ 3.10s 3.13s p=0.743 n=6
Bind Time 0.92s (± 1.44%) 0.92s (± 1.13%) ~ 0.91s 0.93s p=0.523 n=6
Check Time 16.94s (± 0.57%) 16.98s (± 0.51%) ~ 16.89s 17.12s p=0.422 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.99s (± 0.47%) 21.01s (± 0.44%) ~ 20.92s 21.16s p=0.810 n=6
xstate - node (v18.10.0, x64)
Memory used 564,257k (± 0.04%) 563,586k (± 0.03%) -671k (- 0.12%) 563,413k 563,774k p=0.005 n=6
Parse Time 3.82s (± 0.63%) 3.83s (± 0.51%) ~ 3.80s 3.86s p=0.193 n=6
Bind Time 1.64s (± 0.32%) 1.63s (± 0.33%) ~ 1.63s 1.64s p=0.798 n=6
Check Time 2.79s (± 0.37%) 2.80s (± 0.54%) ~ 2.79s 2.83s p=0.210 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 8.33s (± 0.28%) 8.36s (± 0.27%) ~ 8.33s 8.40s p=0.075 n=6
Angular - node (v16.17.1, x64)
Memory used 366,009k (± 0.01%) 366,016k (± 0.01%) ~ 365,996k 366,049k p=0.630 n=6
Parse Time 3.57s (± 0.52%) 3.58s (± 0.60%) ~ 3.54s 3.60s p=0.739 n=6
Bind Time 1.18s (± 0.54%) 1.18s (± 0.35%) ~ 1.18s 1.19s p=0.673 n=6
Check Time 9.69s (± 0.55%) 9.63s (± 0.28%) ~ 9.59s 9.66s p=0.064 n=6
Emit Time 8.04s (± 0.68%) 7.98s (± 0.30%) ~ 7.95s 8.02s p=0.064 n=6
Total Time 22.48s (± 0.36%) 22.36s (± 0.28%) -0.11s (- 0.50%) 22.28s 22.44s p=0.030 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 192,833k (± 0.02%) 192,817k (± 0.04%) ~ 192,699k 192,902k p=0.936 n=6
Parse Time 1.58s (± 1.48%) 1.58s (± 1.72%) ~ 1.55s 1.61s p=0.935 n=6
Bind Time 0.83s (± 0.62%) 0.82s (± 0.92%) ~ 0.81s 0.83s p=0.247 n=6
Check Time 10.16s (± 1.00%) 10.12s (± 0.60%) ~ 10.07s 10.23s p=0.688 n=6
Emit Time 3.03s (± 1.09%) 3.04s (± 1.19%) ~ 3.00s 3.09s p=1.000 n=6
Total Time 15.60s (± 0.58%) 15.55s (± 0.40%) ~ 15.49s 15.63s p=0.520 n=6
Monaco - node (v16.17.1, x64)
Memory used 346,057k (± 0.00%) 346,065k (± 0.01%) ~ 346,035k 346,089k p=0.521 n=6
Parse Time 2.74s (± 0.83%) 2.73s (± 0.43%) ~ 2.71s 2.74s p=0.462 n=6
Bind Time 1.09s (± 0.50%) 1.09s (± 0.47%) -0.01s (- 0.76%) 1.08s 1.09s p=0.038 n=6
Check Time 7.90s (± 0.60%) 7.87s (± 0.46%) ~ 7.82s 7.91s p=0.469 n=6
Emit Time 4.49s (± 0.43%) 4.46s (± 0.31%) -0.02s (- 0.56%) 4.45s 4.48s p=0.043 n=6
Total Time 16.22s (± 0.50%) 16.15s (± 0.31%) ~ 16.09s 16.20s p=0.199 n=6
TFS - node (v16.17.1, x64)
Memory used 300,216k (± 0.01%) 300,204k (± 0.01%) ~ 300,183k 300,232k p=0.297 n=6
Parse Time 2.17s (± 0.50%) 2.17s (± 0.68%) ~ 2.15s 2.19s p=0.805 n=6
Bind Time 1.25s (± 0.33%) 1.24s (± 1.10%) ~ 1.22s 1.26s p=0.078 n=6
Check Time 7.34s (± 0.37%) 7.32s (± 0.51%) ~ 7.29s 7.39s p=0.517 n=6
Emit Time 4.34s (± 0.68%) 4.35s (± 1.05%) ~ 4.31s 4.44s p=1.000 n=6
Total Time 15.10s (± 0.31%) 15.08s (± 0.56%) ~ 15.02s 15.24s p=0.259 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,616k (± 0.01%) 481,634k (± 0.01%) ~ 481,550k 481,665k p=0.470 n=6
Parse Time 3.26s (± 0.66%) 3.26s (± 0.25%) ~ 3.25s 3.27s p=0.934 n=6
Bind Time 0.95s (± 0.79%) 0.95s (± 0.43%) ~ 0.95s 0.96s p=1.000 n=6
Check Time 17.97s (± 0.36%) 17.98s (± 0.73%) ~ 17.79s 18.14s p=1.000 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.19s (± 0.31%) 22.19s (± 0.59%) ~ 21.99s 22.35s p=1.000 n=6
xstate - node (v16.17.1, x64)
Memory used 561,785k (± 0.02%) 561,107k (± 0.01%) -678k (- 0.12%) 561,051k 561,258k p=0.005 n=6
Parse Time 4.00s (± 0.26%) 4.01s (± 0.29%) ~ 3.99s 4.02s p=0.456 n=6
Bind Time 1.73s (± 0.43%) 1.75s (± 0.48%) +0.01s (+ 0.77%) 1.74s 1.76s p=0.020 n=6
Check Time 3.06s (± 0.45%) 3.06s (± 0.43%) ~ 3.04s 3.07s p=0.808 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 5.53%) ~ 0.09s 0.10s p=0.174 n=6
Total Time 8.89s (± 0.33%) 8.90s (± 0.18%) ~ 8.89s 8.93s p=0.467 n=6
Angular - node (v14.21.3, x64)
Memory used 360,063k (± 0.02%) 360,044k (± 0.01%) ~ 359,991k 360,103k p=0.575 n=6
Parse Time 3.68s (± 0.56%) 3.68s (± 0.72%) ~ 3.65s 3.72s p=1.000 n=6
Bind Time 1.22s (± 0.62%) 1.22s (± 0.68%) ~ 1.22s 1.24s p=0.209 n=6
Check Time 10.09s (± 0.44%) 10.12s (± 0.53%) ~ 10.04s 10.18s p=0.294 n=6
Emit Time 8.40s (± 1.02%) 8.39s (± 0.97%) ~ 8.29s 8.51s p=0.936 n=6
Total Time 23.38s (± 0.49%) 23.41s (± 0.57%) ~ 23.23s 23.58s p=0.689 n=6
Compiler-Unions - node (v14.21.3, x64)
Memory used 188,154k (± 0.01%) 188,135k (± 0.01%) ~ 188,097k 188,163k p=0.173 n=6
Parse Time 1.63s (± 0.84%) 1.62s (± 0.39%) ~ 1.61s 1.63s p=0.388 n=6
Bind Time 0.85s (± 0.89%) 0.85s (± 0.65%) ~ 0.84s 0.85s p=0.476 n=6
Check Time 10.29s (± 0.15%) 10.28s (± 0.45%) ~ 10.22s 10.34s p=0.565 n=6
Emit Time 3.14s (± 0.42%) 3.15s (± 1.44%) ~ 3.10s 3.23s p=1.000 n=6
Total Time 15.90s (± 0.14%) 15.89s (± 0.41%) ~ 15.80s 15.96s p=1.000 n=6
Monaco - node (v14.21.3, x64)
Memory used 341,200k (± 0.00%) 341,176k (± 0.00%) -24k (- 0.01%) 341,162k 341,203k p=0.030 n=6
Parse Time 2.80s (± 0.61%) 2.81s (± 0.52%) ~ 2.80s 2.84s p=0.245 n=6
Bind Time 1.12s (± 0.75%) 1.12s (± 1.04%) ~ 1.11s 1.14s p=0.652 n=6
Check Time 8.19s (± 0.22%) 8.16s (± 0.36%) ~ 8.11s 8.20s p=0.052 n=6
Emit Time 4.69s (± 1.09%) 4.70s (± 0.91%) ~ 4.63s 4.75s p=0.936 n=6
Total Time 16.80s (± 0.46%) 16.79s (± 0.44%) ~ 16.70s 16.88s p=0.808 n=6
TFS - node (v14.21.3, x64)
Memory used 295,305k (± 0.00%) 295,307k (± 0.00%) ~ 295,302k 295,314k p=0.809 n=6
Parse Time 2.40s (± 0.61%) 2.40s (± 0.57%) ~ 2.38s 2.41s p=0.734 n=6
Bind Time 1.07s (± 0.59%) 1.07s (± 0.38%) ~ 1.06s 1.07s p=0.673 n=6
Check Time 7.64s (± 0.23%) 7.66s (± 0.95%) ~ 7.57s 7.78s p=0.628 n=6
Emit Time 4.30s (± 1.19%) 4.33s (± 0.74%) ~ 4.30s 4.38s p=0.199 n=6
Total Time 15.41s (± 0.42%) 15.46s (± 0.62%) ~ 15.35s 15.61s p=0.378 n=6
material-ui - node (v14.21.3, x64)
Memory used 477,091k (± 0.01%) 477,112k (± 0.00%) ~ 477,091k 477,130k p=0.230 n=6
Parse Time 3.33s (± 0.30%) 3.33s (± 0.41%) ~ 3.32s 3.36s p=0.591 n=6
Bind Time 1.01s (± 0.51%) 1.01s (± 0.51%) ~ 1.00s 1.01s p=1.000 n=6
Check Time 18.89s (± 0.47%) 18.89s (± 0.42%) ~ 18.78s 19.00s p=0.936 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.23s (± 0.39%) 23.24s (± 0.35%) ~ 23.12s 23.36s p=0.810 n=6
xstate - node (v14.21.3, x64)
Memory used 550,707k (± 0.00%) 550,204k (± 0.00%) -503k (- 0.09%) 550,191k 550,219k p=0.005 n=6
Parse Time 4.24s (± 0.46%) 4.23s (± 0.24%) ~ 4.21s 4.24s p=0.211 n=6
Bind Time 1.67s (± 1.05%) 1.66s (± 1.23%) ~ 1.64s 1.68s p=0.415 n=6
Check Time 3.15s (± 0.56%) 3.14s (± 0.56%) ~ 3.13s 3.17s p=1.000 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 9.15s (± 0.48%) 9.13s (± 0.43%) ~ 9.08s 9.19s p=0.470 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.21.3, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.21.3, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.21.3, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.21.3, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.21.3, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.21.3, x64)
Benchmark Name Iterations
Current 54781 6
Baseline main 6

TSServer

Comparison Report - main..54781
Metric main 54781 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,559ms (± 0.54%) 2,558ms (± 0.37%) ~ 2,542ms 2,569ms p=0.810 n=6
Req 2 - geterr 5,346ms (± 0.72%) 5,344ms (± 0.59%) ~ 5,305ms 5,386ms p=1.000 n=6
Req 3 - references 340ms (± 0.55%) 340ms (± 0.78%) ~ 337ms 343ms p=1.000 n=6
Req 4 - navto 288ms (± 0.42%) 287ms (± 0.60%) ~ 284ms 289ms p=0.804 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 84ms (± 2.99%) 83ms (± 7.07%) ~ 72ms 89ms p=1.000 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,638ms (± 1.21%) 2,650ms (± 1.01%) ~ 2,612ms 2,677ms p=0.470 n=6
Req 2 - geterr 4,112ms (± 0.42%) 4,099ms (± 0.47%) ~ 4,073ms 4,129ms p=0.261 n=6
Req 3 - references 349ms (± 0.71%) 347ms (± 0.54%) ~ 345ms 350ms p=0.222 n=6
Req 4 - navto 290ms (± 1.28%) 287ms (± 0.48%) ~ 285ms 288ms p=0.106 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 64ms (± 3.34%) 64ms (± 3.55%) ~ 62ms 68ms p=0.546 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,116ms (± 0.50%) 3,099ms (± 0.41%) ~ 3,079ms 3,111ms p=0.093 n=6
Req 2 - geterr 1,545ms (± 0.60%) 1,593ms (± 0.38%) +48ms (+ 3.13%) 1,585ms 1,602ms p=0.005 n=6
Req 3 - references 115ms (± 1.42%) 113ms (± 1.25%) ~ 111ms 115ms p=0.120 n=6
Req 4 - navto 369ms (± 0.27%) 368ms (± 0.22%) ~ 367ms 369ms p=0.498 n=6
Req 5 - completionInfo count 2,870 (± 0.00%) 2,870 (± 0.00%) ~ 2,870 2,870 p=1.000 n=6
Req 5 - completionInfo 375ms (± 1.86%) 377ms (± 1.64%) ~ 366ms 382ms p=0.935 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,646ms (± 0.69%) 2,640ms (± 0.91%) ~ 2,614ms 2,672ms p=0.689 n=6
Req 2 - geterr 6,040ms (± 0.81%) 6,020ms (± 0.44%) ~ 5,997ms 6,068ms p=0.471 n=6
Req 3 - references 355ms (± 0.52%) 353ms (± 0.54%) ~ 351ms 356ms p=0.190 n=6
Req 4 - navto 287ms (± 1.61%) 288ms (± 2.11%) ~ 281ms 295ms p=1.000 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 92ms (± 0.88%) 94ms (± 1.61%) ~ 92ms 96ms p=0.116 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,848ms (± 0.52%) 2,836ms (± 0.42%) ~ 2,821ms 2,856ms p=0.173 n=6
Req 2 - geterr 4,674ms (± 0.33%) 4,675ms (± 0.30%) ~ 4,660ms 4,696ms p=0.872 n=6
Req 3 - references 363ms (± 0.95%) 364ms (± 0.60%) ~ 362ms 368ms p=0.676 n=6
Req 4 - navto 284ms (± 0.86%) 281ms (± 1.10%) ~ 277ms 284ms p=0.142 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 67ms (± 1.12%) 67ms (± 0.77%) ~ 67ms 68ms p=0.784 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,229ms (± 0.60%) 3,224ms (± 0.38%) ~ 3,206ms 3,242ms p=0.630 n=6
Req 2 - geterr 1,714ms (± 0.37%) 1,750ms (± 0.40%) +36ms (+ 2.08%) 1,741ms 1,761ms p=0.005 n=6
Req 3 - references 124ms (± 1.19%) 125ms (± 7.11%) ~ 119ms 143ms p=0.361 n=6
Req 4 - navto 351ms (± 0.39%) 354ms (± 0.52%) +3ms (+ 0.95%) 352ms 357ms p=0.007 n=6
Req 5 - completionInfo count 2,870 (± 0.00%) 2,870 (± 0.00%) ~ 2,870 2,870 p=1.000 n=6
Req 5 - completionInfo 416ms (± 1.06%) 403ms (± 1.23%) 🟩-13ms (- 3.05%) 393ms 406ms p=0.005 n=6
Compiler-UnionsTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,782ms (± 0.68%) 2,788ms (± 0.70%) ~ 2,769ms 2,816ms p=0.810 n=6
Req 2 - geterr 6,179ms (± 0.46%) 6,164ms (± 0.41%) ~ 6,128ms 6,194ms p=0.378 n=6
Req 3 - references 362ms (± 1.16%) 363ms (± 1.38%) ~ 358ms 372ms p=0.872 n=6
Req 4 - navto 292ms (± 0.71%) 292ms (± 0.42%) ~ 290ms 293ms p=0.676 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 102ms (± 3.67%) 102ms (± 3.80%) ~ 94ms 104ms p=0.546 n=6
CompilerTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,988ms (± 0.54%) 2,979ms (± 0.62%) ~ 2,955ms 3,002ms p=0.471 n=6
Req 2 - geterr 4,563ms (± 0.70%) 4,522ms (± 0.39%) -41ms (- 0.90%) 4,495ms 4,547ms p=0.045 n=6
Req 3 - references 375ms (± 0.55%) 378ms (± 0.60%) ~ 374ms 381ms p=0.059 n=6
Req 4 - navto 299ms (± 0.39%) 298ms (± 0.42%) ~ 296ms 299ms p=0.278 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 76ms (± 1.53%) 76ms (± 1.07%) ~ 75ms 77ms p=1.000 n=6
xstateTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 3,464ms (± 0.84%) 3,493ms (± 0.60%) ~ 3,460ms 3,517ms p=0.148 n=6
Req 2 - geterr 1,836ms (± 0.55%) 1,845ms (± 0.67%) ~ 1,825ms 1,861ms p=0.228 n=6
Req 3 - references 147ms (± 5.98%) 148ms (± 7.91%) ~ 137ms 160ms p=0.935 n=6
Req 4 - navto 397ms (± 0.37%) 395ms (± 1.65%) ~ 386ms 401ms p=0.935 n=6
Req 5 - completionInfo count 2,870 (± 0.00%) 2,870 (± 0.00%) ~ 2,870 2,870 p=1.000 n=6
Req 5 - completionInfo 430ms (± 1.88%) 439ms (± 1.11%) ~ 432ms 446ms p=0.077 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.21.3, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.21.3, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.21.3, x64)
Benchmark Name Iterations
Current 54781 6
Baseline main 6

Startup

Comparison Report - main..54781
Metric main 54781 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 142.87ms (± 0.22%) 142.77ms (± 0.19%) -0.10ms (- 0.07%) 142.05ms 145.58ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 223.56ms (± 0.50%) 222.26ms (± 0.18%) -1.31ms (- 0.58%) 221.44ms 229.80ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 225.74ms (± 0.41%) 223.68ms (± 0.16%) -2.06ms (- 0.91%) 222.80ms 228.32ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 207.20ms (± 0.37%) 205.73ms (± 0.17%) -1.47ms (- 0.71%) 204.70ms 208.78ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 54781 6
Baseline main 6

Developer Information:

Download Benchmark

@DanielRosenwasser
Copy link
Member Author

@typescript-bot cherry-pick this to release-5.1

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 26, 2023

Heya @DanielRosenwasser, I've started to run the task to cherry-pick this into release-5.1 on this PR at 8d94382. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

Hey @DanielRosenwasser, I've opened #54786 for you.

typescript-bot pushed a commit to typescript-bot/TypeScript that referenced this pull request Jun 26, 2023
Component commits:
8d94382 Do not mark indexed access object type comparisons as unreliable.
@typescript-bot
Copy link
Collaborator

Hey @DanielRosenwasser, the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser Here are the results of running the top-repos suite comparing main and refs/pull/54781/merge:

Everything looks good!

Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely will need to revert this revert eventually, but we absolutely need to make some things more efficient first to compensate.

Can we add a non-strict @types/lodash usage to the extended test suite (as that's what this fixes), since the DT run won't cover it? It's just a new folder in https://github.com/microsoft/typescript-error-deltas/tree/main/userTests right?

@DanielRosenwasser
Copy link
Member Author

Can we add a non-strict @types/lodash usage to the extended test suite (as that's what this fixes), since the DT run won't cover it? It's just a new folder in https://github.com/microsoft/typescript-error-deltas/tree/main/userTests right?

I believe so, @jakebailey can help you there.

@DanielRosenwasser DanielRosenwasser merged commit f762de5 into main Jun 26, 2023
17 checks passed
@DanielRosenwasser DanielRosenwasser deleted the undo52106 branch June 26, 2023 20:21
@typescript-bot
Copy link
Collaborator

@DanielRosenwasser Here are the results of running the top-repos suite comparing main and refs/pull/54781/merge:

Everything looks good!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
4 participants