Skip to content

Improve performance of maybe stack in recursiveTypeRelatedTo #55224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Aug 3, 2023
Merged

Improve performance of maybe stack in recursiveTypeRelatedTo #55224

merged 10 commits into from
Aug 3, 2023

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented Aug 1, 2023 •
edited
Loading

For #54939

I did a pprof profile of #54939 and found that a suspicious amount of time was being spent in recursiveTypeRelatedTo directly (self time).

image

The profile didn't really point to anything except for the loops over maybeKeys. I noticed that they were all operations where we could be looping over and over again when a set + stack could make every operation constant time.

So, I did that, and the test case from #54939 improved quite a bit:

Benchmark 1: old
  Time (mean ± σ):      3.115 s ±  0.067 s    [User: 4.403 s, System: 0.124 s]
  Range (min … max):    3.018 s …  3.196 s    10 runs
 
Benchmark 2: new
  Time (mean ± σ):      2.072 s ±  0.050 s    [User: 3.355 s, System: 0.135 s]
  Range (min … max):    1.985 s …  2.150 s    10 runs
 
Summary
  'new' ran
    1.50 ± 0.05 times faster than 'old'

On an even more inflated version of that test, I tripled the number of properties, and:

Benchmark 1: old
  Time (mean ± σ):      7.369 s ±  0.178 s    [User: 8.785 s, System: 0.225 s]
  Range (min … max):    7.136 s …  7.685 s    10 runs
 
Benchmark 2: new
  Time (mean ± σ):      3.437 s ±  0.073 s    [User: 4.913 s, System: 0.167 s]
  Range (min … max):    3.363 s …  3.566 s    10 runs
 
Summary
  'new' ran
    2.14 ± 0.07 times faster than 'old'

The current version of this PR doesn't appear to regress our regular test suites (some get better).

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Aug 1, 2023
@jakebailey
Copy link
Member Author

@typescript-bot test this
@typescript-bot test top100
@typescript-bot user test this
@typescript-bot user test tsserver
@typescript-bot test tsserver top100
@typescript-bot run dt
@typescript-bot perf test this
@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the extended test suite on this PR at 0009999. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based top-repos suite on this PR at 0009999. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the tarball bundle task on this PR at 0009999. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at 0009999. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based user code test suite (tsserver) on this PR at 0009999. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the parallelized Definitely Typed test suite on this PR at 0009999. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based top-repos suite (tsserver) on this PR at 0009999. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the perf test suite on this PR at 0009999. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/156098/artifacts?artifactName=tgz&fileId=E49FAF5AA96CA28D0EF28A168C4819F4AFF3183D35819C89C86845A911E47DC802&fileName=/typescript-5.2.0-insiders.20230801.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.2.0-pr-55224-9".;

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/55224/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Unknown failure"
  • 1 instance of "Package install failed"

Otherwise...

Something interesting changed - please have a look.

Details

rxjs-src

/mnt/ts_downloads/rxjs-src/build.sh

  • [NEW] error TS2428: All declarations of 'WeakMap' must have identical type parameters.
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55224/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
  • [MISSING] error TS2428: All declarations of 'WeakMap' must have identical type parameters.
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/55224/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..55224
Metric main 55224 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 368,785k (± 0.01%) 368,836k (± 0.00%) +51k (+ 0.01%) 368,808k 368,858k p=0.008 n=6
Parse Time 3.40s (± 0.55%) 3.40s (± 1.01%) ~ 3.36s 3.44s p=0.872 n=6
Bind Time 1.12s (± 0.92%) 1.12s (± 0.67%) ~ 1.11s 1.13s p=0.931 n=6
Check Time 8.95s (± 0.34%) 9.11s (± 1.07%) +0.16s (+ 1.77%) 9.01s 9.25s p=0.005 n=6
Emit Time 7.57s (± 1.03%) 7.57s (± 0.72%) ~ 7.51s 7.65s p=1.000 n=6
Total Time 21.05s (± 0.38%) 21.20s (± 0.71%) ~ 21.03s 21.46s p=0.065 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 193,051k (± 1.18%) 192,078k (± 0.02%) -974k (- 0.50%) 192,045k 192,150k p=0.045 n=6
Parse Time 1.51s (± 0.65%) 1.50s (± 0.70%) ~ 1.49s 1.52s p=0.315 n=6
Bind Time 0.78s (± 0.66%) 0.78s (± 1.08%) ~ 0.76s 0.78s p=0.923 n=6
Check Time 9.47s (± 0.42%) 9.86s (± 0.47%) +0.39s (+ 4.12%) 9.81s 9.93s p=0.005 n=6
Emit Time 2.76s (± 0.38%) 2.74s (± 1.21%) ~ 2.70s 2.79s p=0.375 n=6
Total Time 14.51s (± 0.35%) 14.88s (± 0.39%) +0.37s (+ 2.56%) 14.79s 14.94s p=0.005 n=6
Monaco - node (v18.10.0, x64)
Memory used 347,755k (± 0.01%) 347,799k (± 0.01%) ~ 347,724k 347,866k p=0.093 n=6
Parse Time 2.62s (± 1.05%) 2.62s (± 0.52%) ~ 2.60s 2.64s p=0.746 n=6
Bind Time 1.01s (± 0.74%) 1.02s (± 1.48%) ~ 1.00s 1.04s p=0.555 n=6
Check Time 7.29s (± 0.38%) 7.55s (± 0.47%) +0.26s (+ 3.54%) 7.50s 7.60s p=0.005 n=6
Emit Time 4.24s (± 0.90%) 4.27s (± 0.82%) ~ 4.25s 4.32s p=0.164 n=6
Total Time 15.16s (± 0.51%) 15.46s (± 0.38%) +0.30s (+ 1.96%) 15.38s 15.54s p=0.005 n=6
TFS - node (v18.10.0, x64)
Memory used 301,794k (± 0.01%) 301,803k (± 0.01%) ~ 301,784k 301,837k p=0.296 n=6
Parse Time 2.08s (± 1.23%) 2.10s (± 0.84%) ~ 2.08s 2.12s p=0.511 n=6
Bind Time 1.13s (± 1.03%) 1.12s (± 0.46%) ~ 1.12s 1.13s p=0.191 n=6
Check Time 6.67s (± 0.53%) 6.74s (± 0.50%) +0.06s (+ 0.97%) 6.69s 6.78s p=0.020 n=6
Emit Time 3.86s (± 0.95%) 3.90s (± 0.82%) ~ 3.87s 3.96s p=0.127 n=6
Total Time 13.75s (± 0.50%) 13.86s (± 0.29%) +0.11s (+ 0.81%) 13.80s 13.90s p=0.013 n=6
material-ui - node (v18.10.0, x64)
Memory used 482,550k (± 0.01%) 482,589k (± 0.01%) ~ 482,529k 482,705k p=0.230 n=6
Parse Time 3.11s (± 1.18%) 3.08s (± 2.67%) ~ 2.96s 3.15s p=1.000 n=6
Bind Time 0.92s (± 2.44%) 0.96s (± 5.93%) ~ 0.92s 1.03s p=0.179 n=6
Check Time 17.32s (± 0.73%) 17.99s (± 0.89%) +0.67s (+ 3.86%) 17.74s 18.21s p=0.005 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.36s (± 0.54%) 22.03s (± 0.76%) +0.68s (+ 3.17%) 21.78s 22.28s p=0.005 n=6
xstate - node (v18.10.0, x64)
Memory used 563,842k (± 0.02%) 563,892k (± 0.02%) ~ 563,794k 564,020k p=0.471 n=6
Parse Time 3.85s (± 0.61%) 3.88s (± 0.65%) ~ 3.84s 3.91s p=0.134 n=6
Bind Time 1.64s (± 1.24%) 1.64s (± 0.77%) ~ 1.62s 1.66s p=1.000 n=6
Check Time 2.81s (± 1.00%) 2.82s (± 0.58%) ~ 2.79s 2.84s p=1.000 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 8.39s (± 0.27%) 8.42s (± 0.59%) ~ 8.33s 8.48s p=0.127 n=6
Angular - node (v16.17.1, x64)
Memory used 368,218k (± 0.00%) 368,209k (± 0.01%) ~ 368,147k 368,242k p=0.689 n=6
Parse Time 3.57s (± 0.39%) 3.58s (± 0.48%) ~ 3.56s 3.60s p=0.194 n=6
Bind Time 1.19s (± 0.63%) 1.19s (± 0.83%) ~ 1.17s 1.20s p=0.858 n=6
Check Time 9.76s (± 0.72%) 9.91s (± 0.52%) +0.14s (+ 1.47%) 9.84s 9.97s p=0.005 n=6
Emit Time 8.04s (± 0.80%) 8.02s (± 1.08%) ~ 7.88s 8.11s p=1.000 n=6
Total Time 22.55s (± 0.58%) 22.69s (± 0.59%) ~ 22.48s 22.82s p=0.092 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,823k (± 0.02%) 193,810k (± 0.02%) ~ 193,782k 193,869k p=0.423 n=6
Parse Time 1.59s (± 0.96%) 1.60s (± 0.79%) ~ 1.58s 1.61s p=0.101 n=6
Bind Time 0.82s (± 0.92%) 0.82s (± 1.25%) ~ 0.81s 0.84s p=0.931 n=6
Check Time 10.16s (± 0.48%) 10.60s (± 0.63%) +0.44s (+ 4.33%) 10.50s 10.69s p=0.005 n=6
Emit Time 2.98s (± 0.87%) 2.99s (± 0.79%) ~ 2.95s 3.02s p=0.517 n=6
Total Time 15.56s (± 0.35%) 16.02s (± 0.49%) +0.46s (+ 2.98%) 15.91s 16.11s p=0.005 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,078k (± 0.01%) 347,091k (± 0.00%) ~ 347,077k 347,116k p=0.125 n=6
Parse Time 2.77s (± 0.44%) 2.78s (± 0.83%) ~ 2.74s 2.81s p=0.252 n=6
Bind Time 1.08s (± 0.48%) 1.08s (± 0.00%) ~ 1.08s 1.08s p=0.174 n=6
Check Time 8.04s (± 0.61%) 8.29s (± 0.35%) +0.25s (+ 3.07%) 8.24s 8.32s p=0.005 n=6
Emit Time 4.48s (± 0.56%) 4.50s (± 1.29%) ~ 4.44s 4.57s p=0.629 n=6
Total Time 16.36s (± 0.36%) 16.65s (± 0.45%) +0.29s (+ 1.74%) 16.56s 16.75s p=0.005 n=6
TFS - node (v16.17.1, x64)
Memory used 301,118k (± 0.01%) 301,123k (± 0.01%) ~ 301,098k 301,158k p=0.873 n=6
Parse Time 2.20s (± 0.38%) 2.21s (± 0.83%) ~ 2.19s 2.24s p=0.160 n=6
Bind Time 1.22s (± 1.70%) 1.21s (± 1.35%) ~ 1.19s 1.23s p=0.413 n=6
Check Time 7.34s (± 0.31%) 7.39s (± 0.37%) +0.05s (+ 0.68%) 7.36s 7.43s p=0.008 n=6
Emit Time 4.30s (± 1.17%) 4.34s (± 1.00%) +0.03s (+ 0.81%) 4.31s 4.41s p=0.041 n=6
Total Time 15.06s (± 0.25%) 15.15s (± 0.47%) +0.09s (+ 0.60%) 15.07s 15.27s p=0.030 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,882k (± 0.01%) 481,740k (± 0.05%) ~ 481,438k 481,962k p=0.810 n=6
Parse Time 3.26s (± 0.72%) 3.27s (± 0.72%) ~ 3.24s 3.30s p=0.332 n=6
Bind Time 0.96s (± 0.85%) 0.96s (± 1.22%) ~ 0.94s 0.97s p=0.738 n=6
Check Time 18.34s (± 0.39%) 19.08s (± 0.60%) +0.74s (+ 4.04%) 18.88s 19.20s p=0.005 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.55s (± 0.36%) 23.31s (± 0.55%) +0.75s (+ 3.35%) 23.08s 23.44s p=0.005 n=6
xstate - node (v16.17.1, x64)
Memory used 561,343k (± 0.01%) 561,407k (± 0.02%) ~ 561,276k 561,572k p=0.298 n=6
Parse Time 4.02s (± 0.30%) 4.02s (± 0.46%) ~ 4.00s 4.05s p=1.000 n=6
Bind Time 1.66s (± 5.88%) 1.77s (± 1.26%) +0.11s (+ 6.42%) 1.75s 1.81s p=0.043 n=6
Check Time 3.09s (± 1.60%) 3.05s (± 0.88%) ~ 3.00s 3.07s p=0.420 n=6
Emit Time 0.09s (± 5.53%) 0.09s (± 4.45%) ~ 0.09s 0.10s p=0.595 n=6
Total Time 8.87s (± 0.58%) 8.93s (± 0.39%) ~ 8.88s 8.98s p=0.054 n=6
Angular - node (v14.21.3, x64)
Memory used 362,126k (± 0.01%) 361,856k (± 0.01%) -271k (- 0.07%) 361,814k 361,884k p=0.005 n=6
Parse Time 3.71s (± 0.14%) 3.73s (± 0.28%) +0.02s (+ 0.54%) 3.71s 3.74s p=0.009 n=6
Bind Time 1.22s (± 0.00%) 1.22s (± 0.67%) ~ 1.21s 1.23s p=0.290 n=6
Check Time 10.13s (± 0.37%) 10.29s (± 0.49%) +0.16s (+ 1.60%) 10.23s 10.37s p=0.005 n=6
Emit Time 8.35s (± 0.56%) 8.74s (± 0.97%) +0.39s (+ 4.71%) 8.61s 8.84s p=0.005 n=6
Total Time 23.41s (± 0.27%) 23.98s (± 0.58%) +0.57s (+ 2.45%) 23.79s 24.17s p=0.005 n=6
Compiler-Unions - node (v14.21.3, x64)
Memory used 189,118k (± 0.01%) 189,117k (± 0.02%) ~ 189,079k 189,147k p=0.688 n=6
Parse Time 1.62s (± 0.72%) 1.62s (± 0.50%) ~ 1.61s 1.63s p=0.498 n=6
Bind Time 0.85s (± 0.61%) 0.85s (± 0.00%) ~ 0.85s 0.85s p=0.174 n=6
Check Time 10.32s (± 0.70%) 10.72s (± 0.37%) +0.40s (+ 3.93%) 10.68s 10.79s p=0.005 n=6
Emit Time 3.11s (± 0.55%) 3.13s (± 0.66%) ~ 3.10s 3.16s p=0.073 n=6
Total Time 15.90s (± 0.49%) 16.32s (± 0.26%) +0.42s (+ 2.66%) 16.29s 16.40s p=0.005 n=6
Monaco - node (v14.21.3, x64)
Memory used 342,072k (± 0.01%) 342,090k (± 0.01%) ~ 342,051k 342,137k p=0.575 n=6
Parse Time 2.80s (± 0.87%) 2.81s (± 0.54%) ~ 2.78s 2.82s p=0.256 n=6
Bind Time 1.09s (± 0.50%) 1.10s (± 0.89%) ~ 1.09s 1.12s p=0.201 n=6
Check Time 8.30s (± 0.26%) 8.55s (± 0.58%) +0.25s (+ 2.99%) 8.49s 8.62s p=0.005 n=6
Emit Time 4.67s (± 0.87%) 4.69s (± 0.67%) ~ 4.63s 4.71s p=0.419 n=6
Total Time 16.86s (± 0.40%) 17.14s (± 0.44%) +0.28s (+ 1.66%) 17.02s 17.22s p=0.005 n=6
TFS - node (v14.21.3, x64)
Memory used 296,199k (± 0.01%) 296,210k (± 0.01%) ~ 296,187k 296,234k p=0.423 n=6
Parse Time 2.42s (± 0.43%) 2.44s (± 0.99%) ~ 2.41s 2.48s p=0.105 n=6
Bind Time 1.08s (± 0.91%) 1.08s (± 0.83%) ~ 1.07s 1.09s p=0.798 n=6
Check Time 7.66s (± 0.42%) 7.71s (± 0.73%) ~ 7.61s 7.76s p=0.091 n=6
Emit Time 4.27s (± 0.69%) 4.31s (± 0.87%) ~ 4.26s 4.35s p=0.090 n=6
Total Time 15.44s (± 0.22%) 15.55s (± 0.46%) +0.10s (+ 0.68%) 15.46s 15.64s p=0.016 n=6
material-ui - node (v14.21.3, x64)
Memory used 477,350k (± 0.00%) 477,401k (± 0.00%) +51k (+ 0.01%) 477,386k 477,426k p=0.005 n=6
Parse Time 3.31s (± 0.31%) 3.32s (± 0.70%) ~ 3.29s 3.35s p=0.677 n=6
Bind Time 0.99s (± 0.41%) 0.99s (± 0.52%) ~ 0.99s 1.00s p=0.595 n=6
Check Time 19.15s (± 0.40%) 19.74s (± 0.47%) +0.59s (+ 3.08%) 19.61s 19.86s p=0.005 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.45s (± 0.30%) 24.05s (± 0.41%) +0.60s (+ 2.56%) 23.89s 24.16s p=0.005 n=6
xstate - node (v14.21.3, x64)
Memory used 550,168k (± 0.00%) 550,203k (± 0.01%) ~ 550,178k 550,250k p=0.128 n=6
Parse Time 4.22s (± 0.61%) 4.22s (± 0.62%) ~ 4.19s 4.26s p=0.936 n=6
Bind Time 1.69s (± 0.30%) 1.70s (± 0.61%) ~ 1.69s 1.72s p=0.070 n=6
Check Time 3.13s (± 0.48%) 3.17s (± 0.46%) +0.04s (+ 1.33%) 3.15s 3.19s p=0.005 n=6
Emit Time 0.10s (± 5.76%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.071 n=6
Total Time 9.13s (± 0.35%) 9.18s (± 0.30%) +0.05s (+ 0.58%) 9.16s 9.22s p=0.024 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.21.3, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.21.3, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.21.3, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.21.3, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.21.3, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.21.3, x64)
Benchmark Name Iterations
Current 55224 6
Baseline main 6

TSServer

Comparison Report - main..55224
Metric main 55224 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,525ms (± 1.15%) 2,534ms (± 1.20%) ~ 2,481ms 2,573ms p=0.630 n=6
Req 2 - geterr 5,384ms (± 0.47%) 5,880ms (± 0.72%) +496ms (+ 9.22%) 5,813ms 5,922ms p=0.005 n=6
Req 3 - references 342ms (± 0.86%) 346ms (± 1.37%) ~ 338ms 351ms p=0.196 n=6
Req 4 - navto 289ms (± 0.34%) 288ms (± 0.75%) ~ 285ms 291ms p=0.157 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 76ms (± 1.08%) 83ms (±10.30%) ~ 75ms 94ms p=0.135 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,618ms (± 0.49%) 2,603ms (± 0.78%) ~ 2,580ms 2,634ms p=0.173 n=6
Req 2 - geterr 4,136ms (± 0.41%) 4,173ms (± 0.79%) +38ms (+ 0.91%) 4,128ms 4,226ms p=0.030 n=6
Req 3 - references 350ms (± 1.01%) 347ms (± 0.84%) ~ 343ms 350ms p=0.253 n=6
Req 4 - navto 289ms (± 0.36%) 289ms (± 0.77%) ~ 287ms 293ms p=0.742 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 79ms (± 6.58%) 71ms (± 1.06%) 🟩-8ms (- 9.73%) 70ms 72ms p=0.007 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,088ms (± 0.53%) 3,092ms (± 0.48%) ~ 3,075ms 3,116ms p=0.810 n=6
Req 2 - geterr 1,602ms (± 0.57%) 1,562ms (± 0.60%) -40ms (- 2.51%) 1,549ms 1,578ms p=0.005 n=6
Req 3 - references 114ms (± 1.32%) 108ms (± 1.01%) 🟩-6ms (- 4.99%) 107ms 110ms p=0.004 n=6
Req 4 - navto 369ms (± 0.37%) 375ms (± 0.40%) +6ms (+ 1.62%) 374ms 378ms p=0.005 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 383ms (± 0.98%) 385ms (± 1.15%) ~ 378ms 392ms p=0.221 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,629ms (± 0.40%) 2,621ms (± 0.28%) ~ 2,613ms 2,633ms p=0.198 n=6
Req 2 - geterr 6,081ms (± 0.38%) 6,210ms (± 0.45%) +128ms (+ 2.11%) 6,171ms 6,240ms p=0.005 n=6
Req 3 - references 355ms (± 0.37%) 357ms (± 0.52%) ~ 355ms 360ms p=0.096 n=6
Req 4 - navto 284ms (± 0.41%) 284ms (± 0.47%) ~ 282ms 285ms p=0.547 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 83ms (± 7.49%) 81ms (± 0.63%) ~ 81ms 82ms p=0.666 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,803ms (± 0.52%) 2,805ms (± 0.82%) ~ 2,771ms 2,843ms p=1.000 n=6
Req 2 - geterr 4,689ms (± 0.44%) 4,724ms (± 0.42%) ~ 4,703ms 4,745ms p=0.065 n=6
Req 3 - references 364ms (± 0.23%) 362ms (± 0.76%) ~ 357ms 365ms p=0.096 n=6
Req 4 - navto 281ms (± 0.95%) 282ms (± 0.43%) ~ 279ms 282ms p=0.303 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 77ms (± 1.09%) 77ms (± 0.67%) ~ 76ms 77ms p=0.923 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,230ms (± 0.30%) 3,232ms (± 0.41%) ~ 3,207ms 3,244ms p=0.378 n=6
Req 2 - geterr 1,733ms (± 1.20%) 1,726ms (± 0.71%) ~ 1,712ms 1,741ms p=0.748 n=6
Req 3 - references 130ms (± 7.76%) 129ms (± 8.11%) ~ 117ms 145ms p=0.934 n=6
Req 4 - navto 352ms (± 0.49%) 356ms (± 1.99%) ~ 350ms 366ms p=0.684 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 429ms (± 0.71%) 408ms (± 0.97%) 🟩-22ms (- 5.01%) 402ms 412ms p=0.005 n=6
Compiler-UnionsTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,771ms (± 0.36%) 2,780ms (± 0.98%) ~ 2,732ms 2,808ms p=0.230 n=6
Req 2 - geterr 6,220ms (± 0.51%) 6,457ms (± 1.32%) +237ms (+ 3.81%) 6,357ms 6,602ms p=0.005 n=6
Req 3 - references 361ms (± 0.51%) 365ms (± 0.94%) ~ 362ms 371ms p=0.061 n=6
Req 4 - navto 291ms (± 0.92%) 292ms (± 0.94%) ~ 290ms 297ms p=0.676 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 102ms (± 8.45%) 95ms (± 7.94%) ~ 91ms 110ms p=0.122 n=6
CompilerTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,936ms (± 0.28%) 2,936ms (± 0.52%) ~ 2,911ms 2,952ms p=0.689 n=6
Req 2 - geterr 4,583ms (± 0.44%) 4,625ms (± 0.53%) +42ms (+ 0.92%) 4,580ms 4,647ms p=0.020 n=6
Req 3 - references 370ms (± 0.76%) 379ms (± 1.21%) +9ms (+ 2.39%) 373ms 385ms p=0.010 n=6
Req 4 - navto 299ms (± 0.46%) 297ms (± 0.39%) -2ms (- 0.61%) 295ms 298ms p=0.033 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 86ms (± 8.01%) 85ms (± 3.37%) ~ 83ms 89ms p=0.560 n=6
xstateTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 3,498ms (± 1.79%) 3,491ms (± 1.53%) ~ 3,408ms 3,554ms p=0.575 n=6
Req 2 - geterr 1,863ms (± 0.80%) 1,831ms (± 0.75%) -32ms (- 1.71%) 1,807ms 1,843ms p=0.013 n=6
Req 3 - references 139ms (± 8.58%) 150ms (± 7.23%) ~ 133ms 160ms p=0.054 n=6
Req 4 - navto 391ms (± 0.77%) 393ms (± 2.02%) ~ 382ms 403ms p=0.469 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 420ms (± 1.35%) 424ms (± 1.65%) ~ 416ms 435ms p=0.332 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.21.3, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.21.3, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.21.3, x64)
Benchmark Name Iterations
Current 55224 6
Baseline main 6

Startup

Comparison Report - main..55224
Metric main 55224 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 143.04ms (± 0.25%) 142.94ms (± 0.23%) ~ 141.88ms 150.12ms p=0.181 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 222.23ms (± 0.17%) 222.74ms (± 0.28%) +0.51ms (+ 0.23%) 221.08ms 229.66ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 223.71ms (± 0.17%) 223.69ms (± 0.20%) ~ 222.47ms 230.93ms p=0.103 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 206.38ms (± 0.24%) 206.00ms (± 0.23%) -0.37ms (- 0.18%) 204.51ms 210.43ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 55224 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey jakebailey marked this pull request as draft August 1, 2023 06:00
@jakebailey
Copy link
Member Author

Hm, slower...

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/55224/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Hey @jakebailey, the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/55224/merge:

Everything looks good!

jakebailey
jakebailey commented Aug 1, 2023
View reviewed changes
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the perf test suite on this PR at a703f48. You can monitor the build here.

Update: The results are in!

jakebailey
jakebailey commented Aug 1, 2023
View reviewed changes
@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..55224
Metric main 55224 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 368,808k (± 0.00%) 368,803k (± 0.01%) ~ 368,764k 368,840k p=0.575 n=6
Parse Time 3.43s (± 0.80%) 3.40s (± 1.08%) ~ 3.37s 3.46s p=0.294 n=6
Bind Time 1.12s (± 1.18%) 1.12s (± 0.36%) ~ 1.12s 1.13s p=0.924 n=6
Check Time 8.99s (± 0.67%) 9.01s (± 0.51%) ~ 8.94s 9.06s p=0.574 n=6
Emit Time 7.55s (± 1.02%) 7.53s (± 0.83%) ~ 7.46s 7.60s p=0.629 n=6
Total Time 21.09s (± 0.68%) 21.06s (± 0.53%) ~ 20.95s 21.24s p=0.810 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 193,049k (± 1.18%) 193,936k (± 1.53%) ~ 191,968k 197,802k p=0.689 n=6
Parse Time 1.53s (± 0.68%) 1.51s (± 0.54%) -0.02s (- 1.31%) 1.50s 1.52s p=0.014 n=6
Bind Time 0.78s (± 0.00%) 0.78s (± 0.71%) ~ 0.77s 0.78s p=0.071 n=6
Check Time 9.49s (± 0.68%) 9.56s (± 0.45%) ~ 9.51s 9.61s p=0.065 n=6
Emit Time 2.75s (± 1.22%) 2.72s (± 0.97%) ~ 2.69s 2.76s p=0.141 n=6
Total Time 14.55s (± 0.43%) 14.56s (± 0.36%) ~ 14.50s 14.63s p=0.747 n=6
Monaco - node (v18.10.0, x64)
Memory used 347,755k (± 0.01%) 347,756k (± 0.00%) ~ 347,736k 347,770k p=0.378 n=6
Parse Time 2.63s (± 1.21%) 2.62s (± 0.81%) ~ 2.60s 2.65s p=0.625 n=6
Bind Time 1.01s (± 1.36%) 1.02s (± 1.07%) ~ 1.01s 1.04s p=0.621 n=6
Check Time 7.34s (± 0.54%) 7.36s (± 0.39%) ~ 7.33s 7.40s p=0.256 n=6
Emit Time 4.29s (± 0.67%) 4.27s (± 0.76%) ~ 4.23s 4.31s p=0.195 n=6
Total Time 15.28s (± 0.44%) 15.27s (± 0.49%) ~ 15.17s 15.38s p=0.872 n=6
TFS - node (v18.10.0, x64)
Memory used 301,782k (± 0.00%) 301,784k (± 0.01%) ~ 301,739k 301,802k p=0.378 n=6
Parse Time 2.09s (± 1.23%) 2.08s (± 0.95%) ~ 2.06s 2.10s p=0.289 n=6
Bind Time 1.13s (± 0.74%) 1.13s (± 0.74%) ~ 1.12s 1.14s p=1.000 n=6
Check Time 6.69s (± 0.49%) 6.70s (± 0.30%) ~ 6.67s 6.72s p=0.516 n=6
Emit Time 3.90s (± 0.59%) 3.88s (± 0.67%) ~ 3.86s 3.93s p=0.420 n=6
Total Time 13.81s (± 0.43%) 13.79s (± 0.29%) ~ 13.73s 13.83s p=0.260 n=6
material-ui - node (v18.10.0, x64)
Memory used 482,598k (± 0.01%) 482,609k (± 0.01%) ~ 482,582k 482,666k p=0.810 n=6
Parse Time 3.08s (± 2.62%) 3.10s (± 1.90%) ~ 2.98s 3.14s p=1.000 n=6
Bind Time 0.96s (± 6.47%) 0.93s (± 3.81%) ~ 0.91s 1.00s p=0.079 n=6
Check Time 17.53s (± 0.49%) 17.46s (± 0.85%) ~ 17.31s 17.69s p=0.574 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.57s (± 0.38%) 21.49s (± 0.74%) ~ 21.35s 21.74s p=0.297 n=6
xstate - node (v18.10.0, x64)
Memory used 563,811k (± 0.02%) 563,755k (± 0.01%) ~ 563,658k 563,899k p=0.378 n=6
Parse Time 3.88s (± 0.94%) 3.87s (± 0.31%) ~ 3.85s 3.88s p=0.466 n=6
Bind Time 1.66s (± 0.80%) 1.65s (± 0.73%) ~ 1.63s 1.66s p=0.192 n=6
Check Time 2.82s (± 0.37%) 2.78s (± 0.70%) -0.04s (- 1.59%) 2.75s 2.80s p=0.005 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 8.45s (± 0.47%) 8.38s (± 0.31%) -0.07s (- 0.81%) 8.35s 8.41s p=0.019 n=6
Angular - node (v16.17.1, x64)
Memory used 368,225k (± 0.01%) 368,182k (± 0.00%) -43k (- 0.01%) 368,165k 368,196k p=0.031 n=6
Parse Time 3.58s (± 0.50%) 3.58s (± 0.74%) ~ 3.54s 3.61s p=1.000 n=6
Bind Time 1.19s (± 0.68%) 1.19s (± 0.63%) ~ 1.18s 1.20s p=0.306 n=6
Check Time 9.77s (± 0.56%) 9.80s (± 0.31%) ~ 9.76s 9.85s p=0.377 n=6
Emit Time 8.00s (± 1.15%) 8.02s (± 0.54%) ~ 7.98s 8.08s p=0.747 n=6
Total Time 22.54s (± 0.57%) 22.58s (± 0.15%) ~ 22.55s 22.63s p=0.377 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,838k (± 0.04%) 193,831k (± 0.02%) ~ 193,799k 193,889k p=0.810 n=6
Parse Time 1.59s (± 0.86%) 1.59s (± 0.94%) ~ 1.57s 1.61s p=0.869 n=6
Bind Time 0.83s (± 0.90%) 0.82s (± 0.63%) ~ 0.82s 0.83s p=0.069 n=6
Check Time 10.21s (± 0.42%) 10.26s (± 0.41%) ~ 10.22s 10.32s p=0.171 n=6
Emit Time 3.02s (± 0.65%) 3.02s (± 1.72%) ~ 2.96s 3.10s p=0.747 n=6
Total Time 15.66s (± 0.40%) 15.69s (± 0.50%) ~ 15.58s 15.78s p=0.521 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,074k (± 0.01%) 347,075k (± 0.01%) ~ 347,052k 347,098k p=1.000 n=6
Parse Time 2.79s (± 0.29%) 2.78s (± 0.35%) -0.02s (- 0.54%) 2.76s 2.79s p=0.023 n=6
Bind Time 1.08s (± 0.38%) 1.08s (± 0.48%) ~ 1.07s 1.08s p=0.114 n=6
Check Time 8.06s (± 0.53%) 8.06s (± 0.51%) ~ 8.02s 8.11s p=1.000 n=6
Emit Time 4.49s (± 0.61%) 4.46s (± 0.87%) ~ 4.41s 4.51s p=0.226 n=6
Total Time 16.43s (± 0.36%) 16.37s (± 0.48%) ~ 16.26s 16.47s p=0.229 n=6
TFS - node (v16.17.1, x64)
Memory used 301,104k (± 0.01%) 301,128k (± 0.01%) ~ 301,091k 301,147k p=0.173 n=6
Parse Time 2.21s (± 0.44%) 2.21s (± 0.94%) ~ 2.18s 2.23s p=0.935 n=6
Bind Time 1.21s (± 1.13%) 1.21s (± 1.21%) ~ 1.19s 1.23s p=0.563 n=6
Check Time 7.34s (± 0.20%) 7.35s (± 0.19%) ~ 7.34s 7.37s p=0.087 n=6
Emit Time 4.32s (± 0.49%) 4.33s (± 0.94%) ~ 4.28s 4.38s p=0.935 n=6
Total Time 15.08s (± 0.20%) 15.10s (± 0.35%) ~ 15.03s 15.19s p=0.293 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,918k (± 0.02%) 481,876k (± 0.01%) ~ 481,819k 481,917k p=0.378 n=6
Parse Time 3.27s (± 0.32%) 3.26s (± 0.71%) ~ 3.22s 3.29s p=0.142 n=6
Bind Time 0.97s (± 0.53%) 0.96s (± 0.43%) -0.01s (- 0.86%) 0.95s 0.96s p=0.022 n=6
Check Time 18.46s (± 0.90%) 18.35s (± 0.65%) ~ 18.23s 18.57s p=0.261 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.70s (± 0.73%) 22.57s (± 0.58%) ~ 22.41s 22.79s p=0.199 n=6
xstate - node (v16.17.1, x64)
Memory used 561,352k (± 0.01%) 561,426k (± 0.02%) ~ 561,272k 561,601k p=0.199 n=6
Parse Time 4.05s (± 0.33%) 4.03s (± 0.24%) -0.02s (- 0.58%) 4.02s 4.04s p=0.012 n=6
Bind Time 1.72s (± 6.23%) 1.74s (± 4.55%) ~ 1.58s 1.80s p=0.808 n=6
Check Time 3.13s (± 3.15%) 3.05s (± 2.56%) -0.07s (- 2.35%) 3.00s 3.21s p=0.045 n=6
Emit Time 0.09s (± 4.45%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.405 n=6
Total Time 8.99s (± 0.68%) 8.91s (± 0.23%) ~ 8.88s 8.93s p=0.065 n=6
Angular - node (v14.21.3, x64)
Memory used 362,116k (± 0.01%) 362,112k (± 0.01%) ~ 362,080k 362,141k p=0.810 n=6
Parse Time 3.76s (± 0.78%) 3.73s (± 0.71%) ~ 3.70s 3.77s p=0.170 n=6
Bind Time 1.23s (± 0.51%) 1.22s (± 0.67%) ~ 1.21s 1.23s p=0.177 n=6
Check Time 10.19s (± 0.43%) 10.17s (± 0.34%) ~ 10.11s 10.21s p=0.571 n=6
Emit Time 8.40s (± 0.83%) 8.38s (± 0.70%) ~ 8.32s 8.48s p=0.422 n=6
Total Time 23.57s (± 0.35%) 23.50s (± 0.26%) ~ 23.46s 23.61s p=0.168 n=6
Compiler-Unions - node (v14.21.3, x64)
Memory used 189,091k (± 0.01%) 189,140k (± 0.01%) +50k (+ 0.03%) 189,114k 189,168k p=0.020 n=6
Parse Time 1.63s (± 0.63%) 1.62s (± 0.50%) ~ 1.61s 1.63s p=0.546 n=6
Bind Time 0.86s (± 0.48%) 0.85s (± 0.00%) -0.01s (- 0.97%) 0.85s 0.85s p=0.007 n=6
Check Time 10.40s (± 0.64%) 10.41s (± 0.44%) ~ 10.36s 10.49s p=0.686 n=6
Emit Time 3.12s (± 0.74%) 3.12s (± 0.39%) ~ 3.10s 3.13s p=0.366 n=6
Total Time 16.00s (± 0.46%) 16.00s (± 0.36%) ~ 15.94s 16.10s p=0.748 n=6
Monaco - node (v14.21.3, x64)
Memory used 342,083k (± 0.01%) 342,087k (± 0.01%) ~ 342,047k 342,111k p=0.748 n=6
Parse Time 2.82s (± 0.53%) 2.81s (± 0.66%) ~ 2.78s 2.83s p=0.933 n=6
Bind Time 1.10s (± 0.37%) 1.10s (± 0.68%) ~ 1.09s 1.11s p=0.389 n=6
Check Time 8.30s (± 0.50%) 8.38s (± 0.31%) +0.08s (+ 0.94%) 8.34s 8.42s p=0.008 n=6
Emit Time 4.68s (± 0.77%) 4.66s (± 0.76%) ~ 4.63s 4.73s p=0.333 n=6
Total Time 16.90s (± 0.36%) 16.96s (± 0.25%) ~ 16.91s 17.03s p=0.173 n=6
TFS - node (v14.21.3, x64)
Memory used 296,197k (± 0.00%) 296,204k (± 0.00%) ~ 296,186k 296,215k p=0.378 n=6
Parse Time 2.44s (± 0.76%) 2.43s (± 1.15%) ~ 2.39s 2.46s p=0.686 n=6
Bind Time 1.08s (± 0.75%) 1.08s (± 0.51%) ~ 1.07s 1.08s p=0.088 n=6
Check Time 7.67s (± 0.38%) 7.68s (± 0.33%) ~ 7.65s 7.71s p=0.514 n=6
Emit Time 4.32s (± 0.71%) 4.30s (± 0.62%) ~ 4.27s 4.33s p=0.257 n=6
Total Time 15.51s (± 0.21%) 15.48s (± 0.31%) ~ 15.41s 15.53s p=0.295 n=6
material-ui - node (v14.21.3, x64)
Memory used 477,343k (± 0.00%) 477,350k (± 0.00%) ~ 477,338k 477,357k p=0.377 n=6
Parse Time 3.33s (± 0.45%) 3.30s (± 0.16%) -0.03s (- 0.90%) 3.29s 3.30s p=0.010 n=6
Bind Time 0.99s (± 0.52%) 0.99s (± 0.64%) ~ 0.98s 1.00s p=0.386 n=6
Check Time 19.27s (± 0.75%) 19.27s (± 0.27%) ~ 19.19s 19.35s p=0.936 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.59s (± 0.64%) 23.56s (± 0.21%) ~ 23.49s 23.64s p=0.748 n=6
xstate - node (v14.21.3, x64)
Memory used 550,160k (± 0.01%) 550,223k (± 0.01%) +63k (+ 0.01%) 550,145k 550,278k p=0.045 n=6
Parse Time 4.24s (± 0.89%) 4.23s (± 0.90%) ~ 4.19s 4.28s p=0.809 n=6
Bind Time 1.68s (± 1.87%) 1.70s (± 0.78%) ~ 1.69s 1.72s p=0.565 n=6
Check Time 3.13s (± 0.31%) 3.13s (± 0.65%) ~ 3.10s 3.16s p=0.868 n=6
Emit Time 0.10s (± 4.15%) 0.10s (± 5.34%) ~ 0.09s 0.10s p=0.595 n=6
Total Time 9.15s (± 0.60%) 9.16s (± 0.56%) ~ 9.09s 9.23s p=1.000 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.21.3, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.21.3, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.21.3, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.21.3, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.21.3, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.21.3, x64)
Benchmark Name Iterations
Current 55224 6
Baseline main 6

TSServer

Comparison Report - main..55224
Metric main 55224 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,561ms (± 0.72%) 2,543ms (± 0.73%) ~ 2,519ms 2,565ms p=0.173 n=6
Req 2 - geterr 5,425ms (± 0.64%) 5,426ms (± 0.32%) ~ 5,394ms 5,444ms p=0.575 n=6
Req 3 - references 343ms (± 1.40%) 347ms (± 0.67%) ~ 344ms 350ms p=0.197 n=6
Req 4 - navto 290ms (± 0.28%) 289ms (± 0.48%) ~ 287ms 290ms p=0.183 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 76ms (± 1.08%) 75ms (± 1.00%) ~ 74ms 76ms p=0.383 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,644ms (± 0.66%) 2,613ms (± 0.71%) -31ms (- 1.17%) 2,590ms 2,632ms p=0.020 n=6
Req 2 - geterr 4,147ms (± 0.35%) 4,151ms (± 0.31%) ~ 4,139ms 4,170ms p=0.810 n=6
Req 3 - references 352ms (± 0.53%) 356ms (± 0.65%) +4ms (+ 1.18%) 353ms 359ms p=0.016 n=6
Req 4 - navto 290ms (± 0.36%) 290ms (± 0.48%) ~ 287ms 291ms p=0.933 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 80ms (± 5.70%) 89ms (± 1.18%) +9ms (+10.63%) 87ms 90ms p=0.029 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,096ms (± 0.68%) 3,100ms (± 0.47%) ~ 3,077ms 3,118ms p=0.873 n=6
Req 2 - geterr 1,638ms (± 5.72%) 1,550ms (± 0.56%) 🟩-88ms (- 5.39%) 1,542ms 1,561ms p=0.005 n=6
Req 3 - references 113ms (± 3.14%) 115ms (± 1.07%) ~ 113ms 116ms p=0.454 n=6
Req 4 - navto 371ms (± 0.76%) 369ms (± 0.48%) ~ 366ms 371ms p=0.560 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 385ms (± 1.09%) 385ms (± 1.69%) ~ 377ms 391ms p=0.686 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,629ms (± 0.76%) 2,641ms (± 0.98%) ~ 2,612ms 2,687ms p=0.575 n=6
Req 2 - geterr 6,082ms (± 0.69%) 6,053ms (± 2.31%) ~ 5,772ms 6,148ms p=0.630 n=6
Req 3 - references 356ms (± 0.99%) 355ms (± 1.09%) ~ 352ms 362ms p=0.359 n=6
Req 4 - navto 285ms (± 0.41%) 284ms (± 0.58%) ~ 281ms 286ms p=0.113 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 83ms (± 7.61%) 91ms (± 9.05%) ~ 81ms 101ms p=0.122 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,810ms (± 0.62%) 2,816ms (± 0.54%) ~ 2,799ms 2,833ms p=0.630 n=6
Req 2 - geterr 4,699ms (± 0.40%) 4,703ms (± 0.46%) ~ 4,675ms 4,728ms p=0.689 n=6
Req 3 - references 366ms (± 0.66%) 364ms (± 0.62%) ~ 361ms 366ms p=0.244 n=6
Req 4 - navto 283ms (± 1.26%) 282ms (± 0.83%) ~ 279ms 284ms p=0.747 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 78ms (± 5.06%) 77ms (± 0.98%) ~ 76ms 78ms p=0.796 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,246ms (± 0.44%) 3,233ms (± 0.60%) ~ 3,212ms 3,265ms p=0.230 n=6
Req 2 - geterr 1,751ms (± 1.31%) 1,700ms (± 0.78%) -51ms (- 2.90%) 1,677ms 1,713ms p=0.005 n=6
Req 3 - references 130ms (± 8.50%) 128ms (± 6.86%) ~ 123ms 146ms p=0.517 n=6
Req 4 - navto 352ms (± 0.58%) 350ms (± 0.21%) ~ 349ms 351ms p=0.080 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 424ms (± 0.83%) 422ms (± 0.87%) ~ 418ms 428ms p=0.468 n=6
Compiler-UnionsTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,789ms (± 0.27%) 2,777ms (± 0.60%) ~ 2,761ms 2,800ms p=0.229 n=6
Req 2 - geterr 6,243ms (± 0.43%) 6,315ms (± 1.20%) +73ms (+ 1.16%) 6,224ms 6,449ms p=0.024 n=6
Req 3 - references 365ms (± 1.05%) 366ms (± 1.42%) ~ 359ms 373ms p=0.809 n=6
Req 4 - navto 292ms (± 0.29%) 292ms (± 0.52%) ~ 290ms 294ms p=0.863 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 100ms (± 8.16%) 107ms (± 6.32%) ~ 98ms 112ms p=0.167 n=6
CompilerTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,939ms (± 0.36%) 2,937ms (± 0.82%) ~ 2,913ms 2,968ms p=0.630 n=6
Req 2 - geterr 4,583ms (± 0.58%) 4,605ms (± 0.31%) ~ 4,586ms 4,622ms p=0.128 n=6
Req 3 - references 371ms (± 0.77%) 373ms (± 0.84%) ~ 369ms 377ms p=0.363 n=6
Req 4 - navto 298ms (± 0.81%) 295ms (± 0.72%) ~ 293ms 298ms p=0.071 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 85ms (± 2.66%) 89ms (± 9.17%) ~ 83ms 100ms p=0.622 n=6
xstateTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 3,524ms (± 1.59%) 3,480ms (± 1.41%) ~ 3,414ms 3,520ms p=0.108 n=6
Req 2 - geterr 1,864ms (± 0.72%) 1,819ms (± 0.37%) -45ms (- 2.42%) 1,813ms 1,831ms p=0.005 n=6
Req 3 - references 144ms (± 9.06%) 135ms (± 5.97%) ~ 130ms 151ms p=0.223 n=6
Req 4 - navto 391ms (± 0.98%) 390ms (± 1.03%) ~ 386ms 396ms p=0.686 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 421ms (± 0.69%) 421ms (± 0.98%) ~ 415ms 426ms p=0.872 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.21.3, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.21.3, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.21.3, x64)
Benchmark Name Iterations
Current 55224 6
Baseline main 6

Startup

Comparison Report - main..55224
Metric main 55224 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 143.34ms (± 0.21%) 143.40ms (± 0.15%) +0.06ms (+ 0.04%) 142.34ms 145.39ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 226.06ms (± 0.53%) 223.66ms (± 0.30%) -2.40ms (- 1.06%) 221.83ms 230.20ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 227.58ms (± 0.31%) 224.22ms (± 0.18%) -3.37ms (- 1.48%) 223.06ms 229.15ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 208.93ms (± 0.33%) 206.09ms (± 0.17%) -2.84ms (- 1.36%) 205.03ms 210.64ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 55224 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

@typescript-bot perf test this

jakebailey
jakebailey commented Aug 1, 2023
View reviewed changes
jakebailey
jakebailey commented Aug 1, 2023
View reviewed changes
DanielRosenwasser
DanielRosenwasser reviewed Aug 1, 2023
View reviewed changes
DanielRosenwasser
DanielRosenwasser reviewed Aug 1, 2023
View reviewed changes
@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at 42d68cb. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..55224

Metric main 55224 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 368,225k (± 0.01%) 368,206k (± 0.00%) ~ 368,190k 368,229k p=0.521 n=6
Parse Time 3.58s (± 0.50%) 3.56s (± 1.24%) ~ 3.52s 3.64s p=0.332 n=6
Bind Time 1.19s (± 0.68%) 1.19s (± 0.63%) ~ 1.18s 1.20s p=0.306 n=6
Check Time 9.77s (± 0.56%) 9.76s (± 0.29%) ~ 9.73s 9.79s p=0.872 n=6
Emit Time 8.00s (± 1.15%) 8.00s (± 0.40%) ~ 7.95s 8.04s p=1.000 n=6
Total Time 22.54s (± 0.57%) 22.50s (± 0.24%) ~ 22.46s 22.58s p=0.630 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,838k (± 0.04%) 193,841k (± 0.02%) ~ 193,786k 193,917k p=0.689 n=6
Parse Time 1.59s (± 0.86%) 1.59s (± 0.69%) ~ 1.57s 1.60s p=0.621 n=6
Bind Time 0.83s (± 0.90%) 0.82s (± 0.50%) -0.01s (- 1.20%) 0.82s 0.83s p=0.029 n=6
Check Time 10.21s (± 0.42%) 10.22s (± 0.88%) ~ 10.13s 10.37s p=0.872 n=6
Emit Time 3.02s (± 0.65%) 2.97s (± 0.66%) -0.06s (- 1.93%) 2.94s 3.00s p=0.006 n=6
Total Time 15.66s (± 0.40%) 15.60s (± 0.56%) ~ 15.50s 15.75s p=0.199 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,074k (± 0.01%) 347,094k (± 0.01%) ~ 347,073k 347,136k p=0.093 n=6
Parse Time 2.79s (± 0.29%) 2.76s (± 0.49%) -0.03s (- 1.07%) 2.75s 2.78s p=0.007 n=6
Bind Time 1.08s (± 0.38%) 1.08s (± 0.51%) ~ 1.07s 1.08s p=0.054 n=6
Check Time 8.06s (± 0.53%) 8.06s (± 0.45%) ~ 7.99s 8.09s p=0.627 n=6
Emit Time 4.49s (± 0.61%) 4.44s (± 0.79%) ~ 4.40s 4.49s p=0.053 n=6
Total Time 16.43s (± 0.36%) 16.34s (± 0.34%) -0.09s (- 0.57%) 16.25s 16.41s p=0.016 n=6
TFS - node (v16.17.1, x64)
Memory used 301,104k (± 0.01%) 301,127k (± 0.01%) ~ 301,100k 301,148k p=0.077 n=6
Parse Time 2.21s (± 0.44%) 2.20s (± 0.23%) ~ 2.20s 2.21s p=0.417 n=6
Bind Time 1.21s (± 1.13%) 1.23s (± 1.30%) +0.02s (+ 1.80%) 1.22s 1.26s p=0.021 n=6
Check Time 7.34s (± 0.20%) 7.36s (± 0.44%) ~ 7.32s 7.40s p=0.326 n=6
Emit Time 4.32s (± 0.49%) 4.29s (± 0.29%) -0.03s (- 0.73%) 4.28s 4.31s p=0.022 n=6
Total Time 15.08s (± 0.20%) 15.07s (± 0.33%) ~ 15.02s 15.16s p=0.746 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,918k (± 0.02%) 481,838k (± 0.00%) ~ 481,817k 481,861k p=0.230 n=6
Parse Time 3.27s (± 0.32%) 3.27s (± 0.65%) ~ 3.25s 3.31s p=0.188 n=6
Bind Time 0.97s (± 0.53%) 0.96s (± 0.54%) -0.01s (- 1.03%) 0.95s 0.96s p=0.018 n=6
Check Time 18.46s (± 0.90%) 18.38s (± 0.66%) ~ 18.23s 18.58s p=0.574 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.70s (± 0.73%) 22.61s (± 0.60%) ~ 22.45s 22.85s p=0.470 n=6
xstate - node (v16.17.1, x64)
Memory used 561,352k (± 0.01%) 561,368k (± 0.01%) ~ 561,283k 561,501k p=0.810 n=6
Parse Time 4.05s (± 0.33%) 4.02s (± 0.31%) -0.03s (- 0.78%) 4.00s 4.04s p=0.007 n=6
Bind Time 1.72s (± 6.23%) 1.72s (± 4.33%) ~ 1.57s 1.76s p=0.331 n=6
Check Time 3.13s (± 3.15%) 3.03s (± 2.77%) 🟩-0.10s (- 3.04%) 2.98s 3.20s p=0.045 n=6
Emit Time 0.09s (± 4.45%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.405 n=6
Total Time 8.99s (± 0.68%) 8.86s (± 0.15%) -0.12s (- 1.35%) 8.85s 8.88s p=0.013 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 55224 6
Baseline main 6

Developer Information:

Download Benchmark

@ahejlsberg
Copy link
Member

I'm certain @weswigham had a PR to this effect some time ago, but now I can't find it.

DanielRosenwasser
DanielRosenwasser reviewed Aug 2, 2023
View reviewed changes
@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 3, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at b59f2b6. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..55224

Metric main 55224 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 368,218k (± 0.01%) 368,184k (± 0.01%) -34k (- 0.01%) 368,162k 368,218k p=0.031 n=6
Parse Time 3.56s (± 0.60%) 3.56s (± 0.47%) ~ 3.54s 3.58s p=1.000 n=6
Bind Time 1.19s (± 0.53%) 1.19s (± 0.34%) ~ 1.19s 1.20s p=0.673 n=6
Check Time 9.75s (± 0.75%) 9.79s (± 0.44%) ~ 9.72s 9.85s p=0.422 n=6
Emit Time 7.95s (± 0.76%) 7.96s (± 0.38%) ~ 7.93s 8.00s p=0.520 n=6
Total Time 22.44s (± 0.47%) 22.51s (± 0.21%) ~ 22.42s 22.56s p=0.199 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,830k (± 0.01%) 194,344k (± 0.68%) ~ 193,722k 197,052k p=0.936 n=6
Parse Time 1.59s (± 0.51%) 1.59s (± 0.76%) ~ 1.57s 1.60s p=0.354 n=6
Bind Time 0.82s (± 0.99%) 0.82s (± 1.09%) ~ 0.81s 0.83s p=0.550 n=6
Check Time 10.18s (± 0.42%) 10.17s (± 0.40%) ~ 10.13s 10.25s p=1.000 n=6
Emit Time 3.00s (± 1.31%) 2.98s (± 1.13%) ~ 2.96s 3.05s p=0.225 n=6
Total Time 15.60s (± 0.41%) 15.56s (± 0.21%) ~ 15.52s 15.62s p=0.366 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,086k (± 0.00%) 347,076k (± 0.00%) ~ 347,065k 347,099k p=0.229 n=6
Parse Time 2.76s (± 0.51%) 2.76s (± 0.36%) ~ 2.75s 2.78s p=0.932 n=6
Bind Time 1.07s (± 0.48%) 1.08s (± 0.48%) ~ 1.07s 1.08s p=0.311 n=6
Check Time 8.01s (± 0.47%) 8.05s (± 0.54%) ~ 8.00s 8.11s p=0.146 n=6
Emit Time 4.46s (± 0.60%) 4.45s (± 0.76%) ~ 4.42s 4.51s p=0.746 n=6
Total Time 16.30s (± 0.34%) 16.35s (± 0.48%) ~ 16.25s 16.48s p=0.298 n=6
TFS - node (v16.17.1, x64)
Memory used 301,133k (± 0.01%) 301,121k (± 0.01%) ~ 301,095k 301,146k p=0.574 n=6
Parse Time 2.19s (± 0.34%) 2.20s (± 0.47%) +0.02s (+ 0.69%) 2.19s 2.22s p=0.025 n=6
Bind Time 1.22s (± 1.84%) 1.22s (± 1.41%) ~ 1.19s 1.24s p=0.806 n=6
Check Time 7.36s (± 0.33%) 7.34s (± 0.33%) ~ 7.30s 7.37s p=0.745 n=6
Emit Time 4.31s (± 0.85%) 4.29s (± 0.34%) ~ 4.27s 4.31s p=0.170 n=6
Total Time 15.08s (± 0.37%) 15.05s (± 0.37%) ~ 14.97s 15.11s p=0.630 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,936k (± 0.01%) 481,875k (± 0.01%) -61k (- 0.01%) 481,846k 481,929k p=0.020 n=6
Parse Time 3.26s (± 0.38%) 3.24s (± 0.36%) ~ 3.22s 3.25s p=0.056 n=6
Bind Time 0.95s (± 0.43%) 0.96s (± 0.66%) +0.01s (+ 0.88%) 0.95s 0.97s p=0.033 n=6
Check Time 18.33s (± 0.44%) 18.36s (± 0.39%) ~ 18.26s 18.47s p=0.520 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.53s (± 0.33%) 22.56s (± 0.36%) ~ 22.47s 22.69s p=0.810 n=6
xstate - node (v16.17.1, x64)
Memory used 561,443k (± 0.02%) 561,390k (± 0.02%) ~ 561,256k 561,581k p=0.471 n=6
Parse Time 4.03s (± 0.46%) 4.00s (± 0.26%) -0.02s (- 0.54%) 3.99s 4.02s p=0.033 n=6
Bind Time 1.75s (± 4.39%) 1.70s (± 6.66%) ~ 1.55s 1.81s p=0.520 n=6
Check Time 3.10s (± 2.44%) 3.10s (± 3.87%) ~ 2.99s 3.27s p=0.295 n=6
Emit Time 0.09s (± 5.53%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.174 n=6
Total Time 8.97s (± 0.22%) 8.90s (± 0.55%) -0.06s (- 0.69%) 8.83s 8.96s p=0.043 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 55224 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 3, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the tarball bundle task on this PR at 6156ade. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 3, 2023 •
edited
Loading

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/156206/artifacts?artifactName=tgz&fileId=A23F0B6FAC0AFB5D85599135F628EAD10B21EAAE0379EC502AC9AB95F5DB736702&fileName=/typescript-5.2.0-insiders.20230803.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.2.0-pr-55224-37".;

@jakebailey jakebailey mentioned this pull request Aug 3, 2023
Open
@jakebailey jakebailey merged commit 607d96f into microsoft:main Aug 3, 2023
@jakebailey jakebailey deleted the fix-54939 branch August 3, 2023 22:53
@jakebailey jakebailey mentioned this pull request Aug 3, 2023
Closed
@jakebailey jakebailey mentioned this pull request Mar 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DanielRosenwasser DanielRosenwasser DanielRosenwasser approved these changes

@weswigham weswigham Awaiting requested review from weswigham

@ahejlsberg ahejlsberg Awaiting requested review from ahejlsberg

Assignees

@jakebailey jakebailey

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jakebailey @typescript-bot @ahejlsberg @DanielRosenwasser