Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Convert debug namespace into a module, direct import #51455

Draft
wants to merge 42 commits into
base: main
Choose a base branch
from
Draft

Convert debug namespace into a module, direct import #51455

wants to merge 42 commits into from

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented Nov 8, 2022 •
edited

For #51441.

This is best viewed without whitespace: https://github.com/microsoft/TypeScript/pull/51455/files?w=1

All of the real changes are in src/compiler/debug.ts.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Nov 8, 2022
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 8, 2022 •
edited

Heya @jakebailey, I've started to run the perf test suite on this PR at efa0cae. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 341,173k (± 0.02%) 341,127k (± 0.03%) -46k (- 0.01%) 340,894k 341,243k
Parse Time 1.56s (± 0.65%) 1.55s (± 0.54%) -0.01s (- 0.45%) 1.53s 1.57s
Bind Time 0.54s (± 0.92%) 0.53s (± 1.10%) -0.01s (- 1.68%) 0.52s 0.54s
Check Time 3.99s (± 0.59%) 4.00s (± 0.40%) +0.01s (+ 0.18%) 3.96s 4.03s
Emit Time 4.26s (± 1.20%) 4.27s (± 0.88%) +0.01s (+ 0.26%) 4.20s 4.34s
Total Time 10.34s (± 0.47%) 10.35s (± 0.44%) +0.00s (+ 0.04%) 10.25s 10.45s
Compiler-Unions - node (v18.10.0, x64)
Memory used 190,491k (± 0.03%) 188,792k (± 1.01%) -1,699k (- 0.89%) 184,707k 190,570k
Parse Time 0.61s (± 0.54%) 0.61s (± 0.97%) +0.00s (+ 0.66%) 0.60s 0.63s
Bind Time 0.33s (± 1.04%) 0.33s (± 1.14%) -0.00s (- 0.61%) 0.32s 0.33s
Check Time 4.95s (± 0.81%) 4.96s (± 0.68%) +0.00s (+ 0.04%) 4.88s 5.04s
Emit Time 1.53s (± 0.87%) 1.51s (± 0.98%) -0.02s (- 1.18%) 1.47s 1.54s
Total Time 7.42s (± 0.59%) 7.41s (± 0.33%) -0.02s (- 0.22%) 7.35s 7.46s
Monaco - node (v18.10.0, x64)
Memory used 320,548k (± 0.01%) 320,467k (± 0.02%) -81k (- 0.03%) 320,338k 320,589k
Parse Time 1.15s (± 0.87%) 1.15s (± 0.99%) +0.00s (+ 0.17%) 1.13s 1.18s
Bind Time 0.49s (± 1.63%) 0.48s (± 0.93%) -0.01s (- 2.85%) 0.47s 0.49s
Check Time 3.83s (± 0.85%) 3.83s (± 0.82%) -0.00s (- 0.05%) 3.76s 3.92s
Emit Time 2.24s (± 0.93%) 2.26s (± 0.91%) +0.02s (+ 0.76%) 2.23s 2.32s
Total Time 7.72s (± 0.67%) 7.72s (± 0.68%) +0.00s (+ 0.05%) 7.64s 7.90s
TFS - node (v18.10.0, x64)
Memory used 284,078k (± 0.24%) 283,417k (± 0.22%) -661k (- 0.23%) 282,823k 284,762k
Parse Time 0.96s (± 1.22%) 0.94s (± 1.23%) -0.02s (- 1.66%) 0.92s 0.97s
Bind Time 0.45s (± 6.05%) 0.43s (± 3.52%) 🟩-0.02s (- 4.42%) 0.42s 0.49s
Check Time 3.78s (± 0.81%) 3.76s (± 0.43%) -0.02s (- 0.50%) 3.73s 3.80s
Emit Time 2.18s (± 0.91%) 2.19s (± 0.79%) +0.01s (+ 0.51%) 2.14s 2.22s
Total Time 7.37s (± 0.41%) 7.32s (± 0.38%) -0.04s (- 0.60%) 7.26s 7.39s
material-ui - node (v18.10.0, x64)
Memory used 435,970k (± 0.01%) 435,912k (± 0.01%) -58k (- 0.01%) 435,865k 436,005k
Parse Time 1.34s (± 0.80%) 1.32s (± 0.47%) -0.02s (- 1.49%) 1.30s 1.33s
Bind Time 0.49s (± 1.13%) 0.48s (± 2.40%) -0.01s (- 1.63%) 0.44s 0.50s
Check Time 10.34s (± 0.86%) 10.22s (± 0.78%) -0.12s (- 1.18%) 10.09s 10.45s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 12.18s (± 0.76%) 12.03s (± 0.66%) -0.15s (- 1.23%) 11.91s 12.26s
xstate - node (v18.10.0, x64)
Memory used 518,691k (± 0.01%) 518,600k (± 0.01%) -91k (- 0.02%) 518,493k 518,790k
Parse Time 1.93s (± 0.58%) 1.91s (± 0.64%) -0.02s (- 0.83%) 1.89s 1.93s
Bind Time 0.77s (± 3.49%) 0.76s (± 3.24%) -0.01s (- 1.81%) 0.71s 0.81s
Check Time 1.03s (± 0.60%) 1.04s (± 0.84%) +0.01s (+ 0.87%) 1.02s 1.06s
Emit Time 0.05s (± 0.00%) 0.05s (± 0.00%) 0.00s ( 0.00%) 0.05s 0.05s
Total Time 3.78s (± 0.86%) 3.75s (± 0.54%) -0.03s (- 0.79%) 3.71s 3.80s
Angular - node (v16.17.1, x64)
Memory used 340,557k (± 0.02%) 340,466k (± 0.01%) -92k (- 0.03%) 340,293k 340,526k
Parse Time 1.90s (± 0.40%) 1.87s (± 0.39%) -0.03s (- 1.48%) 1.85s 1.88s
Bind Time 0.65s (± 0.68%) 0.65s (± 0.76%) -0.01s (- 0.92%) 0.64s 0.66s
Check Time 5.15s (± 0.47%) 5.15s (± 0.49%) -0.00s (- 0.10%) 5.10s 5.21s
Emit Time 5.15s (± 0.83%) 5.09s (± 0.60%) -0.05s (- 1.05%) 5.05s 5.17s
Total Time 12.85s (± 0.51%) 12.76s (± 0.37%) -0.09s (- 0.72%) 12.68s 12.88s
Compiler-Unions - node (v16.17.1, x64)
Memory used 187,281k (± 0.53%) 188,229k (± 0.66%) +949k (+ 0.51%) 186,523k 190,076k
Parse Time 0.79s (± 0.86%) 0.78s (± 0.95%) -0.01s (- 1.39%) 0.77s 0.80s
Bind Time 0.42s (± 0.00%) 0.42s (± 0.53%) +0.00s (+ 0.24%) 0.42s 0.43s
Check Time 6.03s (± 0.51%) 5.99s (± 0.92%) -0.04s (- 0.66%) 5.92s 6.13s
Emit Time 1.90s (± 0.59%) 1.88s (± 1.00%) -0.02s (- 0.95%) 1.83s 1.92s
Total Time 9.14s (± 0.37%) 9.07s (± 0.76%) -0.07s (- 0.74%) 8.95s 9.25s
Monaco - node (v16.17.1, x64)
Memory used 319,834k (± 0.01%) 319,749k (± 0.01%) -85k (- 0.03%) 319,663k 319,844k
Parse Time 1.43s (± 0.54%) 1.41s (± 0.76%) -0.02s (- 1.54%) 1.39s 1.44s
Bind Time 0.59s (± 0.57%) 0.59s (± 0.88%) -0.00s (- 0.67%) 0.58s 0.60s
Check Time 4.89s (± 0.28%) 4.86s (± 0.52%) -0.03s (- 0.59%) 4.77s 4.89s
Emit Time 2.74s (± 0.95%) 2.74s (± 0.73%) -0.00s (- 0.07%) 2.69s 2.78s
Total Time 9.65s (± 0.32%) 9.59s (± 0.51%) -0.06s (- 0.60%) 9.44s 9.69s
TFS - node (v16.17.1, x64)
Memory used 282,289k (± 0.01%) 282,208k (± 0.01%) -81k (- 0.03%) 282,176k 282,266k
Parse Time 1.18s (± 0.75%) 1.15s (± 0.58%) -0.02s (- 2.04%) 1.14s 1.17s
Bind Time 0.65s (± 3.64%) 0.64s (± 4.33%) -0.01s (- 1.08%) 0.59s 0.68s
Check Time 4.75s (± 0.46%) 4.75s (± 0.45%) -0.01s (- 0.15%) 4.72s 4.81s
Emit Time 2.75s (± 1.84%) 2.74s (± 1.66%) -0.01s (- 0.29%) 2.67s 2.83s
Total Time 9.32s (± 0.52%) 9.28s (± 0.58%) -0.04s (- 0.48%) 9.17s 9.42s
material-ui - node (v16.17.1, x64)
Memory used 435,297k (± 0.00%) 435,253k (± 0.00%) -44k (- 0.01%) 435,223k 435,305k
Parse Time 1.65s (± 0.36%) 1.64s (± 0.54%) -0.01s (- 0.55%) 1.62s 1.66s
Bind Time 0.50s (± 1.11%) 0.49s (± 1.20%) -0.01s (- 1.59%) 0.48s 0.51s
Check Time 11.88s (± 0.76%) 11.85s (± 1.32%) -0.03s (- 0.28%) 11.55s 12.27s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.03s (± 0.64%) 13.98s (± 1.17%) -0.05s (- 0.36%) 13.67s 14.42s
xstate - node (v16.17.1, x64)
Memory used 516,277k (± 0.01%) 516,173k (± 0.01%) -103k (- 0.02%) 516,071k 516,393k
Parse Time 2.32s (± 0.43%) 2.29s (± 0.26%) -0.02s (- 0.95%) 2.28s 2.31s
Bind Time 0.84s (± 0.88%) 0.84s (± 1.42%) -0.00s (- 0.24%) 0.82s 0.87s
Check Time 1.36s (± 0.70%) 1.34s (± 1.09%) -0.01s (- 1.11%) 1.32s 1.38s
Emit Time 0.06s (± 0.00%) 0.06s (± 0.00%) 0.00s ( 0.00%) 0.06s 0.06s
Total Time 4.58s (± 0.25%) 4.54s (± 0.34%) -0.04s (- 0.76%) 4.51s 4.58s
Angular - node (v14.15.1, x64)
Memory used 334,045k (± 0.01%) 333,975k (± 0.01%) -70k (- 0.02%) 333,902k 334,035k
Parse Time 2.05s (± 0.53%) 2.04s (± 0.47%) -0.01s (- 0.63%) 2.02s 2.06s
Bind Time 0.70s (± 0.32%) 0.69s (± 0.72%) -0.00s (- 0.57%) 0.69s 0.71s
Check Time 5.48s (± 0.36%) 5.44s (± 0.28%) -0.04s (- 0.73%) 5.42s 5.48s
Emit Time 5.19s (± 0.39%) 5.22s (± 0.57%) +0.02s (+ 0.46%) 5.13s 5.26s
Total Time 13.43s (± 0.30%) 13.40s (± 0.25%) -0.03s (- 0.23%) 13.32s 13.47s
Compiler-Unions - node (v14.15.1, x64)
Memory used 181,903k (± 0.40%) 181,507k (± 0.02%) -396k (- 0.22%) 181,388k 181,560k
Parse Time 0.89s (± 0.33%) 0.89s (± 0.65%) -0.00s (- 0.56%) 0.88s 0.90s
Bind Time 0.46s (± 0.74%) 0.45s (± 0.50%) -0.01s (- 1.75%) 0.44s 0.45s
Check Time 6.32s (± 0.64%) 6.27s (± 0.37%) -0.05s (- 0.78%) 6.22s 6.34s
Emit Time 2.03s (± 1.30%) 2.03s (± 0.75%) -0.00s (- 0.15%) 2.00s 2.06s
Total Time 9.70s (± 0.56%) 9.64s (± 0.35%) -0.06s (- 0.61%) 9.59s 9.75s
Monaco - node (v14.15.1, x64)
Memory used 314,624k (± 0.01%) 314,540k (± 0.01%) -84k (- 0.03%) 314,463k 314,599k
Parse Time 1.57s (± 0.86%) 1.56s (± 0.32%) -0.01s (- 0.83%) 1.54s 1.56s
Bind Time 0.63s (± 0.78%) 0.63s (± 0.53%) -0.00s (- 0.63%) 0.62s 0.64s
Check Time 5.20s (± 0.50%) 5.14s (± 0.51%) -0.05s (- 0.98%) 5.10s 5.21s
Emit Time 2.88s (± 0.53%) 2.89s (± 0.85%) +0.01s (+ 0.45%) 2.82s 2.94s
Total Time 10.27s (± 0.34%) 10.22s (± 0.45%) -0.05s (- 0.53%) 10.11s 10.31s
TFS - node (v14.15.1, x64)
Memory used 279,313k (± 0.01%) 279,259k (± 0.00%) -55k (- 0.02%) 279,225k 279,287k
Parse Time 1.33s (± 0.80%) 1.33s (± 0.94%) -0.01s (- 0.60%) 1.31s 1.36s
Bind Time 0.59s (± 0.98%) 0.59s (± 1.13%) -0.00s (- 0.51%) 0.58s 0.61s
Check Time 5.08s (± 0.45%) 5.05s (± 0.44%) -0.03s (- 0.65%) 5.02s 5.12s
Emit Time 3.05s (± 0.79%) 3.04s (± 0.73%) -0.01s (- 0.39%) 3.00s 3.11s
Total Time 10.06s (± 0.21%) 10.01s (± 0.37%) -0.06s (- 0.57%) 9.93s 10.10s
material-ui - node (v14.15.1, x64)
Memory used 430,731k (± 0.01%) 430,659k (± 0.00%) -72k (- 0.02%) 430,603k 430,711k
Parse Time 1.88s (± 0.67%) 1.86s (± 0.39%) -0.02s (- 0.90%) 1.85s 1.88s
Bind Time 0.53s (± 0.56%) 0.53s (± 0.42%) -0.00s (- 0.19%) 0.53s 0.54s
Check Time 12.27s (± 0.73%) 12.18s (± 0.61%) -0.10s (- 0.77%) 12.03s 12.35s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.68s (± 0.66%) 14.57s (± 0.52%) -0.11s (- 0.77%) 14.41s 14.74s
xstate - node (v14.15.1, x64)
Memory used 504,416k (± 0.00%) 504,407k (± 0.01%) -9k (- 0.00%) 504,350k 504,453k
Parse Time 2.63s (± 0.50%) 2.59s (± 0.67%) -0.04s (- 1.37%) 2.57s 2.64s
Bind Time 0.84s (± 0.88%) 0.84s (± 0.40%) +0.00s (+ 0.36%) 0.84s 0.85s
Check Time 1.47s (± 0.56%) 1.46s (± 0.55%) -0.02s (- 1.09%) 1.44s 1.47s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.02s (± 0.22%) 4.96s (± 0.38%) -0.05s (- 1.06%) 4.94s 5.02s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 10
Baseline main 10

TSServer

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,050ms (± 1.11%) 1,035ms (± 0.78%) -15ms (- 1.45%) 1,016ms 1,051ms
Req 2 - geterr 2,534ms (± 0.67%) 2,531ms (± 0.85%) -3ms (- 0.13%) 2,499ms 2,597ms
Req 3 - references 166ms (± 0.35%) 165ms (± 0.72%) -1ms (- 0.54%) 162ms 168ms
Req 4 - navto 138ms (± 1.03%) 137ms (± 0.95%) -1ms (- 0.36%) 134ms 140ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 61ms (± 1.92%) 58ms (± 2.83%) 🟩-3ms (- 4.60%) 55ms 61ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,097ms (± 0.53%) 1,091ms (± 0.49%) -6ms (- 0.51%) 1,078ms 1,100ms
Req 2 - geterr 1,559ms (± 1.11%) 1,559ms (± 0.55%) +1ms (+ 0.04%) 1,542ms 1,578ms
Req 3 - references 173ms (± 4.12%) 171ms (± 3.65%) -2ms (- 1.10%) 167ms 196ms
Req 4 - navto 151ms (± 0.70%) 150ms (± 0.64%) -1ms (- 0.73%) 148ms 152ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 51ms (± 2.96%) 53ms (± 2.69%) +1ms (+ 2.73%) 48ms 55ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,523ms (± 0.50%) 1,498ms (± 0.45%) -25ms (- 1.65%) 1,488ms 1,520ms
Req 2 - geterr 557ms (± 0.79%) 553ms (± 0.94%) -4ms (- 0.63%) 542ms 565ms
Req 3 - references 59ms (± 2.14%) 58ms (± 2.59%) -1ms (- 1.36%) 57ms 64ms
Req 4 - navto 197ms (± 0.62%) 195ms (± 0.62%) -3ms (- 1.27%) 192ms 197ms
Req 5 - completionInfo count 3,149 (± 0.00%) 3,149 (± 0.00%) 0 ( 0.00%) 3,149 3,149
Req 5 - completionInfo 211ms (± 0.83%) 209ms (± 1.36%) -2ms (- 0.81%) 204ms 217ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,301ms (± 0.78%) 1,286ms (± 0.38%) -15ms (- 1.15%) 1,277ms 1,299ms
Req 2 - geterr 3,161ms (± 0.62%) 3,127ms (± 1.19%) -34ms (- 1.07%) 3,061ms 3,212ms
Req 3 - references 192ms (± 1.11%) 191ms (± 0.90%) -1ms (- 0.26%) 189ms 195ms
Req 4 - navto 151ms (± 0.72%) 151ms (± 0.72%) 0ms ( 0.00%) 149ms 154ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 59ms (± 2.74%) 61ms (±16.72%) +1ms (+ 2.36%) 54ms 101ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,392ms (± 1.10%) 1,376ms (± 0.56%) -16ms (- 1.18%) 1,354ms 1,391ms
Req 2 - geterr 2,073ms (± 0.58%) 2,041ms (± 0.65%) -31ms (- 1.51%) 2,017ms 2,071ms
Req 3 - references 200ms (± 0.50%) 198ms (± 0.99%) -2ms (- 1.15%) 196ms 205ms
Req 4 - navto 166ms (± 0.92%) 163ms (± 0.76%) -3ms (- 1.63%) 161ms 166ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 57ms (± 2.65%) 58ms (± 2.68%) +2ms (+ 3.19%) 55ms 61ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,831ms (± 0.48%) 1,804ms (± 0.57%) -27ms (- 1.47%) 1,787ms 1,833ms
Req 2 - geterr 713ms (± 0.70%) 708ms (± 0.73%) -5ms (- 0.71%) 699ms 721ms
Req 3 - references 67ms (± 0.50%) 68ms (± 1.13%) +1ms (+ 0.75%) 66ms 69ms
Req 4 - navto 198ms (± 0.85%) 197ms (± 1.06%) -1ms (- 0.61%) 193ms 201ms
Req 5 - completionInfo count 3,149 (± 0.00%) 3,149 (± 0.00%) 0 ( 0.00%) 3,149 3,149
Req 5 - completionInfo 255ms (± 1.13%) 249ms (± 0.78%) -5ms (- 2.04%) 246ms 253ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,460ms (± 0.60%) 1,440ms (± 0.44%) -20ms (- 1.34%) 1,429ms 1,457ms
Req 2 - geterr 3,401ms (± 0.41%) 3,381ms (± 0.58%) -20ms (- 0.58%) 3,359ms 3,449ms
Req 3 - references 206ms (± 1.06%) 203ms (± 0.40%) -3ms (- 1.50%) 202ms 206ms
Req 4 - navto 161ms (± 0.87%) 161ms (± 0.80%) -0ms (- 0.19%) 158ms 165ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 65ms (± 7.18%) 66ms (± 6.62%) +1ms (+ 2.17%) 57ms 72ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,530ms (± 0.63%) 1,524ms (± 0.61%) -6ms (- 0.39%) 1,504ms 1,543ms
Req 2 - geterr 2,239ms (± 0.42%) 2,239ms (± 0.59%) +0ms (+ 0.01%) 2,206ms 2,261ms
Req 3 - references 215ms (± 1.66%) 210ms (± 1.16%) -5ms (- 2.14%) 205ms 217ms
Req 4 - navto 174ms (± 0.81%) 172ms (± 1.10%) -2ms (- 1.09%) 169ms 176ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 64ms (± 7.93%) 62ms (± 8.69%) 🟩-2ms (- 3.43%) 55ms 73ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,019ms (± 0.30%) 1,999ms (± 0.56%) -20ms (- 1.01%) 1,978ms 2,024ms
Req 2 - geterr 744ms (± 0.59%) 737ms (± 0.42%) -7ms (- 0.87%) 732ms 745ms
Req 3 - references 71ms (± 1.89%) 70ms (± 2.35%) -2ms (- 2.11%) 67ms 75ms
Req 4 - navto 220ms (± 0.70%) 217ms (± 0.52%) -2ms (- 1.09%) 215ms 220ms
Req 5 - completionInfo count 3,149 (± 0.00%) 3,149 (± 0.00%) 0 ( 0.00%) 3,149 3,149
Req 5 - completionInfo 274ms (± 1.93%) 269ms (± 1.57%) -6ms (- 2.08%) 263ms 280ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 10
Baseline main 10

Startup

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 119.24ms (± 0.54%) 117.66ms (± 0.36%) -1.57ms (- 1.32%) 115.75ms 123.67ms
tsserver-startup - node (v16.17.1, x64)
Execution time 199.17ms (± 0.35%) 198.19ms (± 0.31%) -0.98ms (- 0.49%) 195.18ms 205.94ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 187.76ms (± 0.30%) 188.05ms (± 0.31%) +0.29ms (+ 0.16%) 185.36ms 196.33ms
typescript-startup - node (v16.17.1, x64)
Execution time 173.77ms (± 0.32%) 173.75ms (± 0.33%) -0.02ms (- 0.01%) 170.83ms 181.67ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51455 10
Baseline main 10

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

Seems like a couple percent; I'll have to retry this once the benchmarker is stable.

This was referenced Nov 14, 2022
Closed
Merged
@jakebailey jakebailey mentioned this pull request Nov 21, 2022
Merged
11 tasks
@jakebailey jakebailey changed the title [wip] Convert debug namespace into a module Convert debug namespace into a module, direct import Jan 17, 2023
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 17, 2023 •
edited

Heya @jakebailey, I've started to run the perf test suite on this PR at 2b510af. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 358,872k (± 0.01%) 358,780k (± 0.01%) -92k (- 0.03%) 358,750k 358,821k
Parse Time 3.92s (± 0.64%) 3.94s (± 0.87%) +0.02s (+ 0.55%) 3.90s 3.99s
Bind Time 1.19s (± 0.63%) 1.19s (± 0.59%) -0.00s (- 0.14%) 1.18s 1.20s
Check Time 8.82s (± 0.68%) 8.77s (± 0.50%) -0.05s (- 0.60%) 8.71s 8.83s
Emit Time 7.48s (± 0.34%) 7.57s (± 1.08%) +0.09s (+ 1.18%) 7.44s 7.65s
Total Time 21.41s (± 0.37%) 21.47s (± 0.61%) +0.06s (+ 0.28%) 21.30s 21.65s
Compiler-Unions - node (v18.10.0, x64)
Memory used 193,851k (± 1.56%) 194,907k (± 1.62%) +1,055k (+ 0.54%) 191,994k 197,913k
Parse Time 1.66s (± 0.71%) 1.64s (± 1.45%) -0.02s (- 1.30%) 1.59s 1.66s
Bind Time 0.80s (± 0.82%) 0.79s (± 0.64%) -0.01s (- 1.14%) 0.79s 0.80s
Check Time 9.59s (± 0.93%) 9.72s (± 0.74%) +0.13s (+ 1.37%) 9.62s 9.81s
Emit Time 2.94s (± 5.82%) 2.80s (± 0.78%) 🟩-0.14s (- 4.79%) 2.77s 2.83s
Total Time 14.99s (± 1.45%) 14.95s (± 0.52%) -0.04s (- 0.29%) 14.84s 15.07s
Monaco - node (v18.10.0, x64)
Memory used 345,746k (± 0.01%) 345,613k (± 0.01%) -133k (- 0.04%) 345,593k 345,646k
Parse Time 2.93s (± 0.48%) 2.96s (± 1.07%) +0.02s (+ 0.80%) 2.92s 3.00s
Bind Time 1.06s (± 0.92%) 1.06s (± 0.40%) +0.00s (+ 0.30%) 1.06s 1.07s
Check Time 7.30s (± 0.40%) 7.28s (± 0.48%) -0.02s (- 0.26%) 7.25s 7.33s
Emit Time 4.35s (± 1.05%) 4.33s (± 0.67%) -0.02s (- 0.51%) 4.28s 4.36s
Total Time 15.64s (± 0.22%) 15.63s (± 0.55%) -0.01s (- 0.07%) 15.50s 15.74s
TFS - node (v18.10.0, x64)
Memory used 300,532k (± 0.01%) 300,425k (± 0.01%) -107k (- 0.04%) 300,387k 300,452k
Parse Time 2.25s (± 1.08%) 2.26s (± 1.26%) +0.01s (+ 0.53%) 2.22s 2.30s
Bind Time 1.18s (± 0.39%) 1.18s (± 1.45%) -0.00s (- 0.42%) 1.16s 1.21s
Check Time 6.85s (± 0.21%) 6.80s (± 0.36%) -0.06s (- 0.84%) 6.76s 6.82s
Emit Time 3.95s (± 1.21%) 3.92s (± 0.60%) -0.03s (- 0.87%) 3.90s 3.96s
Total Time 14.24s (± 0.27%) 14.16s (± 0.23%) -0.08s (- 0.60%) 14.11s 14.21s
material-ui - node (v18.10.0, x64)
Memory used 476,398k (± 0.01%) 476,331k (± 0.02%) -68k (- 0.01%) 476,256k 476,467k
Parse Time 3.52s (± 0.72%) 3.53s (± 0.41%) +0.00s (+ 0.07%) 3.52s 3.55s
Bind Time 0.97s (± 1.42%) 0.98s (± 0.79%) +0.01s (+ 1.14%) 0.97s 0.99s
Check Time 17.07s (± 0.70%) 17.08s (± 0.55%) +0.01s (+ 0.07%) 16.95s 17.19s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.56s (± 0.70%) 21.58s (± 0.46%) +0.02s (+ 0.10%) 21.44s 21.69s
xstate - node (v18.10.0, x64)
Memory used 546,282k (± 0.01%) 546,090k (± 0.01%) -192k (- 0.04%) 546,020k 546,142k
Parse Time 4.42s (± 0.81%) 4.40s (± 0.65%) -0.02s (- 0.39%) 4.36s 4.44s
Bind Time 1.71s (± 0.92%) 1.71s (± 1.18%) +0.00s (+ 0.09%) 1.68s 1.73s
Check Time 2.74s (± 0.85%) 2.73s (± 0.42%) -0.01s (- 0.38%) 2.71s 2.74s
Emit Time 0.08s (± 0.21%) 0.08s (± 4.96%) +0.00s (+ 1.88%) 0.08s 0.09s
Total Time 8.94s (± 0.49%) 8.92s (± 0.49%) -0.01s (- 0.16%) 8.85s 8.97s
Angular - node (v16.17.1, x64)
Memory used 358,285k (± 0.01%) 358,169k (± 0.01%) -116k (- 0.03%) 358,116k 358,218k
Parse Time 4.11s (± 0.20%) 4.08s (± 0.51%) -0.02s (- 0.59%) 4.06s 4.11s
Bind Time 1.26s (± 0.61%) 1.24s (± 0.69%) -0.01s (- 1.06%) 1.23s 1.25s
Check Time 9.48s (± 0.34%) 9.42s (± 0.44%) -0.06s (- 0.60%) 9.37s 9.48s
Emit Time 7.94s (± 0.53%) 7.93s (± 0.63%) -0.01s (- 0.14%) 7.89s 8.02s
Total Time 22.78s (± 0.28%) 22.67s (± 0.40%) -0.11s (- 0.47%) 22.61s 22.85s
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,832k (± 0.04%) 193,749k (± 0.03%) -84k (- 0.04%) 193,676k 193,845k
Parse Time 1.81s (± 1.13%) 1.80s (± 0.69%) -0.01s (- 0.53%) 1.78s 1.81s
Bind Time 0.85s (± 0.21%) 0.84s (± 0.76%) -0.01s (- 1.24%) 0.83s 0.85s
Check Time 10.29s (± 0.63%) 10.33s (± 0.87%) +0.04s (+ 0.39%) 10.20s 10.47s
Emit Time 3.09s (± 4.41%) 3.04s (± 0.73%) -0.05s (- 1.57%) 3.02s 3.07s
Total Time 16.03s (± 0.65%) 16.00s (± 0.61%) -0.03s (- 0.17%) 15.88s 16.17s
Monaco - node (v16.17.1, x64)
Memory used 345,098k (± 0.02%) 345,013k (± 0.01%) -84k (- 0.02%) 344,966k 345,036k
Parse Time 3.11s (± 1.24%) 3.14s (± 0.51%) +0.04s (+ 1.22%) 3.11s 3.16s
Bind Time 1.11s (± 1.40%) 1.11s (± 0.08%) -0.00s (- 0.07%) 1.11s 1.11s
Check Time 7.89s (± 0.81%) 7.86s (± 0.42%) -0.02s (- 0.28%) 7.81s 7.91s
Emit Time 4.52s (± 0.90%) 4.53s (± 0.71%) +0.01s (+ 0.23%) 4.48s 4.58s
Total Time 16.63s (± 0.64%) 16.65s (± 0.38%) +0.02s (+ 0.14%) 16.57s 16.73s
TFS - node (v16.17.1, x64)
Memory used 299,909k (± 0.01%) 299,773k (± 0.01%) -135k (- 0.05%) 299,739k 299,791k
Parse Time 2.44s (± 0.64%) 2.48s (± 1.20%) +0.04s (+ 1.70%) 2.44s 2.52s
Bind Time 1.26s (± 1.17%) 1.26s (± 0.64%) +0.00s (+ 0.00%) 1.24s 1.27s
Check Time 7.42s (± 0.43%) 7.39s (± 0.34%) -0.03s (- 0.47%) 7.36s 7.41s
Emit Time 4.22s (± 0.44%) 4.33s (± 1.85%) +0.11s (+ 2.56%) 4.20s 4.39s
Total Time 15.34s (± 0.35%) 15.45s (± 0.45%) +0.11s (+ 0.75%) 15.37s 15.56s
material-ui - node (v16.17.1, x64)
Memory used 475,701k (± 0.01%) 475,593k (± 0.00%) -109k (- 0.02%) 475,576k 475,605k
Parse Time 3.66s (± 0.34%) 3.63s (± 0.23%) -0.04s (- 0.96%) 3.62s 3.64s
Bind Time 1.01s (± 0.44%) 1.01s (± 0.51%) -0.00s (- 0.26%) 1.01s 1.02s
Check Time 18.03s (± 1.39%) 17.97s (± 0.44%) -0.06s (- 0.34%) 17.83s 18.04s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 22.70s (± 1.17%) 22.61s (± 0.36%) -0.09s (- 0.41%) 22.46s 22.68s
xstate - node (v16.17.1, x64)
Memory used 543,947k (± 0.03%) 543,736k (± 0.02%) -211k (- 0.04%) 543,625k 543,853k
Parse Time 4.57s (± 0.26%) 4.55s (± 0.40%) -0.03s (- 0.62%) 4.52s 4.57s
Bind Time 1.79s (± 0.56%) 1.79s (± 0.67%) +0.00s (+ 0.09%) 1.78s 1.81s
Check Time 2.93s (± 0.56%) 2.94s (± 0.40%) +0.01s (+ 0.23%) 2.92s 2.95s
Emit Time 0.08s (± 0.17%) 0.08s (± 4.84%) +0.00s (+ 2.08%) 0.08s 0.09s
Total Time 9.39s (± 0.29%) 9.37s (± 0.26%) -0.02s (- 0.21%) 9.33s 9.41s
Angular - node (v14.15.1, x64)
Memory used 351,891k (± 0.01%) 351,840k (± 0.00%) -52k (- 0.01%) 351,828k 351,858k
Parse Time 4.14s (± 0.40%) 4.15s (± 0.65%) +0.01s (+ 0.32%) 4.12s 4.18s
Bind Time 1.28s (± 0.82%) 1.28s (± 0.69%) -0.00s (- 0.04%) 1.27s 1.30s
Check Time 9.78s (± 0.66%) 9.77s (± 0.49%) -0.01s (- 0.06%) 9.72s 9.86s
Emit Time 8.27s (± 0.55%) 8.35s (± 0.76%) +0.08s (+ 0.96%) 8.26s 8.42s
Total Time 23.48s (± 0.41%) 23.56s (± 0.53%) +0.08s (+ 0.35%) 23.38s 23.72s
Compiler-Unions - node (v14.15.1, x64)
Memory used 188,989k (± 0.01%) 189,415k (± 0.73%) +426k (+ 0.23%) 188,839k 192,241k
Parse Time 1.83s (± 0.27%) 1.83s (± 0.84%) +0.00s (+ 0.27%) 1.81s 1.86s
Bind Time 0.88s (± 0.47%) 0.86s (± 0.51%) -0.02s (- 1.90%) 0.86s 0.87s
Check Time 10.32s (± 0.27%) 10.31s (± 0.49%) -0.01s (- 0.13%) 10.22s 10.36s
Emit Time 3.14s (± 1.18%) 3.26s (± 0.84%) +0.12s (+ 3.97%) 3.24s 3.31s
Total Time 16.16s (± 0.30%) 16.26s (± 0.29%) +0.10s (+ 0.62%) 16.19s 16.33s
Monaco - node (v14.15.1, x64)
Memory used 339,910k (± 0.01%) 339,857k (± 0.01%) -53k (- 0.02%) 339,834k 339,890k
Parse Time 3.25s (± 0.93%) 3.23s (± 0.87%) -0.02s (- 0.53%) 3.19s 3.26s
Bind Time 1.14s (± 1.08%) 1.14s (± 0.30%) -0.01s (- 0.55%) 1.13s 1.14s
Check Time 8.20s (± 0.48%) 8.15s (± 0.44%) -0.04s (- 0.54%) 8.13s 8.22s
Emit Time 4.75s (± 0.80%) 4.79s (± 0.54%) +0.03s (+ 0.73%) 4.75s 4.82s
Total Time 17.34s (± 0.35%) 17.31s (± 0.22%) -0.03s (- 0.17%) 17.25s 17.36s
TFS - node (v14.15.1, x64)
Memory used 294,805k (± 0.00%) 294,826k (± 0.00%) +21k (+ 0.01%) 294,814k 294,837k
Parse Time 2.72s (± 0.39%) 2.76s (± 0.86%) +0.04s (+ 1.42%) 2.72s 2.78s
Bind Time 1.11s (± 0.38%) 1.11s (± 1.94%) +0.00s (+ 0.44%) 1.08s 1.15s
Check Time 7.74s (± 0.51%) 7.73s (± 0.56%) -0.01s (- 0.19%) 7.68s 7.79s
Emit Time 4.59s (± 0.75%) 4.65s (± 0.77%) +0.06s (+ 1.32%) 4.61s 4.70s
Total Time 16.15s (± 0.22%) 16.24s (± 0.60%) +0.09s (+ 0.54%) 16.13s 16.38s
material-ui - node (v14.15.1, x64)
Memory used 471,164k (± 0.00%) 471,073k (± 0.00%) -91k (- 0.02%) 471,052k 471,086k
Parse Time 3.86s (± 0.72%) 3.86s (± 0.42%) +0.00s (+ 0.07%) 3.84s 3.88s
Bind Time 1.03s (± 0.88%) 1.04s (± 0.71%) +0.00s (+ 0.39%) 1.03s 1.05s
Check Time 18.90s (± 0.49%) 18.87s (± 0.34%) -0.02s (- 0.11%) 18.76s 18.93s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 23.79s (± 0.44%) 23.77s (± 0.27%) -0.01s (- 0.06%) 23.65s 23.82s
xstate - node (v14.15.1, x64)
Memory used 532,149k (± 0.01%) 531,920k (± 0.00%) -229k (- 0.04%) 531,892k 531,961k
Parse Time 5.00s (± 0.76%) 4.97s (± 0.96%) -0.03s (- 0.60%) 4.93s 5.03s
Bind Time 1.66s (± 1.16%) 1.67s (± 0.70%) +0.01s (+ 0.60%) 1.66s 1.69s
Check Time 3.07s (± 0.33%) 3.08s (± 0.49%) +0.01s (+ 0.30%) 3.06s 3.09s
Emit Time 0.10s (± 0.27%) 0.09s (± 0.13%) 🟩-0.01s (- 9.73%) 0.09s 0.09s
Total Time 9.84s (± 0.52%) 9.81s (± 0.47%) -0.03s (- 0.31%) 9.76s 9.89s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

TSServer

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,458ms (± 6.25%) 2,522ms (± 1.12%) +64ms (+ 2.60%) 2,479ms 2,562ms
Req 2 - geterr 5,176ms (± 6.65%) 5,418ms (± 1.08%) +242ms (+ 4.67%) 5,341ms 5,489ms
Req 3 - references 359ms (±10.35%) 377ms (± 2.65%) +18ms (+ 4.96%) 365ms 394ms
Req 4 - navto 275ms (± 6.42%) 288ms (± 1.50%) +13ms (+ 4.71%) 281ms 292ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 67ms (±10.52%) 102ms (±87.84%) 🔻+35ms (+51.83%) 62ms 283ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,635ms (± 0.90%) 2,662ms (± 0.59%) +27ms (+ 1.02%) 2,645ms 2,682ms
Req 2 - geterr 3,965ms (± 0.45%) 3,992ms (± 0.54%) +27ms (+ 0.69%) 3,959ms 4,027ms
Req 3 - references 375ms (± 1.05%) 376ms (± 1.58%) +0ms (+ 0.02%) 367ms 383ms
Req 4 - navto 289ms (± 0.61%) 291ms (± 0.65%) +2ms (+ 0.69%) 288ms 293ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 82ms (± 2.48%) 82ms (± 1.18%) +0ms (+ 0.23%) 80ms 83ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,347ms (± 1.03%) 3,328ms (± 0.36%) -19ms (- 0.57%) 3,316ms 3,350ms
Req 2 - geterr 1,408ms (± 1.38%) 1,394ms (± 0.88%) -13ms (- 0.96%) 1,381ms 1,416ms
Req 3 - references 245ms (± 0.47%) 245ms (± 0.76%) +0ms (+ 0.12%) 242ms 247ms
Req 4 - navto 360ms (± 0.90%) 360ms (± 1.19%) +0ms (+ 0.10%) 356ms 366ms
Req 5 - completionInfo count 3,159 (± 0.00%) 3,159 (± 0.00%) 0 ( 0.00%) 3,159 3,159
Req 5 - completionInfo 445ms (± 0.69%) 429ms (± 1.55%) 🟩-16ms (- 3.55%) 417ms 437ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,684ms (± 5.38%) 2,733ms (± 0.66%) +49ms (+ 1.83%) 2,715ms 2,758ms
Req 2 - geterr 5,716ms (± 5.98%) 5,779ms (± 0.55%) +63ms (+ 1.10%) 5,737ms 5,829ms
Req 3 - references 400ms (±20.75%) 376ms (± 0.26%) 🟩-24ms (- 5.90%) 375ms 377ms
Req 4 - navto 269ms (± 5.61%) 271ms (± 0.50%) +2ms (+ 0.82%) 269ms 273ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 221ms (±49.68%) 295ms (± 2.17%) 🔻+74ms (+33.26%) 287ms 302ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,874ms (± 0.75%) 2,661ms (±22.01%) 🟩-213ms (- 7.41%) 1,465ms 2,942ms
Req 2 - geterr 4,319ms (± 0.54%) 3,948ms (±21.98%) 🟩-372ms (- 8.61%) 2,175ms 4,341ms
Req 3 - references 386ms (± 1.49%) 356ms (±21.76%) 🟩-30ms (- 7.77%) 198ms 392ms
Req 4 - navto 299ms (± 0.32%) 274ms (±21.79%) 🟩-25ms (- 8.42%) 152ms 301ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 79ms (± 0.65%) 73ms (±21.79%) 🟩-7ms (- 8.27%) 40ms 81ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,461ms (± 0.30%) 3,449ms (± 0.60%) -12ms (- 0.36%) 3,417ms 3,472ms
Req 2 - geterr 1,656ms (± 5.00%) 1,648ms (± 3.90%) -7ms (- 0.45%) 1,541ms 1,709ms
Req 3 - references 167ms (±51.40%) 170ms (±43.93%) +3ms (+ 1.59%) 110ms 282ms
Req 4 - navto 344ms (± 1.85%) 340ms (± 1.74%) -4ms (- 1.11%) 334ms 347ms
Req 5 - completionInfo count 3,159 (± 0.00%) 3,159 (± 0.00%) 0 ( 0.00%) 3,159 3,159
Req 5 - completionInfo 481ms (± 1.26%) 467ms (± 1.52%) -14ms (- 2.99%) 456ms 475ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,863ms (± 0.19%) 2,858ms (± 0.55%) -5ms (- 0.19%) 2,840ms 2,878ms
Req 2 - geterr 6,172ms (± 0.41%) 6,227ms (± 0.70%) +55ms (+ 0.89%) 6,164ms 6,282ms
Req 3 - references 389ms (± 1.05%) 387ms (± 0.97%) -2ms (- 0.42%) 384ms 393ms
Req 4 - navto 285ms (± 0.99%) 289ms (± 0.94%) +3ms (+ 1.14%) 285ms 291ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 82ms (± 9.27%) 87ms (± 6.86%) +4ms (+ 5.43%) 76ms 91ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,005ms (± 0.64%) 3,006ms (± 0.53%) +2ms (+ 0.06%) 2,980ms 3,025ms
Req 2 - geterr 4,618ms (± 0.81%) 4,581ms (± 0.87%) -37ms (- 0.79%) 4,523ms 4,620ms
Req 3 - references 412ms (± 0.67%) 411ms (± 1.13%) -1ms (- 0.19%) 407ms 419ms
Req 4 - navto 309ms (± 0.79%) 304ms (± 1.68%) -4ms (- 1.42%) 295ms 310ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 79ms (± 1.72%) 81ms (± 5.62%) +2ms (+ 2.13%) 77ms 89ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,797ms (± 0.68%) 3,797ms (± 0.33%) +0ms (+ 0.01%) 3,775ms 3,806ms
Req 2 - geterr 1,471ms (± 1.32%) 1,473ms (± 1.34%) +1ms (+ 0.09%) 1,453ms 1,501ms
Req 3 - references 130ms (± 1.82%) 132ms (± 2.60%) +2ms (+ 1.56%) 126ms 136ms
Req 4 - navto 401ms (± 0.81%) 399ms (± 1.35%) -2ms (- 0.39%) 389ms 404ms
Req 5 - completionInfo count 3,159 (± 0.00%) 3,159 (± 0.00%) 0 ( 0.00%) 3,159 3,159
Req 5 - completionInfo 506ms (± 1.18%) 498ms (± 1.51%) -8ms (- 1.51%) 487ms 508ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

Startup

Comparison Report - main..51455
Metric main 51455 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 139.33ms (± 0.24%) 139.02ms (± 0.45%) -0.30ms (- 0.22%) 137.67ms 147.58ms
tsserver-startup - node (v16.17.1, x64)
Execution time 231.25ms (± 0.22%) 233.15ms (± 0.39%) +1.90ms (+ 0.82%) 230.01ms 238.38ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 233.62ms (± 0.19%) 237.38ms (± 0.32%) +3.76ms (+ 1.61%) 233.64ms 243.42ms
typescript-startup - node (v16.17.1, x64)
Execution time 215.22ms (± 0.25%) 219.12ms (± 0.27%) +3.90ms (+ 1.81%) 215.46ms 224.78ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51455 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey jakebailey marked this pull request as ready for review January 17, 2023 22:46
@jakebailey
Copy link
Member Author

jakebailey commented Jan 17, 2023 •
edited

@a-tarasyuk If you're bored, you may want to try rebasing your direct import PR on top of this and see what happens; I think this should fix the runtime errors due to cycles, but, I can't be totally sure.

(I'm not planning on getting this one looked at until after the beta cutoff.)

@a-tarasyuk
Copy link
Contributor

a-tarasyuk commented Jan 18, 2023 •
edited

@jakebailey I've added changes to #51590 , however, it doesn't solve the issue of running tests with the --bundle=false flag.

@jakebailey
Copy link
Member Author

Hm, seems to be just performanceCore; likely some of that can be made lazy as I don't see a major reason why we need to have it available until one of the functions is called.

@a-tarasyuk
Copy link
Contributor

I’ve added a couple of changes and the build seems to have passed. 😱

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 26, 2023 •
edited

Heya @jakebailey, I've started to run the tsc-only perf test suite on this PR at d627097. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 300,241k (± 0.01%) 300,178k (± 0.01%) -64k (- 0.02%) 300,148k 300,208k p=0.005 n=6
Parse Time 3.03s (± 0.29%) 3.01s (± 0.27%) -0.02s (- 0.77%) 3.00s 3.02s p=0.007 n=6
Bind Time 0.93s (± 0.88%) 0.95s (± 0.54%) +0.02s (+ 2.16%) 0.94s 0.95s p=0.003 n=6
Check Time 9.28s (± 0.15%) 9.20s (± 0.20%) -0.08s (- 0.92%) 9.18s 9.23s p=0.004 n=6
Emit Time 7.61s (± 0.24%) 7.59s (± 0.36%) ~ 7.55s 7.62s p=0.105 n=6
Total Time 20.86s (± 0.15%) 20.74s (± 0.16%) -0.12s (- 0.58%) 20.69s 20.78s p=0.005 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 194,488k (± 0.66%) 193,820k (± 0.02%) -668k (- 0.34%) 193,776k 193,896k p=0.005 n=6
Parse Time 1.57s (± 0.78%) 1.57s (± 0.33%) ~ 1.56s 1.57s p=0.051 n=6
Bind Time 0.80s (± 0.65%) 0.79s (± 0.52%) -0.01s (- 1.05%) 0.78s 0.79s p=0.022 n=6
Check Time 9.92s (± 0.27%) 9.92s (± 0.47%) ~ 9.87s 9.97s p=0.747 n=6
Emit Time 2.74s (± 0.23%) 2.76s (± 0.54%) +0.02s (+ 0.85%) 2.75s 2.79s p=0.007 n=6
Total Time 15.03s (± 0.23%) 15.03s (± 0.33%) ~ 14.99s 15.11s p=0.808 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,148k (± 0.01%) 347,035k (± 0.00%) -113k (- 0.03%) 347,023k 347,046k p=0.005 n=6
Parse Time 2.69s (± 0.15%) 2.72s (± 0.40%) +0.03s (+ 1.05%) 2.71s 2.73s p=0.003 n=6
Bind Time 0.99s (± 0.00%) 1.01s (± 0.40%) +0.02s (+ 2.19%) 1.01s 1.02s p=0.002 n=6
Check Time 7.91s (± 0.10%) 7.90s (± 0.12%) -0.01s (- 0.17%) 7.89s 7.91s p=0.044 n=6
Emit Time 4.26s (± 0.46%) 4.29s (± 0.29%) +0.03s (+ 0.74%) 4.28s 4.31s p=0.023 n=6
Total Time 15.85s (± 0.19%) 15.92s (± 0.08%) +0.07s (+ 0.44%) 15.90s 15.93s p=0.005 n=6
TFS - node (v16.17.1, x64)
Memory used 301,155k (± 0.00%) 301,047k (± 0.00%) -108k (- 0.04%) 301,035k 301,073k p=0.005 n=6
Parse Time 2.17s (± 0.61%) 2.17s (± 0.24%) ~ 2.17s 2.18s p=0.802 n=6
Bind Time 1.11s (± 0.46%) 1.11s (± 1.68%) ~ 1.08s 1.13s p=0.667 n=6
Check Time 7.20s (± 0.18%) 7.23s (± 0.18%) +0.03s (+ 0.42%) 7.21s 7.25s p=0.009 n=6
Emit Time 3.98s (± 0.30%) 4.03s (± 0.66%) +0.05s (+ 1.26%) 4.00s 4.08s p=0.006 n=6
Total Time 14.46s (± 0.13%) 14.55s (± 0.15%) +0.08s (+ 0.55%) 14.51s 14.57s p=0.005 n=6
material-ui - node (v16.17.1, x64)
Memory used 479,455k (± 0.00%) 479,465k (± 0.00%) ~ 479,447k 479,473k p=0.054 n=6
Parse Time 3.15s (± 0.24%) 3.22s (± 0.20%) +0.07s (+ 2.28%) 3.21s 3.23s p=0.004 n=6
Bind Time 0.91s (± 0.00%) 0.91s (± 0.00%) ~ 0.91s 0.91s p=1.000 n=6
Check Time 17.81s (± 0.39%) 17.78s (± 0.35%) ~ 17.70s 17.87s p=0.630 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.86s (± 0.33%) 21.92s (± 0.30%) ~ 21.83s 22.01s p=0.229 n=6
xstate - node (v16.17.1, x64)
Memory used 542,868k (± 0.01%) 542,778k (± 0.01%) ~ 542,710k 542,901k p=0.065 n=6
Parse Time 3.71s (± 0.41%) 3.71s (± 0.15%) ~ 3.70s 3.71s p=0.865 n=6
Bind Time 1.40s (± 4.82%) 1.58s (± 0.53%) +0.18s (+13.08%) 1.58s 1.60s p=0.004 n=6
Check Time 3.25s (± 2.61%) 3.15s (± 0.53%) -0.10s (- 2.98%) 3.12s 3.17s p=0.012 n=6
Emit Time 0.08s (± 6.19%) 0.08s (± 6.19%) ~ 0.08s 0.09s p=0.923 n=6
Total Time 8.43s (± 0.28%) 8.53s (± 0.21%) +0.10s (+ 1.23%) 8.50s 8.55s p=0.005 n=6
System info unknown
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

jakebailey
jakebailey commented Aug 30, 2023
View reviewed changes
scripts/eslint/rules/no-ts-debug.cjs Outdated
description: ``,
},
messages: {
importDebugError: `ts.Debug should be imported directly from src/compiler/debug.ts`,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be nice to have an auto fix for this, but I'd have to know where to put the import. Maybe just sticking it on the first line would be sufficient, and the lint can fix it later.

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster
@typescript-bot perf test this bun

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 15, 2023 •
edited

Heya @jakebailey, I've started to run the bun perf test suite on this PR at 2305d87. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 15, 2023 •
edited

Heya @jakebailey, I've started to run the tsc-only perf test suite on this PR at 2305d87. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 294,934k (± 0.01%) 294,876k (± 0.01%) -58k (- 0.02%) 294,824k 294,941k p=0.031 n=6
Parse Time 2.63s (± 0.37%) 2.65s (± 0.39%) +0.02s (+ 0.57%) 2.63s 2.66s p=0.042 n=6
Bind Time 0.84s (± 0.61%) 0.85s (± 0.96%) ~ 0.83s 0.85s p=0.235 n=6
Check Time 8.01s (± 0.15%) 8.00s (± 0.34%) ~ 7.97s 8.05s p=0.287 n=6
Emit Time 7.03s (± 0.24%) 7.08s (± 0.26%) +0.05s (+ 0.78%) 7.07s 7.12s p=0.005 n=6
Total Time 18.51s (± 0.17%) 18.58s (± 0.13%) +0.07s (+ 0.36%) 18.56s 18.62s p=0.005 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,468k (± 1.19%) 192,499k (± 1.52%) ~ 190,576k 196,291k p=0.230 n=6
Parse Time 1.35s (± 0.47%) 1.35s (± 1.09%) ~ 1.33s 1.37s p=0.737 n=6
Bind Time 0.74s (± 0.00%) 0.75s (± 0.84%) +0.01s (+ 1.35%) 0.74s 0.76s p=0.008 n=6
Check Time 9.08s (± 0.66%) 9.08s (± 0.18%) ~ 9.06s 9.10s p=0.686 n=6
Emit Time 2.60s (± 0.67%) 2.61s (± 0.52%) ~ 2.60s 2.63s p=0.363 n=6
Total Time 13.77s (± 0.47%) 13.80s (± 0.18%) ~ 13.77s 13.84s p=0.335 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,158k (± 0.01%) 347,166k (± 0.00%) ~ 347,137k 347,179k p=0.748 n=6
Parse Time 2.47s (± 0.50%) 2.49s (± 0.98%) +0.02s (+ 1.01%) 2.45s 2.52s p=0.038 n=6
Bind Time 0.94s (± 0.43%) 0.95s (± 0.43%) +0.01s (+ 1.07%) 0.94s 0.95s p=0.008 n=6
Check Time 6.86s (± 0.35%) 6.90s (± 0.49%) ~ 6.86s 6.95s p=0.076 n=6
Emit Time 4.05s (± 0.62%) 4.08s (± 0.41%) ~ 4.05s 4.10s p=0.105 n=6
Total Time 14.32s (± 0.13%) 14.42s (± 0.36%) +0.10s (+ 0.68%) 14.36s 14.51s p=0.005 n=6
TFS - node (v18.15.0, x64)
Memory used 302,449k (± 0.00%) 302,369k (± 0.00%) -80k (- 0.03%) 302,356k 302,390k p=0.005 n=6
Parse Time 2.00s (± 1.21%) 2.02s (± 1.27%) ~ 1.99s 2.05s p=0.332 n=6
Bind Time 0.99s (± 0.82%) 1.00s (± 0.41%) ~ 1.00s 1.01s p=0.056 n=6
Check Time 6.23s (± 0.63%) 6.25s (± 0.33%) ~ 6.21s 6.27s p=0.520 n=6
Emit Time 3.53s (± 0.47%) 3.56s (± 0.41%) +0.03s (+ 0.90%) 3.54s 3.58s p=0.019 n=6
Total Time 12.76s (± 0.40%) 12.83s (± 0.28%) +0.08s (+ 0.61%) 12.79s 12.89s p=0.031 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,457k (± 0.00%) 470,462k (± 0.01%) ~ 470,432k 470,494k p=0.936 n=6
Parse Time 2.59s (± 0.51%) 2.61s (± 0.56%) +0.02s (+ 0.77%) 2.60s 2.64s p=0.020 n=6
Bind Time 1.00s (± 0.75%) 1.00s (± 1.03%) ~ 0.99s 1.01s p=0.664 n=6
Check Time 16.61s (± 0.38%) 16.58s (± 0.08%) ~ 16.56s 16.60s p=0.571 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.20s (± 0.25%) 20.20s (± 0.10%) ~ 20.18s 20.23s p=0.935 n=6
xstate - node (v18.15.0, x64)
Memory used 512,178k (± 0.01%) 512,183k (± 0.01%) ~ 512,073k 512,295k p=1.000 n=6
Parse Time 3.26s (± 0.60%) 3.29s (± 0.17%) +0.04s (+ 1.18%) 3.29s 3.30s p=0.004 n=6
Bind Time 1.55s (± 0.33%) 1.55s (± 0.26%) ~ 1.55s 1.56s p=0.114 n=6
Check Time 2.80s (± 0.29%) 2.78s (± 0.59%) ~ 2.76s 2.80s p=0.059 n=6
Emit Time 0.08s (± 6.19%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=0.595 n=6
Total Time 7.68s (± 0.28%) 7.70s (± 0.19%) ~ 7.68s 7.72s p=0.142 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - bun (v1.0.2, x64)
Memory used 316,452k (± 0.16%) 317,285k (± 0.14%) +832k (+ 0.26%) 316,164k 318,012k p=0.023 n=12
Parse Time 2.28s (± 0.31%) 2.28s (± 0.43%) ~ 2.26s 2.32s p=0.743 n=12
Bind Time 0.89s (± 1.60%) 0.89s (± 1.36%) ~ 0.86s 0.93s p=0.430 n=12
Check Time 7.91s (± 0.56%) 7.91s (± 0.24%) ~ 7.86s 7.97s p=0.816 n=12
Emit Time 6.41s (± 0.18%) 6.36s (± 0.40%) -0.04s (- 0.64%) 6.32s 6.44s p=0.012 n=12
Total Time 17.49s (± 0.29%) 17.44s (± 0.17%) ~ 17.39s 17.56s p=0.068 n=12
Compiler-Unions - bun (v1.0.2, x64)
Memory used 245,804k (± 0.80%) 246,224k (± 0.59%) ~ 239,186k 248,506k p=0.544 n=12
Parse Time 1.15s (± 1.98%) 1.14s (± 1.73%) ~ 1.11s 1.21s p=0.703 n=12
Bind Time 0.83s (± 0.75%) 0.83s (± 0.95%) ~ 0.80s 0.84s p=0.088 n=12
Check Time 8.79s (± 0.50%) 8.78s (± 0.31%) ~ 8.70s 8.84s p=0.908 n=12
Emit Time 2.79s (± 0.51%) 2.80s (± 0.50%) ~ 2.75s 2.84s p=0.195 n=12
Total Time 13.56s (± 0.43%) 13.54s (± 0.17%) ~ 13.47s 13.62s p=0.884 n=12
Monaco - bun (v1.0.2, x64)
Memory used 378,521k (± 0.15%) 378,148k (± 0.14%) ~ 377,091k 379,609k p=0.285 n=12
Parse Time 1.96s (± 0.70%) 1.96s (± 0.48%) ~ 1.94s 1.98s p=0.722 n=12
Bind Time 0.97s (± 2.03%) 0.98s (± 1.84%) ~ 0.94s 1.03s p=0.642 n=12
Check Time 7.34s (± 0.32%) 7.36s (± 0.50%) ~ 7.25s 7.45s p=0.172 n=12
Emit Time 3.78s (± 0.27%) 3.74s (± 0.47%) -0.03s (- 0.88%) 3.69s 3.80s p=0.002 n=12
Total Time 14.05s (± 0.21%) 14.05s (± 0.25%) ~ 13.94s 14.13s p=0.908 n=12
TFS - bun (v1.0.2, x64)
Memory used 317,177k (± 0.17%) 316,979k (± 0.17%) ~ 315,866k 318,181k p=0.403 n=12
Parse Time 1.75s (± 0.67%) 1.76s (± 0.43%) +0.01s (+ 0.81%) 1.74s 1.78s p=0.007 n=12
Bind Time 0.98s (± 1.85%) 0.99s (± 2.17%) ~ 0.90s 1.03s p=0.129 n=12
Check Time 6.60s (± 0.32%) 6.57s (± 0.35%) ~ 6.52s 6.62s p=0.063 n=12
Emit Time 3.46s (± 0.59%) 3.46s (± 0.67%) ~ 3.40s 3.51s p=1.000 n=12
Total Time 12.78s (± 0.18%) 12.78s (± 0.27%) ~ 12.69s 12.87s p=0.839 n=12
material-ui - bun (v1.0.2, x64)
Memory used 495,505k (± 2.02%) 481,912k (± 1.18%) ~ 474,301k 509,827k p=0.053 n=12
Parse Time 2.23s (± 0.41%) 2.22s (± 0.41%) ~ 2.20s 2.25s p=0.312 n=12
Bind Time 0.75s (± 2.62%) 0.77s (± 0.72%) ~ 0.76s 0.79s p=0.171 n=12
Check Time 15.75s (± 0.49%) 15.77s (± 0.49%) ~ 15.59s 16.00s p=0.840 n=12
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=12
Total Time 18.73s (± 0.36%) 18.76s (± 0.42%) ~ 18.59s 19.01s p=0.525 n=12
xstate - bun (v1.0.2, x64)
Memory used 402,231k (± 5.49%) 408,685k (± 5.91%) ~ 380,152k 461,329k p=0.931 n=12
Parse Time 3.23s (± 0.23%) 3.22s (± 0.42%) ~ 3.20s 3.27s p=0.262 n=12
Bind Time 1.26s (± 0.75%) 1.25s (± 0.72%) ~ 1.22s 1.27s p=0.403 n=12
Check Time 3.59s (± 0.41%) 3.56s (± 0.36%) -0.03s (- 0.81%) 3.52s 3.60s p=0.005 n=12
Emit Time 0.20s (± 1.40%) 0.20s (± 1.93%) ~ 0.20s 0.22s p=0.744 n=12
Total Time 8.28s (± 0.21%) 8.24s (± 0.22%) -0.04s (- 0.45%) 8.20s 8.28s p=0.014 n=12
System info unknown
Hosts
  • bun (v1.0.2, x64)
Scenarios
  • Angular - bun (v1.0.2, x64)
  • Compiler-Unions - bun (v1.0.2, x64)
  • Monaco - bun (v1.0.2, x64)
  • TFS - bun (v1.0.2, x64)
  • material-ui - bun (v1.0.2, x64)
  • xstate - bun (v1.0.2, x64)
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - bun (v1.0.2, x64)
Execution time 424.23ms (± 0.11%) 422.03ms (± 0.10%) -2.19ms (- 0.52%) 421.14ms 430.65ms p=0.000 n=600
tsserverlibrary-startup - bun (v1.0.2, x64)
Execution time 657.39ms (± 0.10%) 656.79ms (± 0.16%) -0.60ms (- 0.09%) 654.81ms 672.17ms p=0.000 n=600
typescript-startup - bun (v1.0.2, x64)
Execution time 657.76ms (± 0.14%) 657.20ms (± 0.14%) -0.56ms (- 0.09%) 655.51ms 673.29ms p=0.000 n=600
System info unknown
Hosts
  • bun (v1.0.2, x64)
Scenarios
  • tsc-startup - bun (v1.0.2, x64)
  • tsserverlibrary-startup - bun (v1.0.2, x64)
  • typescript-startup - bun (v1.0.2, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey jakebailey mentioned this pull request Sep 27, 2023
Draft
@jakebailey jakebailey marked this pull request as draft January 19, 2024 20:53
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 19, 2024 •
edited

Heya @jakebailey, I've started to run the regular perf test suite on this PR at 042b96c. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,644k (± 0.01%) 295,620k (± 0.01%) ~ 295,592k 295,645k p=0.230 n=6
Parse Time 2.66s (± 0.19%) 2.67s (± 0.33%) ~ 2.66s 2.68s p=0.190 n=6
Bind Time 0.83s (± 0.91%) 0.83s (± 1.40%) ~ 0.82s 0.85s p=0.734 n=6
Check Time 8.18s (± 0.41%) 8.17s (± 0.25%) ~ 8.15s 8.20s p=0.804 n=6
Emit Time 7.10s (± 0.35%) 7.18s (± 0.29%) +0.08s (+ 1.06%) 7.15s 7.21s p=0.005 n=6
Total Time 18.78s (± 0.27%) 18.85s (± 0.13%) +0.07s (+ 0.40%) 18.82s 18.88s p=0.036 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,907k (± 1.49%) 192,418k (± 1.27%) -1,490k (- 0.77%) 191,394k 197,429k p=0.045 n=6
Parse Time 1.36s (± 0.86%) 1.34s (± 0.56%) -0.02s (- 1.47%) 1.33s 1.35s p=0.011 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.57%) ~ 0.72s 0.73s p=0.405 n=6
Check Time 9.38s (± 0.30%) 9.37s (± 0.20%) ~ 9.34s 9.39s p=0.517 n=6
Emit Time 2.61s (± 0.87%) 2.62s (± 0.78%) ~ 2.58s 2.64s p=0.285 n=6
Total Time 14.07s (± 0.37%) 14.05s (± 0.13%) ~ 14.02s 14.07s p=0.196 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,469k (± 0.00%) 347,463k (± 0.00%) ~ 347,449k 347,485k p=0.470 n=6
Parse Time 2.49s (± 0.47%) 2.51s (± 0.33%) +0.03s (+ 1.07%) 2.51s 2.53s p=0.004 n=6
Bind Time 0.93s (± 0.44%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=0.405 n=6
Check Time 6.91s (± 0.54%) 6.93s (± 0.36%) ~ 6.90s 6.96s p=0.334 n=6
Emit Time 4.06s (± 0.29%) 4.10s (± 0.34%) +0.04s (+ 1.03%) 4.08s 4.12s p=0.006 n=6
Total Time 14.38s (± 0.22%) 14.48s (± 0.24%) +0.09s (+ 0.65%) 14.42s 14.51s p=0.005 n=6
TFS - node (v18.15.0, x64)
Memory used 302,837k (± 0.01%) 302,758k (± 0.01%) -79k (- 0.03%) 302,732k 302,812k p=0.008 n=6
Parse Time 2.01s (± 0.73%) 2.02s (± 1.03%) ~ 1.99s 2.05s p=1.000 n=6
Bind Time 1.00s (± 1.22%) 1.01s (± 0.51%) ~ 1.00s 1.01s p=0.672 n=6
Check Time 6.33s (± 0.51%) 6.39s (± 0.31%) +0.05s (+ 0.82%) 6.36s 6.40s p=0.012 n=6
Emit Time 3.59s (± 0.23%) 3.60s (± 0.68%) ~ 3.56s 3.63s p=0.452 n=6
Total Time 12.95s (± 0.29%) 13.01s (± 0.23%) +0.06s (+ 0.48%) 12.95s 13.03s p=0.020 n=6
material-ui - node (v18.15.0, x64)
Memory used 511,309k (± 0.00%) 511,284k (± 0.01%) ~ 511,252k 511,329k p=0.128 n=6
Parse Time 2.64s (± 0.65%) 2.72s (± 0.63%) +0.08s (+ 2.90%) 2.69s 2.74s p=0.005 n=6
Bind Time 1.00s (± 0.75%) 1.00s (± 0.75%) ~ 0.99s 1.01s p=1.000 n=6
Check Time 17.21s (± 0.41%) 17.25s (± 0.48%) ~ 17.12s 17.33s p=0.630 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.86s (± 0.30%) 20.96s (± 0.44%) ~ 20.83s 21.06s p=0.078 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,695,968k (± 0.00%) 1,695,935k (± 0.00%) ~ 1,695,915k 1,695,960k p=0.065 n=6
Parse Time 6.54s (± 0.16%) 6.59s (± 0.47%) +0.06s (+ 0.89%) 6.56s 6.64s p=0.005 n=6
Bind Time 2.28s (± 6.54%) 2.35s (± 0.70%) ~ 2.33s 2.38s p=0.113 n=6
Check Time 55.53s (± 0.39%) 55.36s (± 0.34%) ~ 55.15s 55.66s p=0.229 n=6
Emit Time 0.16s (± 0.00%) 0.16s (± 0.00%) ~ 0.16s 0.16s p=1.000 n=6
Total Time 64.51s (± 0.47%) 64.47s (± 0.31%) ~ 64.29s 64.79s p=0.575 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,412,834k (± 0.02%) 2,414,087k (± 0.03%) +1,253k (+ 0.05%) 2,413,086k 2,415,371k p=0.020 n=6
Parse Time 4.95s (± 0.71%) 4.94s (± 0.71%) ~ 4.89s 4.98s p=0.810 n=6
Bind Time 1.88s (± 0.73%) 1.88s (± 0.55%) ~ 1.87s 1.90s p=0.391 n=6
Check Time 33.33s (± 0.39%) 33.30s (± 0.28%) ~ 33.14s 33.41s p=0.748 n=6
Emit Time 2.73s (± 1.59%) 2.80s (± 1.65%) +0.08s (+ 2.87%) 2.73s 2.87s p=0.020 n=6
Total Time 42.91s (± 0.21%) 42.93s (± 0.36%) ~ 42.70s 43.16s p=0.810 n=6
self-compiler - node (v18.15.0, x64)
Memory used 419,666k (± 0.01%) 419,369k (± 0.01%) -297k (- 0.07%) 419,326k 419,414k p=0.005 n=6
Parse Time 2.76s (± 2.87%) 2.77s (± 2.55%) ~ 2.67s 2.84s p=0.747 n=6
Bind Time 1.12s (± 6.82%) 1.11s (± 6.76%) ~ 1.06s 1.21s p=0.139 n=6
Check Time 15.10s (± 0.23%) 15.03s (± 0.31%) -0.07s (- 0.49%) 14.96s 15.07s p=0.020 n=6
Emit Time 1.16s (± 0.54%) 1.14s (± 0.66%) -0.02s (- 1.58%) 1.13s 1.15s p=0.006 n=6
Total Time 20.14s (± 0.29%) 20.05s (± 0.35%) ~ 19.97s 20.13s p=0.090 n=6
vscode - node (v18.15.0, x64)
Memory used 2,807,297k (± 0.00%) 2,807,087k (± 0.00%) -209k (- 0.01%) 2,807,045k 2,807,142k p=0.005 n=6
Parse Time 10.63s (± 0.55%) 10.63s (± 0.31%) ~ 10.59s 10.69s p=0.466 n=6
Bind Time 3.38s (± 0.37%) 3.39s (± 0.24%) ~ 3.39s 3.41s p=0.057 n=6
Check Time 59.74s (± 0.25%) 59.56s (± 0.11%) -0.19s (- 0.31%) 59.46s 59.66s p=0.045 n=6
Emit Time 16.12s (± 0.47%) 16.08s (± 0.35%) ~ 16.00s 16.15s p=0.260 n=6
Total Time 89.88s (± 0.18%) 89.66s (± 0.13%) ~ 89.54s 89.86s p=0.054 n=6
webpack - node (v18.15.0, x64)
Memory used 392,365k (± 0.01%) 392,255k (± 0.01%) -110k (- 0.03%) 392,179k 392,322k p=0.005 n=6
Parse Time 3.06s (± 1.07%) 3.08s (± 1.01%) ~ 3.04s 3.13s p=0.257 n=6
Bind Time 1.40s (± 0.98%) 1.40s (± 0.70%) ~ 1.38s 1.41s p=1.000 n=6
Check Time 13.98s (± 0.23%) 13.96s (± 0.24%) ~ 13.94s 14.03s p=0.332 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 18.44s (± 0.30%) 18.44s (± 0.22%) ~ 18.38s 18.49s p=0.873 n=6
xstate - node (v18.15.0, x64)
Memory used 513,440k (± 0.02%) 513,427k (± 0.01%) ~ 513,359k 513,482k p=0.936 n=6
Parse Time 3.29s (± 0.42%) 3.31s (± 0.12%) +0.02s (+ 0.71%) 3.30s 3.31s p=0.032 n=6
Bind Time 1.54s (± 0.41%) 1.56s (± 0.33%) +0.02s (+ 1.08%) 1.55s 1.56s p=0.006 n=6
Check Time 2.85s (± 0.36%) 2.88s (± 0.92%) ~ 2.83s 2.90s p=0.063 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 7.76s (± 0.12%) 7.82s (± 0.35%) +0.06s (+ 0.73%) 7.78s 7.85s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,344ms (± 0.98%) 2,331ms (± 1.04%) ~ 2,291ms 2,361ms p=0.471 n=6
Req 2 - geterr 5,524ms (± 1.23%) 5,494ms (± 1.11%) ~ 5,431ms 5,603ms p=0.471 n=6
Req 3 - references 325ms (± 0.71%) 324ms (± 1.14%) ~ 320ms 331ms p=0.199 n=6
Req 4 - navto 275ms (± 1.37%) 276ms (± 0.99%) ~ 270ms 277ms p=0.730 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 85ms (± 8.11%) 81ms (± 6.98%) ~ 78ms 92ms p=0.084 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,482ms (± 0.72%) 2,481ms (± 0.87%) ~ 2,460ms 2,511ms p=0.873 n=6
Req 2 - geterr 4,186ms (± 2.01%) 4,182ms (± 2.10%) ~ 4,097ms 4,270ms p=0.521 n=6
Req 3 - references 333ms (± 1.50%) 331ms (± 0.32%) ~ 329ms 332ms p=0.219 n=6
Req 4 - navto 285ms (± 0.56%) 286ms (± 0.96%) ~ 284ms 291ms p=1.000 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 79ms (± 7.52%) 75ms (± 2.71%) ~ 73ms 77ms p=0.139 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,610ms (± 0.37%) 2,642ms (± 0.36%) +32ms (+ 1.24%) 2,628ms 2,656ms p=0.005 n=6
Req 2 - geterr 1,748ms (± 1.38%) 1,712ms (± 2.47%) ~ 1,662ms 1,756ms p=0.093 n=6
Req 3 - references 117ms (±10.20%) 116ms (± 9.42%) ~ 106ms 127ms p=1.000 n=6
Req 4 - navto 371ms (± 0.42%) 368ms (± 1.08%) ~ 363ms 373ms p=0.195 n=6
Req 5 - completionInfo count 2,078 (± 0.00%) 2,078 (± 0.00%) ~ 2,078 2,078 p=1.000 n=6
Req 5 - completionInfo 310ms (± 1.28%) 304ms (± 1.17%) -6ms (- 1.88%) 301ms 310ms p=0.037 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 153.92ms (± 0.20%) 153.98ms (± 0.25%) +0.06ms (+ 0.04%) 152.76ms 166.41ms p=0.045 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 229.84ms (± 0.16%) 229.68ms (± 0.14%) -0.16ms (- 0.07%) 228.43ms 232.31ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 232.25ms (± 0.18%) 232.92ms (± 0.16%) +0.67ms (+ 0.29%) 231.43ms 239.04ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 231.26ms (± 0.19%) 231.26ms (± 0.16%) ~ 229.73ms 235.50ms p=0.664 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2024 •
edited

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,725k (± 0.01%) 295,581k (± 0.01%) -144k (- 0.05%) 295,563k 295,605k p=0.005 n=6
Parse Time 2.66s (± 0.34%) 2.67s (± 0.24%) ~ 2.66s 2.68s p=0.070 n=6
Bind Time 0.83s (± 0.49%) 0.83s (± 0.49%) ~ 0.83s 0.84s p=0.218 n=6
Check Time 8.21s (± 0.38%) 8.19s (± 0.29%) ~ 8.15s 8.21s p=0.195 n=6
Emit Time 7.13s (± 0.17%) 7.06s (± 0.24%) -0.07s (- 0.94%) 7.04s 7.08s p=0.005 n=6
Total Time 18.83s (± 0.21%) 18.75s (± 0.19%) -0.08s (- 0.42%) 18.71s 18.80s p=0.020 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,701k (± 1.00%) 194,254k (± 0.96%) ~ 191,827k 195,529k p=1.000 n=6
Parse Time 1.36s (± 1.72%) 1.36s (± 1.20%) ~ 1.34s 1.38s p=1.000 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.88%) ~ 0.71s 0.73s p=1.000 n=6
Check Time 9.51s (± 1.05%) 9.43s (± 0.23%) ~ 9.40s 9.46s p=0.170 n=6
Emit Time 2.65s (± 0.74%) 2.58s (± 0.47%) -0.06s (- 2.39%) 2.57s 2.60s p=0.005 n=6
Total Time 14.23s (± 0.84%) 14.08s (± 0.28%) ~ 14.05s 14.16s p=0.064 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,377k (± 0.00%) 347,372k (± 0.00%) ~ 347,356k 347,396k p=0.470 n=6
Parse Time 2.48s (± 0.79%) 2.52s (± 0.68%) +0.04s (+ 1.54%) 2.49s 2.54s p=0.014 n=6
Bind Time 0.93s (± 0.44%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=0.405 n=6
Check Time 7.01s (± 0.36%) 7.02s (± 0.29%) ~ 6.99s 7.05s p=0.332 n=6
Emit Time 4.06s (± 0.19%) 4.09s (± 0.34%) +0.02s (+ 0.57%) 4.07s 4.11s p=0.007 n=6
Total Time 14.48s (± 0.24%) 14.56s (± 0.18%) +0.08s (+ 0.56%) 14.52s 14.59s p=0.008 n=6
TFS - node (v18.15.0, x64)
Memory used 302,757k (± 0.01%) 302,611k (± 0.01%) -146k (- 0.05%) 302,587k 302,654k p=0.005 n=6
Parse Time 2.01s (± 0.51%) 2.01s (± 0.73%) ~ 2.00s 2.04s p=0.605 n=6
Bind Time 1.00s (± 1.03%) 1.01s (± 0.81%) ~ 1.00s 1.02s p=0.546 n=6
Check Time 6.33s (± 0.59%) 6.32s (± 0.48%) ~ 6.26s 6.34s p=0.809 n=6
Emit Time 3.61s (± 0.61%) 3.59s (± 0.75%) ~ 3.57s 3.63s p=0.195 n=6
Total Time 12.95s (± 0.35%) 12.93s (± 0.27%) ~ 12.89s 12.98s p=0.294 n=6
material-ui - node (v18.15.0, x64)
Memory used 511,368k (± 0.00%) 511,355k (± 0.00%) ~ 511,334k 511,366k p=0.226 n=6
Parse Time 2.65s (± 0.58%) 2.71s (± 0.40%) +0.06s (+ 2.26%) 2.69s 2.72s p=0.005 n=6
Bind Time 0.98s (± 0.91%) 0.99s (± 0.83%) ~ 0.98s 1.00s p=0.270 n=6
Check Time 17.28s (± 0.27%) 17.33s (± 0.28%) ~ 17.27s 17.39s p=0.078 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.91s (± 0.23%) 21.03s (± 0.19%) +0.12s (+ 0.57%) 20.98s 21.09s p=0.006 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,719,121k (± 0.00%) 1,719,049k (± 0.00%) -72k (- 0.00%) 1,719,011k 1,719,078k p=0.005 n=6
Parse Time 6.52s (± 0.47%) 6.61s (± 0.63%) +0.09s (+ 1.33%) 6.57s 6.68s p=0.006 n=6
Bind Time 2.36s (± 0.32%) 2.38s (± 0.86%) ~ 2.35s 2.41s p=0.060 n=6
Check Time 56.38s (± 0.37%) 56.18s (± 0.34%) ~ 55.85s 56.40s p=0.128 n=6
Emit Time 0.13s (± 0.00%) 0.13s (± 3.10%) ~ 0.13s 0.14s p=0.405 n=6
Total Time 65.39s (± 0.29%) 65.29s (± 0.36%) ~ 64.90s 65.58s p=0.575 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,395,371k (± 0.02%) 2,397,074k (± 0.04%) +1,703k (+ 0.07%) 2,396,172k 2,398,156k p=0.010 n=6
Parse Time 4.98s (± 1.07%) 5.01s (± 0.53%) ~ 4.99s 5.05s p=0.108 n=6
Bind Time 1.90s (± 1.17%) 1.90s (± 0.74%) ~ 1.88s 1.92s p=0.935 n=6
Check Time 33.63s (± 0.16%) 33.47s (± 0.42%) ~ 33.32s 33.69s p=0.066 n=6
Emit Time 2.72s (± 1.20%) 2.69s (± 3.42%) ~ 2.58s 2.82s p=0.630 n=6
Total Time 43.25s (± 0.22%) 43.10s (± 0.33%) ~ 42.94s 43.32s p=0.065 n=6
self-compiler - node (v18.15.0, x64)
Memory used 416,068k (± 0.01%) 415,688k (± 0.01%) -380k (- 0.09%) 415,646k 415,718k p=0.005 n=6
Parse Time 2.84s (± 1.05%) 2.82s (± 2.02%) ~ 2.71s 2.86s p=1.000 n=6
Bind Time 1.07s (± 0.51%) 1.09s (± 5.41%) ~ 1.06s 1.21s p=0.476 n=6
Check Time 15.32s (± 0.44%) 15.25s (± 0.35%) ~ 15.19s 15.34s p=0.092 n=6
Emit Time 1.16s (± 1.33%) 1.12s (± 0.80%) 🟩-0.04s (- 3.45%) 1.11s 1.13s p=0.005 n=6
Total Time 20.39s (± 0.29%) 20.28s (± 0.30%) -0.10s (- 0.51%) 20.23s 20.40s p=0.030 n=6
vscode - node (v18.15.0, x64)
Memory used 2,883,638k (± 0.00%) 2,883,267k (± 0.00%) -371k (- 0.01%) 2,883,114k 2,883,385k p=0.005 n=6
Parse Time 10.82s (± 0.21%) 10.84s (± 0.35%) ~ 10.78s 10.90s p=0.462 n=6
Bind Time 3.46s (± 0.61%) 3.48s (± 0.49%) ~ 3.46s 3.51s p=0.120 n=6
Check Time 61.61s (± 0.37%) 61.75s (± 0.47%) ~ 61.38s 62.24s p=0.423 n=6
Emit Time 16.43s (± 0.77%) 17.58s (±10.01%) ~ 16.40s 19.93s p=0.336 n=6
Total Time 92.33s (± 0.27%) 93.65s (± 2.14%) ~ 92.18s 96.34s p=0.295 n=6
webpack - node (v18.15.0, x64)
Memory used 408,015k (± 0.02%) 407,858k (± 0.02%) -157k (- 0.04%) 407,764k 407,935k p=0.013 n=6
Parse Time 3.23s (± 0.36%) 3.19s (± 0.17%) -0.05s (- 1.44%) 3.18s 3.19s p=0.004 n=6
Bind Time 1.38s (± 0.00%) 1.39s (± 0.74%) ~ 1.37s 1.40s p=0.126 n=6
Check Time 14.29s (± 0.23%) 14.23s (± 0.55%) ~ 14.14s 14.37s p=0.065 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 18.91s (± 0.21%) 18.80s (± 0.42%) -0.11s (- 0.59%) 18.72s 18.94s p=0.029 n=6
xstate - node (v18.15.0, x64)
Memory used 513,120k (± 0.01%) 513,119k (± 0.01%) ~ 513,026k 513,216k p=1.000 n=6
Parse Time 3.27s (± 0.27%) 3.31s (± 0.44%) +0.04s (+ 1.27%) 3.30s 3.33s p=0.004 n=6
Bind Time 1.54s (± 0.00%) 1.56s (± 0.66%) +0.02s (+ 1.08%) 1.54s 1.57s p=0.009 n=6
Check Time 2.85s (± 0.52%) 2.86s (± 0.61%) ~ 2.83s 2.88s p=0.254 n=6
Emit Time 0.07s (± 5.69%) 0.08s (± 0.00%) 🔻+0.01s (+11.63%) 0.08s 0.08s p=0.007 n=6
Total Time 7.74s (± 0.26%) 7.80s (± 0.39%) +0.07s (+ 0.84%) 7.75s 7.83s p=0.010 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,341ms (± 0.65%) 2,319ms (± 0.62%) ~ 2,303ms 2,338ms p=0.065 n=6
Req 2 - geterr 5,809ms (± 0.48%) 5,790ms (± 0.37%) ~ 5,764ms 5,815ms p=0.261 n=6
Req 3 - references 324ms (± 0.39%) 324ms (± 0.26%) ~ 323ms 325ms p=0.530 n=6
Req 4 - navto 272ms (± 0.19%) 273ms (± 0.19%) +1ms (+ 0.37%) 273ms 274ms p=0.018 n=6
Req 5 - completionInfo count 1,357 (± 0.00%) 1,357 (± 0.00%) ~ 1,357 1,357 p=1.000 n=6
Req 5 - completionInfo 95ms (± 0.43%) 94ms (± 0.55%) -1ms (- 1.23%) 93ms 94ms p=0.007 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,508ms (± 0.66%) 2,443ms (± 0.57%) -65ms (- 2.60%) 2,425ms 2,460ms p=0.005 n=6
Req 2 - geterr 4,259ms (± 0.11%) 4,260ms (± 0.33%) ~ 4,249ms 4,287ms p=0.296 n=6
Req 3 - references 331ms (± 0.33%) 332ms (± 0.94%) ~ 330ms 338ms p=0.867 n=6
Req 4 - navto 285ms (± 0.36%) 285ms (± 0.37%) ~ 283ms 286ms p=0.203 n=6
Req 5 - completionInfo count 1,519 (± 0.00%) 1,519 (± 0.00%) ~ 1,519 1,519 p=1.000 n=6
Req 5 - completionInfo 78ms (± 1.15%) 80ms (± 5.88%) ~ 77ms 89ms p=0.932 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,618ms (± 0.29%) 2,611ms (± 0.34%) ~ 2,595ms 2,620ms p=0.199 n=6
Req 2 - geterr 1,732ms (± 3.33%) 1,696ms (± 0.95%) ~ 1,673ms 1,713ms p=0.575 n=6
Req 3 - references 121ms (± 7.81%) 124ms (± 6.85%) ~ 107ms 130ms p=0.744 n=6
Req 4 - navto 372ms (± 1.42%) 371ms (± 0.74%) ~ 368ms 375ms p=0.746 n=6
Req 5 - completionInfo count 2,079 (± 0.00%) 2,079 (± 0.00%) ~ 2,079 2,079 p=1.000 n=6
Req 5 - completionInfo 311ms (± 2.76%) 306ms (± 1.46%) ~ 299ms 313ms p=0.625 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 154.14ms (± 0.17%) 154.31ms (± 0.18%) +0.17ms (+ 0.11%) 153.11ms 156.69ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.39ms (± 0.16%) 230.09ms (± 0.14%) -0.30ms (- 0.13%) 228.72ms 233.30ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 223.47ms (± 0.17%) 223.55ms (± 0.17%) +0.08ms (+ 0.04%) 221.95ms 231.23ms p=0.039 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 221.95ms (± 0.15%) 222.18ms (± 0.17%) +0.22ms (+ 0.10%) 220.76ms 225.95ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey jakebailey mentioned this pull request Apr 3, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewbranch andrewbranch andrewbranch approved these changes

@sheetalkamat sheetalkamat sheetalkamat approved these changes

@sandersn sandersn Awaiting requested review from sandersn

@rbuckton rbuckton Awaiting requested review from rbuckton

Assignees

@jakebailey jakebailey

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@jakebailey @typescript-bot @a-tarasyuk @andrewbranch @rbuckton @sheetalkamat