Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use strict variance checks for strict subtype checks unconditionally #48123

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

weswigham
Copy link
Member

@weswigham weswigham commented Mar 4, 2022

I was reviewing some older PRs in my review queue and realized I had some extra insight into one in particular since I'd last commented on it.

This is an improvement to #41995 that, rather than making anything dependent on the strictFunctionTypes flag, instead changes the strict subtype relation to always use strict function argument variance, which both fixes #41977, and should prevent order-dependence issues like those described in #46814 (but with methods instead of readonly props, like so) as well.

@fatcerberus
Copy link

Is this comment in #41977 relevant at all?

I'm convinced that changing the strict subtype relation to always check function types strictly (regardless of the --strictFunctionTypes setting) is a breaking change that we probably can't do. Too much code out there relies on the fact that we pick the "first" of two mutually assignable types in subtype reduction and would break if we pick a stricter type.

@weswigham
Copy link
Member Author

For the subtype relation, that's probably true, however for the strictSubtypeRelation, which governs pretty much just subtype reduction, it's really hard to rely on current behavior - because if you are, your code can change behavior when reordered! Take the RegExpMatchArray | never[] that popped up in the changed baselines - previously it was just RegExpMatchArray, however if you constructed a never[] before loading the lib (or put one into the lib before regexps), the behavior would swap you'd you'd get never[]! This alone means that your code can be really fragile under skipLibCheck, which makes the lib load order depend on usage order.

@weswigham
Copy link
Member Author

(You can see the playground link in the OP for an example of the very problematic order dependence issue)

@weswigham
Copy link
Member Author

@typescript-bot pack this
@typescript-bot user test this inline
@typescript-bot test this
@typescript-bot run dt
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 11, 2022

Heya @weswigham, I've started to run the extended test suite on this PR at 949daee. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 11, 2022

Heya @weswigham, I've started to run the tarball bundle task on this PR at 949daee. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 11, 2022

Heya @weswigham, I've started to run the perf test suite on this PR at 949daee. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 11, 2022

Heya @weswigham, I've started to run the diff-based community code test suite on this PR at 949daee. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 11, 2022

Heya @weswigham, I've started to run the parallelized Definitely Typed test suite on this PR at 949daee. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the user tests run you requested are in!

Here they are:

Comparison Report - main..refs/pull/48123/merge

[bcryptjs]

1 of 1 projects failed to build with the old tsc

/mnt/ts_downloads/bcryptjs/tsconfig.json

  • error TS2769: No overload matches this call.
    • /mnt/ts_downloads/bcryptjs/node_modules/bcryptjs/src/bcrypt/impl.js(490,22)
    • /mnt/ts_downloads/bcryptjs/node_modules/bcryptjs/src/bcrypt/impl.js(498,22)
    • /mnt/ts_downloads/bcryptjs/node_modules/bcryptjs/src/bcrypt/impl.js(583,22)
    • /mnt/ts_downloads/bcryptjs/node_modules/bcryptjs/src/bcrypt/impl.js(595,22)
    • /mnt/ts_downloads/bcryptjs/node_modules/bcryptjs/src/bcrypt/impl.js(609,26)
    • /mnt/ts_downloads/bcryptjs/node_modules/bcryptjs/src/bcrypt/impl.js(621,22)

@@ -36,8 +36,8 @@ var cs = [a, b, c]; // { x: number; y?: number };[]
>c : { x: number; a?: number; }

var ds = [(x: Object) => 1, (x: string) => 2]; // { (x:Object) => number }[]
>ds : ((x: Object) => number)[]
>[(x: Object) => 1, (x: string) => 2] : ((x: Object) => number)[]
>ds : ((x: string) => number)[]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pretty big correctness improvement, so we should document it as a breaking change

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..48123

Metric main 48123 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 355,907k (± 0.03%) 356,273k (± 0.01%) +366k (+ 0.10%) 356,148k 356,367k
Parse Time 2.04s (± 0.57%) 2.05s (± 0.61%) +0.01s (+ 0.39%) 2.02s 2.07s
Bind Time 0.85s (± 0.70%) 0.86s (± 0.90%) +0.01s (+ 0.59%) 0.84s 0.88s
Check Time 5.72s (± 0.40%) 5.75s (± 0.41%) +0.03s (+ 0.54%) 5.67s 5.80s
Emit Time 5.97s (± 0.54%) 5.96s (± 0.52%) -0.01s (- 0.13%) 5.91s 6.03s
Total Time 14.58s (± 0.36%) 14.61s (± 0.35%) +0.03s (+ 0.20%) 14.49s 14.74s
Compiler-Unions - node (v10.16.3, x64)
Memory used 205,570k (± 0.03%) 425,603k (± 0.14%) 🔻+220,033k (+107.04%) 423,188k 426,038k
Parse Time 0.84s (± 0.86%) 0.84s (± 0.35%) +0.00s (+ 0.00%) 0.84s 0.85s
Bind Time 0.52s (± 1.91%) 0.51s (± 1.09%) -0.01s (- 2.10%) 0.50s 0.52s
Check Time 7.87s (± 0.45%) 12.91s (± 0.61%) 🔻+5.04s (+64.08%) 12.75s 13.13s
Emit Time 2.51s (± 0.65%) 2.52s (± 1.19%) +0.01s (+ 0.32%) 2.48s 2.60s
Total Time 11.74s (± 0.38%) 16.78s (± 0.54%) 🔻+5.04s (+42.87%) 16.58s 16.98s
Monaco - node (v10.16.3, x64)
Memory used 343,333k (± 0.02%) 343,646k (± 0.01%) +314k (+ 0.09%) 343,559k 343,750k
Parse Time 1.56s (± 0.54%) 1.56s (± 0.48%) +0.00s (+ 0.13%) 1.55s 1.58s
Bind Time 0.76s (± 0.63%) 0.76s (± 1.08%) +0.00s (+ 0.00%) 0.74s 0.77s
Check Time 5.69s (± 0.39%) 5.72s (± 0.48%) +0.02s (+ 0.44%) 5.66s 5.77s
Emit Time 3.25s (± 0.82%) 3.24s (± 0.61%) -0.01s (- 0.46%) 3.19s 3.29s
Total Time 11.26s (± 0.43%) 11.27s (± 0.28%) +0.02s (+ 0.13%) 11.20s 11.35s
TFS - node (v10.16.3, x64)
Memory used 305,171k (± 0.02%) 305,300k (± 0.02%) +130k (+ 0.04%) 305,195k 305,488k
Parse Time 1.27s (± 0.59%) 1.27s (± 0.41%) +0.00s (+ 0.16%) 1.26s 1.28s
Bind Time 0.71s (± 0.51%) 0.71s (± 1.06%) +0.00s (+ 0.14%) 0.70s 0.73s
Check Time 5.20s (± 0.40%) 5.18s (± 0.53%) -0.02s (- 0.35%) 5.13s 5.28s
Emit Time 3.40s (± 1.21%) 3.42s (± 0.95%) +0.02s (+ 0.62%) 3.37s 3.52s
Total Time 10.59s (± 0.49%) 10.59s (± 0.41%) +0.01s (+ 0.05%) 10.52s 10.69s
material-ui - node (v10.16.3, x64)
Memory used 475,642k (± 0.01%) 476,244k (± 0.01%) +602k (+ 0.13%) 476,117k 476,388k
Parse Time 1.80s (± 0.42%) 1.81s (± 0.59%) +0.01s (+ 0.67%) 1.77s 1.82s
Bind Time 0.67s (± 0.54%) 0.67s (± 0.50%) -0.00s (- 0.59%) 0.66s 0.68s
Check Time 22.76s (± 0.82%) 22.75s (± 0.31%) -0.01s (- 0.06%) 22.63s 22.97s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 25.23s (± 0.74%) 25.23s (± 0.24%) -0.01s (- 0.02%) 25.11s 25.40s
xstate - node (v10.16.3, x64)
Memory used 574,688k (± 1.30%) 571,721k (± 0.02%) -2,967k (- 0.52%) 571,434k 571,921k
Parse Time 2.55s (± 0.30%) 2.57s (± 0.28%) +0.01s (+ 0.55%) 2.55s 2.59s
Bind Time 1.01s (± 0.76%) 1.00s (± 0.49%) -0.00s (- 0.10%) 0.99s 1.01s
Check Time 1.51s (± 0.45%) 1.51s (± 0.40%) +0.01s (+ 0.60%) 1.51s 1.53s
Emit Time 0.07s (± 0.00%) 0.07s (± 3.14%) +0.00s (+ 1.43%) 0.07s 0.08s
Total Time 5.13s (± 0.29%) 5.15s (± 0.16%) +0.02s (+ 0.45%) 5.14s 5.18s
Angular - node (v12.1.0, x64)
Memory used 333,673k (± 0.02%) 334,111k (± 0.01%) +437k (+ 0.13%) 334,034k 334,264k
Parse Time 2.03s (± 0.79%) 2.04s (± 0.62%) +0.01s (+ 0.29%) 2.02s 2.07s
Bind Time 0.82s (± 0.83%) 0.83s (± 0.82%) +0.00s (+ 0.24%) 0.81s 0.84s
Check Time 5.59s (± 0.70%) 5.55s (± 0.32%) -0.04s (- 0.80%) 5.52s 5.59s
Emit Time 6.21s (± 0.82%) 6.18s (± 0.33%) -0.03s (- 0.50%) 6.14s 6.24s
Total Time 14.66s (± 0.50%) 14.60s (± 0.22%) -0.06s (- 0.44%) 14.55s 14.70s
Compiler-Unions - node (v12.1.0, x64)
Memory used 192,965k (± 0.12%) 411,625k (± 0.15%) 🔻+218,660k (+113.32%) 409,149k 412,041k
Parse Time 0.84s (± 0.73%) 0.84s (± 0.41%) -0.00s (- 0.48%) 0.83s 0.84s
Bind Time 0.53s (± 0.56%) 0.53s (± 0.89%) +0.00s (+ 0.19%) 0.53s 0.55s
Check Time 7.37s (± 0.30%) 11.54s (± 0.49%) 🔻+4.17s (+56.59%) 11.41s 11.67s
Emit Time 2.55s (± 1.70%) 2.53s (± 1.08%) -0.02s (- 0.86%) 2.48s 2.61s
Total Time 11.29s (± 0.46%) 15.44s (± 0.50%) 🔻+4.15s (+36.73%) 15.26s 15.66s
Monaco - node (v12.1.0, x64)
Memory used 326,403k (± 0.03%) 326,465k (± 0.09%) +61k (+ 0.02%) 325,603k 326,889k
Parse Time 1.54s (± 1.01%) 1.55s (± 0.80%) +0.02s (+ 1.04%) 1.53s 1.58s
Bind Time 0.74s (± 0.40%) 0.74s (± 0.30%) +0.00s (+ 0.41%) 0.74s 0.75s
Check Time 5.55s (± 0.48%) 5.56s (± 0.53%) +0.01s (+ 0.18%) 5.53s 5.67s
Emit Time 3.25s (± 0.84%) 3.27s (± 1.20%) +0.02s (+ 0.52%) 3.21s 3.39s
Total Time 11.08s (± 0.40%) 11.12s (± 0.53%) +0.04s (+ 0.40%) 11.05s 11.33s
TFS - node (v12.1.0, x64)
Memory used 289,873k (± 0.02%) 289,910k (± 0.02%) +37k (+ 0.01%) 289,829k 290,018k
Parse Time 1.28s (± 0.73%) 1.29s (± 0.69%) +0.00s (+ 0.23%) 1.27s 1.31s
Bind Time 0.71s (± 0.78%) 0.71s (± 0.91%) 0.00s ( 0.00%) 0.70s 0.72s
Check Time 5.13s (± 0.52%) 5.12s (± 0.59%) -0.01s (- 0.16%) 5.05s 5.18s
Emit Time 3.48s (± 0.84%) 3.48s (± 0.75%) +0.00s (+ 0.09%) 3.41s 3.53s
Total Time 10.61s (± 0.55%) 10.60s (± 0.31%) -0.00s (- 0.03%) 10.53s 10.68s
material-ui - node (v12.1.0, x64)
Memory used 454,455k (± 0.05%) 455,128k (± 0.08%) +674k (+ 0.15%) 454,174k 455,416k
Parse Time 1.80s (± 0.49%) 1.81s (± 0.46%) +0.00s (+ 0.11%) 1.79s 1.83s
Bind Time 0.64s (± 0.62%) 0.64s (± 0.46%) +0.00s (+ 0.16%) 0.64s 0.65s
Check Time 20.41s (± 1.24%) 20.28s (± 0.72%) -0.13s (- 0.63%) 20.04s 20.75s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 22.86s (± 1.07%) 22.73s (± 0.65%) -0.13s (- 0.56%) 22.48s 23.20s
xstate - node (v12.1.0, x64)
Memory used 537,056k (± 0.02%) 537,383k (± 0.01%) +326k (+ 0.06%) 537,180k 537,578k
Parse Time 2.50s (± 0.31%) 2.50s (± 0.26%) +0.00s (+ 0.04%) 2.49s 2.51s
Bind Time 1.04s (± 0.66%) 1.03s (± 0.50%) -0.01s (- 0.48%) 1.02s 1.04s
Check Time 1.46s (± 0.46%) 1.47s (± 0.32%) +0.00s (+ 0.20%) 1.46s 1.48s
Emit Time 0.07s (± 3.14%) 0.07s (± 0.00%) -0.00s (- 1.41%) 0.07s 0.07s
Total Time 5.07s (± 0.20%) 5.07s (± 0.13%) -0.00s (- 0.06%) 5.05s 5.08s
Angular - node (v14.15.1, x64)
Memory used 331,965k (± 0.01%) 332,410k (± 0.01%) +445k (+ 0.13%) 332,378k 332,447k
Parse Time 2.01s (± 0.30%) 2.02s (± 0.51%) +0.01s (+ 0.40%) 2.00s 2.05s
Bind Time 0.87s (± 0.79%) 0.86s (± 0.58%) -0.00s (- 0.12%) 0.86s 0.88s
Check Time 5.58s (± 0.36%) 5.58s (± 0.43%) +0.00s (+ 0.02%) 5.53s 5.63s
Emit Time 6.26s (± 0.63%) 6.27s (± 0.86%) +0.01s (+ 0.14%) 6.15s 6.42s
Total Time 14.72s (± 0.33%) 14.73s (± 0.42%) +0.01s (+ 0.10%) 14.58s 14.89s
Compiler-Unions - node (v14.15.1, x64)
Memory used 194,663k (± 0.36%) 412,159k (± 0.30%) 🔻+217,497k (+111.73%) 410,319k 414,390k
Parse Time 0.85s (± 0.58%) 0.85s (± 0.43%) 0.00s ( 0.00%) 0.85s 0.86s
Bind Time 0.56s (± 0.65%) 0.56s (± 0.67%) -0.00s (- 0.18%) 0.55s 0.56s
Check Time 7.38s (± 0.53%) 11.55s (± 0.42%) 🔻+4.17s (+56.41%) 11.46s 11.69s
Emit Time 2.49s (± 0.38%) 2.50s (± 1.49%) +0.01s (+ 0.52%) 2.45s 2.61s
Total Time 11.28s (± 0.37%) 15.46s (± 0.34%) 🔻+4.18s (+37.04%) 15.33s 15.56s
Monaco - node (v14.15.1, x64)
Memory used 325,131k (± 0.01%) 325,432k (± 0.01%) +301k (+ 0.09%) 325,410k 325,514k
Parse Time 1.56s (± 0.77%) 1.57s (± 0.42%) +0.01s (+ 0.38%) 1.56s 1.58s
Bind Time 0.77s (± 0.98%) 0.78s (± 1.03%) +0.00s (+ 0.52%) 0.76s 0.79s
Check Time 5.46s (± 0.56%) 5.50s (± 0.49%) +0.04s (+ 0.73%) 5.45s 5.58s
Emit Time 3.30s (± 0.58%) 3.31s (± 0.66%) +0.01s (+ 0.42%) 3.26s 3.36s
Total Time 11.10s (± 0.34%) 11.17s (± 0.33%) +0.07s (+ 0.60%) 11.10s 11.27s
TFS - node (v14.15.1, x64)
Memory used 288,742k (± 0.01%) 288,826k (± 0.01%) +84k (+ 0.03%) 288,790k 288,895k
Parse Time 1.31s (± 1.68%) 1.32s (± 1.38%) +0.01s (+ 0.69%) 1.29s 1.36s
Bind Time 0.73s (± 0.41%) 0.74s (± 1.01%) +0.00s (+ 0.68%) 0.73s 0.76s
Check Time 5.11s (± 0.65%) 5.11s (± 0.29%) -0.00s (- 0.08%) 5.07s 5.14s
Emit Time 3.45s (± 1.86%) 3.52s (± 2.07%) +0.06s (+ 1.85%) 3.38s 3.61s
Total Time 10.61s (± 0.87%) 10.68s (± 0.93%) +0.07s (+ 0.66%) 10.46s 10.84s
material-ui - node (v14.15.1, x64)
Memory used 453,068k (± 0.06%) 453,590k (± 0.05%) +522k (+ 0.12%) 452,625k 453,734k
Parse Time 1.84s (± 0.32%) 1.85s (± 0.51%) +0.02s (+ 0.98%) 1.84s 1.88s
Bind Time 0.71s (± 0.67%) 0.70s (± 0.88%) -0.01s (- 0.85%) 0.69s 0.71s
Check Time 20.36s (± 0.73%) 20.32s (± 0.45%) -0.04s (- 0.21%) 20.07s 20.55s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 22.90s (± 0.66%) 22.87s (± 0.41%) -0.03s (- 0.14%) 22.62s 23.11s
xstate - node (v14.15.1, x64)
Memory used 534,781k (± 0.00%) 535,100k (± 0.01%) +319k (+ 0.06%) 535,046k 535,154k
Parse Time 2.55s (± 0.36%) 2.56s (± 0.43%) +0.02s (+ 0.67%) 2.54s 2.59s
Bind Time 1.14s (± 0.68%) 1.14s (± 0.97%) -0.00s (- 0.18%) 1.12s 1.17s
Check Time 1.50s (± 0.20%) 1.50s (± 0.38%) +0.00s (+ 0.13%) 1.49s 1.51s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.26s (± 0.29%) 5.28s (± 0.16%) +0.02s (+ 0.30%) 5.26s 5.30s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory4 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v10.16.3, x64)
  • xstate - node (v12.1.0, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 48123 10
Baseline main 10

Developer Information:

Download Benchmark

@weswigham
Copy link
Member Author

Man, xstate is fine but compiler-unions really hates this; but... Why? None of the unions in compiler-unions even have methods... Hm. Probably warrants investigation.

@sandersn sandersn added the For Milestone Bug PRs that fix a bug with a specific milestone label Mar 17, 2022
@weswigham
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 23, 2022

Heya @weswigham, I've started to run the perf test suite on this PR at afb0126. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..48123

Metric main 48123 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 356,412k (± 0.04%) 356,423k (± 0.02%) +11k (+ 0.00%) 356,268k 356,556k
Parse Time 2.05s (± 0.36%) 2.06s (± 0.68%) +0.01s (+ 0.39%) 2.03s 2.09s
Bind Time 0.86s (± 0.95%) 0.87s (± 0.79%) +0.01s (+ 1.05%) 0.85s 0.88s
Check Time 5.74s (± 0.52%) 5.80s (± 0.57%) +0.06s (+ 1.03%) 5.74s 5.89s
Emit Time 5.97s (± 0.43%) 6.03s (± 0.82%) +0.07s (+ 1.14%) 5.96s 6.17s
Total Time 14.61s (± 0.34%) 14.76s (± 0.50%) +0.15s (+ 1.01%) 14.60s 14.98s
Compiler-Unions - node (v10.16.3, x64)
Memory used 205,687k (± 0.04%) 425,966k (± 0.01%) 🔻+220,280k (+107.09%) 425,829k 426,094k
Parse Time 0.84s (± 0.59%) 0.84s (± 0.43%) -0.00s (- 0.12%) 0.84s 0.85s
Bind Time 0.52s (± 0.97%) 0.52s (± 2.02%) +0.00s (+ 0.97%) 0.51s 0.56s
Check Time 7.89s (± 0.58%) 13.06s (± 0.66%) 🔻+5.17s (+65.47%) 12.89s 13.22s
Emit Time 2.52s (± 1.07%) 2.51s (± 0.91%) -0.01s (- 0.44%) 2.46s 2.57s
Total Time 11.77s (± 0.52%) 16.93s (± 0.49%) 🔻+5.16s (+43.82%) 16.78s 17.10s
Monaco - node (v10.16.3, x64)
Memory used 343,727k (± 0.01%) 343,679k (± 0.01%) -49k (- 0.01%) 343,578k 343,756k
Parse Time 1.57s (± 0.41%) 1.57s (± 0.76%) -0.00s (- 0.00%) 1.54s 1.60s
Bind Time 0.76s (± 0.49%) 0.76s (± 0.63%) +0.00s (+ 0.26%) 0.75s 0.77s
Check Time 5.70s (± 0.27%) 5.77s (± 0.24%) +0.07s (+ 1.23%) 5.73s 5.79s
Emit Time 3.22s (± 0.41%) 3.24s (± 0.57%) +0.02s (+ 0.71%) 3.18s 3.27s
Total Time 11.24s (± 0.20%) 11.34s (± 0.26%) +0.09s (+ 0.83%) 11.26s 11.38s
TFS - node (v10.16.3, x64)
Memory used 305,340k (± 0.02%) 305,331k (± 0.03%) -9k (- 0.00%) 305,040k 305,449k
Parse Time 1.28s (± 0.75%) 1.28s (± 0.81%) +0.00s (+ 0.08%) 1.26s 1.30s
Bind Time 0.71s (± 0.69%) 0.72s (± 0.80%) +0.01s (+ 0.84%) 0.71s 0.73s
Check Time 5.19s (± 0.59%) 5.23s (± 0.71%) +0.04s (+ 0.81%) 5.17s 5.32s
Emit Time 3.43s (± 1.40%) 3.43s (± 0.92%) -0.00s (- 0.12%) 3.36s 3.49s
Total Time 10.62s (± 0.64%) 10.66s (± 0.44%) +0.05s (+ 0.43%) 10.57s 10.79s
material-ui - node (v10.16.3, x64)
Memory used 476,235k (± 0.01%) 476,245k (± 0.01%) +10k (+ 0.00%) 476,144k 476,363k
Parse Time 1.81s (± 0.41%) 1.82s (± 0.42%) +0.01s (+ 0.28%) 1.80s 1.83s
Bind Time 0.67s (± 0.92%) 0.68s (± 0.82%) +0.01s (+ 1.04%) 0.67s 0.69s
Check Time 22.99s (± 1.02%) 23.07s (± 0.95%) +0.08s (+ 0.34%) 22.71s 23.78s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 25.48s (± 0.93%) 25.56s (± 0.88%) +0.09s (+ 0.35%) 25.17s 26.28s
xstate - node (v10.16.3, x64)
Memory used 571,957k (± 0.01%) 573,975k (± 1.25%) +2,017k (+ 0.35%) 570,565k 603,013k
Parse Time 2.58s (± 0.29%) 2.58s (± 0.45%) +0.00s (+ 0.00%) 2.55s 2.60s
Bind Time 1.01s (± 0.47%) 1.01s (± 0.79%) -0.00s (- 0.20%) 0.99s 1.02s
Check Time 1.51s (± 0.44%) 1.53s (± 0.49%) +0.02s (+ 1.39%) 1.52s 1.55s
Emit Time 0.07s (± 0.00%) 0.07s (± 3.14%) +0.00s (+ 1.43%) 0.07s 0.08s
Total Time 5.17s (± 0.24%) 5.20s (± 0.32%) +0.03s (+ 0.48%) 5.15s 5.23s
Angular - node (v12.1.0, x64)
Memory used 334,174k (± 0.03%) 334,236k (± 0.02%) +62k (+ 0.02%) 334,044k 334,378k
Parse Time 2.04s (± 0.44%) 2.04s (± 0.51%) +0.00s (+ 0.15%) 2.01s 2.06s
Bind Time 0.82s (± 0.73%) 0.83s (± 1.12%) +0.00s (+ 0.36%) 0.82s 0.86s
Check Time 5.56s (± 0.38%) 5.64s (± 0.46%) +0.08s (+ 1.44%) 5.60s 5.70s
Emit Time 6.20s (± 0.41%) 6.24s (± 0.40%) +0.04s (+ 0.58%) 6.19s 6.30s
Total Time 14.63s (± 0.26%) 14.75s (± 0.30%) +0.12s (+ 0.83%) 14.68s 14.84s
Compiler-Unions - node (v12.1.0, x64)
Memory used 193,182k (± 0.11%) 412,045k (± 0.02%) 🔻+218,863k (+113.29%) 411,901k 412,209k
Parse Time 0.84s (± 1.03%) 0.84s (± 0.62%) -0.00s (- 0.12%) 0.83s 0.85s
Bind Time 0.54s (± 0.68%) 0.53s (± 0.65%) -0.01s (- 1.68%) 0.52s 0.53s
Check Time 7.37s (± 0.65%) 11.62s (± 0.56%) 🔻+4.25s (+57.58%) 11.48s 11.81s
Emit Time 2.54s (± 0.85%) 2.52s (± 0.81%) -0.02s (- 0.75%) 2.48s 2.56s
Total Time 11.29s (± 0.45%) 15.51s (± 0.52%) 🔻+4.22s (+37.38%) 15.32s 15.71s
Monaco - node (v12.1.0, x64)
Memory used 326,691k (± 0.02%) 326,553k (± 0.06%) -138k (- 0.04%) 325,896k 326,867k
Parse Time 1.55s (± 0.48%) 1.54s (± 0.63%) -0.01s (- 0.77%) 1.52s 1.57s
Bind Time 0.74s (± 0.78%) 0.74s (± 0.67%) -0.00s (- 0.54%) 0.73s 0.75s
Check Time 5.54s (± 0.27%) 5.63s (± 0.31%) +0.09s (+ 1.59%) 5.60s 5.68s
Emit Time 3.26s (± 0.50%) 3.25s (± 0.53%) -0.01s (- 0.21%) 3.21s 3.30s
Total Time 11.09s (± 0.15%) 11.16s (± 0.27%) +0.07s (+ 0.62%) 11.11s 11.25s
TFS - node (v12.1.0, x64)
Memory used 289,877k (± 0.06%) 290,028k (± 0.04%) +151k (+ 0.05%) 289,871k 290,414k
Parse Time 1.28s (± 0.52%) 1.30s (± 0.62%) +0.02s (+ 1.41%) 1.28s 1.32s
Bind Time 0.71s (± 0.63%) 0.70s (± 0.74%) -0.01s (- 0.99%) 0.69s 0.71s
Check Time 5.11s (± 0.36%) 5.16s (± 0.73%) +0.05s (+ 1.04%) 5.08s 5.26s
Emit Time 3.46s (± 1.00%) 3.47s (± 1.12%) +0.02s (+ 0.52%) 3.39s 3.57s
Total Time 10.56s (± 0.43%) 10.64s (± 0.67%) +0.08s (+ 0.73%) 10.51s 10.79s
material-ui - node (v12.1.0, x64)
Memory used 454,815k (± 0.09%) 455,310k (± 0.01%) +495k (+ 0.11%) 455,242k 455,414k
Parse Time 1.80s (± 0.70%) 1.80s (± 0.43%) 0.00s ( 0.00%) 1.78s 1.82s
Bind Time 0.64s (± 0.47%) 0.65s (± 1.05%) +0.01s (+ 1.25%) 0.63s 0.66s
Check Time 20.32s (± 1.03%) 20.34s (± 0.78%) +0.01s (+ 0.05%) 20.00s 20.65s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 22.77s (± 0.94%) 22.78s (± 0.71%) +0.02s (+ 0.07%) 22.45s 23.09s
xstate - node (v12.1.0, x64)
Memory used 540,946k (± 1.35%) 536,407k (± 0.02%) -4,539k (- 0.84%) 536,207k 536,645k
Parse Time 2.52s (± 0.61%) 2.52s (± 0.92%) +0.01s (+ 0.36%) 2.49s 2.59s
Bind Time 1.03s (± 0.65%) 1.03s (± 0.73%) +0.00s (+ 0.10%) 1.01s 1.05s
Check Time 1.47s (± 0.65%) 1.48s (± 0.71%) +0.01s (+ 0.95%) 1.46s 1.50s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.08s (± 0.25%) 5.11s (± 0.53%) +0.03s (+ 0.51%) 5.06s 5.18s
Angular - node (v14.15.1, x64)
Memory used 332,449k (± 0.01%) 332,424k (± 0.01%) -25k (- 0.01%) 332,377k 332,495k
Parse Time 2.02s (± 0.49%) 2.03s (± 0.40%) +0.01s (+ 0.59%) 2.01s 2.05s
Bind Time 0.86s (± 0.57%) 0.86s (± 0.43%) +0.00s (+ 0.12%) 0.86s 0.87s
Check Time 5.58s (± 0.30%) 5.64s (± 0.37%) +0.06s (+ 1.15%) 5.58s 5.68s
Emit Time 6.30s (± 0.50%) 6.27s (± 0.53%) -0.02s (- 0.35%) 6.21s 6.36s
Total Time 14.76s (± 0.34%) 14.81s (± 0.30%) +0.05s (+ 0.36%) 14.74s 14.94s
Compiler-Unions - node (v14.15.1, x64)
Memory used 192,954k (± 0.58%) 412,759k (± 0.32%) 🔻+219,805k (+113.92%) 410,559k 414,577k
Parse Time 0.86s (± 0.86%) 0.86s (± 0.65%) -0.01s (- 0.58%) 0.85s 0.87s
Bind Time 0.56s (± 1.77%) 0.56s (± 0.59%) -0.00s (- 0.53%) 0.55s 0.57s
Check Time 7.47s (± 0.49%) 11.61s (± 0.57%) 🔻+4.14s (+55.51%) 11.49s 11.81s
Emit Time 2.51s (± 0.81%) 2.52s (± 1.03%) +0.01s (+ 0.52%) 2.49s 2.62s
Total Time 11.40s (± 0.42%) 15.55s (± 0.43%) 🔻+4.15s (+36.38%) 15.41s 15.73s
Monaco - node (v14.15.1, x64)
Memory used 325,419k (± 0.00%) 325,448k (± 0.01%) +29k (+ 0.01%) 325,407k 325,503k
Parse Time 1.58s (± 0.82%) 1.58s (± 0.38%) 0.00s ( 0.00%) 1.56s 1.59s
Bind Time 0.77s (± 0.47%) 0.78s (± 0.93%) +0.01s (+ 1.03%) 0.77s 0.80s
Check Time 5.47s (± 0.46%) 5.54s (± 0.41%) +0.07s (+ 1.22%) 5.48s 5.58s
Emit Time 3.31s (± 0.78%) 3.30s (± 0.91%) -0.00s (- 0.12%) 3.24s 3.37s
Total Time 11.13s (± 0.28%) 11.20s (± 0.36%) +0.07s (+ 0.64%) 11.12s 11.30s
TFS - node (v14.15.1, x64)
Memory used 288,889k (± 0.01%) 288,904k (± 0.01%) +14k (+ 0.00%) 288,880k 288,966k
Parse Time 1.32s (± 1.07%) 1.32s (± 1.45%) -0.01s (- 0.45%) 1.29s 1.37s
Bind Time 0.73s (± 0.79%) 0.73s (± 0.61%) -0.00s (- 0.27%) 0.72s 0.74s
Check Time 5.11s (± 0.32%) 5.16s (± 0.50%) +0.05s (+ 1.02%) 5.10s 5.21s
Emit Time 3.52s (± 1.67%) 3.45s (± 1.39%) -0.08s (- 2.19%) 3.39s 3.62s
Total Time 10.69s (± 0.59%) 10.65s (± 0.54%) -0.03s (- 0.32%) 10.59s 10.86s
material-ui - node (v14.15.1, x64)
Memory used 453,590k (± 0.03%) 453,682k (± 0.01%) +92k (+ 0.02%) 453,633k 453,731k
Parse Time 1.87s (± 0.79%) 1.86s (± 0.58%) -0.01s (- 0.37%) 1.84s 1.89s
Bind Time 0.70s (± 1.06%) 0.70s (± 1.16%) +0.00s (+ 0.43%) 0.69s 0.72s
Check Time 20.49s (± 1.02%) 20.54s (± 0.90%) +0.05s (+ 0.23%) 20.22s 20.92s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 23.06s (± 0.95%) 23.11s (± 0.83%) +0.04s (+ 0.19%) 22.79s 23.49s
xstate - node (v14.15.1, x64)
Memory used 535,371k (± 0.01%) 534,104k (± 0.00%) -1,267k (- 0.24%) 534,058k 534,160k
Parse Time 2.57s (± 0.43%) 2.58s (± 0.54%) +0.00s (+ 0.19%) 2.55s 2.61s
Bind Time 1.15s (± 1.04%) 1.14s (± 0.86%) -0.01s (- 0.52%) 1.12s 1.16s
Check Time 1.50s (± 0.50%) 1.52s (± 0.41%) +0.02s (+ 1.06%) 1.51s 1.53s
Emit Time 0.07s (± 3.14%) 0.07s (± 0.00%) -0.00s (- 1.41%) 0.07s 0.07s
Total Time 5.30s (± 0.23%) 5.32s (± 0.35%) +0.01s (+ 0.23%) 5.28s 5.37s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory4 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v10.16.3, x64)
  • xstate - node (v12.1.0, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 48123 10
Baseline main 10

Developer Information:

Download Benchmark

@sandersn
Copy link
Member

sandersn commented Dec 4, 2023

@weswigham did this pan out? Did you figure out why compiler-unions slowed down? It's old enough that I'd like to close it if it didn't.

@weswigham weswigham marked this pull request as draft August 13, 2024 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
For Milestone Bug PRs that fix a bug with a specific milestone
Projects
Status: Waiting on author
Development

Successfully merging this pull request may close these issues.

Unexpected behavior when static methods are used in an array
5 participants