Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Omit effects-free conditional constructs from control flow graph #58013

Merged
merged 3 commits into from Apr 4, 2024

Conversation

ahejlsberg
Copy link
Member

@ahejlsberg ahejlsberg commented Mar 31, 2024

With this PR we optimize control flow graph construction to omit conditional constructs that have no control flow effects. Consider this example:

function test(x: A | B | C) {
  let str = x instanceof A ? doWithA(x) :
    x instanceof B ? doWithB(x) :
    doWithC(x) :
  foo(x);
  return str;
}

This currently creates the following control flow graph for the reference to x in the foo(x) call:

graphviz

With this PR, the effects-free conditional operator expression is removed from the control flow graph for references that follow it in the graph:

graphviz (1)

Intuitively, this optimization is possible because the observed control flow types at the bottom BranchLabel node are the same as the observed control flow types at the Start node for any variable, parameter, or property reference.

The shallower control flow graphs reduce check time up to 2% in several projects.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Mar 31, 2024
@ahejlsberg
Copy link
Member Author

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 31, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started 👀 Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

Hey @ahejlsberg, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,678k (± 0.01%) 295,450k (± 0.01%) -228k (- 0.08%) 295,415k 295,473k p=0.005 n=6
Parse Time 2.66s (± 0.15%) 2.66s (± 0.34%) ~ 2.65s 2.67s p=0.787 n=6
Bind Time 0.83s (± 1.00%) 0.84s (± 1.43%) ~ 0.83s 0.86s p=0.227 n=6
Check Time 8.25s (± 0.17%) 8.19s (± 0.21%) -0.06s (- 0.75%) 8.17s 8.22s p=0.005 n=6
Emit Time 7.04s (± 0.45%) 7.04s (± 0.54%) ~ 7.02s 7.12s p=1.000 n=6
Total Time 18.79s (± 0.20%) 18.74s (± 0.25%) ~ 18.70s 18.83s p=0.092 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,352k (± 0.94%) 192,190k (± 0.74%) -1,161k (- 0.60%) 191,569k 195,095k p=0.030 n=6
Parse Time 1.36s (± 0.90%) 1.37s (± 0.97%) ~ 1.35s 1.38s p=0.729 n=6
Bind Time 0.72s (± 0.75%) 0.73s (± 0.56%) ~ 0.73s 0.74s p=0.054 n=6
Check Time 9.56s (± 0.62%) 9.38s (± 0.22%) -0.17s (- 1.80%) 9.36s 9.42s p=0.005 n=6
Emit Time 2.63s (± 0.80%) 2.62s (± 0.45%) ~ 2.61s 2.64s p=0.622 n=6
Total Time 14.27s (± 0.30%) 14.11s (± 0.24%) -0.17s (- 1.17%) 14.06s 14.16s p=0.005 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,404k (± 0.00%) 347,143k (± 0.00%) -261k (- 0.08%) 347,127k 347,162k p=0.005 n=6
Parse Time 2.49s (± 0.55%) 2.48s (± 0.51%) ~ 2.46s 2.49s p=0.357 n=6
Bind Time 0.93s (± 0.44%) 0.92s (± 0.00%) -0.01s (- 0.90%) 0.92s 0.92s p=0.007 n=6
Check Time 7.00s (± 0.60%) 7.01s (± 0.28%) ~ 6.99s 7.04s p=0.568 n=6
Emit Time 4.07s (± 0.30%) 4.07s (± 0.46%) ~ 4.05s 4.10s p=1.000 n=6
Total Time 14.49s (± 0.29%) 14.48s (± 0.27%) ~ 14.44s 14.55s p=0.629 n=6
TFS - node (v18.15.0, x64)
Memory used 302,806k (± 0.01%) 302,284k (± 0.01%) -522k (- 0.17%) 302,265k 302,323k p=0.005 n=6
Parse Time 2.41s (± 1.57%) 2.41s (± 1.18%) ~ 2.38s 2.46s p=0.744 n=6
Bind Time 1.21s (± 0.52%) 1.19s (± 0.34%) -0.02s (- 1.52%) 1.19s 1.20s p=0.004 n=6
Check Time 7.49s (± 0.20%) 7.48s (± 0.22%) ~ 7.47s 7.51s p=0.684 n=6
Emit Time 4.28s (± 0.31%) 4.26s (± 0.36%) ~ 4.25s 4.29s p=0.088 n=6
Total Time 15.39s (± 0.22%) 15.35s (± 0.24%) ~ 15.31s 15.41s p=0.092 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,042k (± 0.00%) 509,974k (± 0.00%) -68k (- 0.01%) 509,951k 510,012k p=0.008 n=6
Parse Time 2.66s (± 0.24%) 2.67s (± 0.66%) ~ 2.64s 2.69s p=0.454 n=6
Bind Time 0.99s (± 1.23%) 1.00s (± 0.84%) ~ 0.98s 1.00s p=0.934 n=6
Check Time 17.34s (± 0.55%) 17.26s (± 0.28%) ~ 17.19s 17.33s p=0.227 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.00s (± 0.51%) 20.92s (± 0.28%) ~ 20.84s 21.01s p=0.335 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,744,665k (± 0.00%) 1,742,824k (± 0.00%) -1,841k (- 0.11%) 1,742,764k 1,742,866k p=0.005 n=6
Parse Time 7.81s (± 0.48%) 7.89s (± 2.14%) ~ 7.79s 8.23s p=0.622 n=6
Bind Time 2.82s (± 1.13%) 2.81s (± 0.61%) ~ 2.79s 2.84s p=0.332 n=6
Check Time 67.17s (± 0.27%) 66.43s (± 0.62%) -0.73s (- 1.09%) 65.93s 67.17s p=0.020 n=6
Emit Time 0.16s (± 5.58%) 0.16s (± 3.29%) ~ 0.15s 0.16s p=0.541 n=6
Total Time 77.96s (± 0.25%) 77.29s (± 0.57%) -0.67s (- 0.86%) 76.73s 78.01s p=0.020 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,400,146k (± 0.06%) 2,397,534k (± 0.04%) -2,612k (- 0.11%) 2,395,667k 2,398,430k p=0.020 n=6
Parse Time 7.48s (± 0.75%) 7.48s (± 1.04%) ~ 7.33s 7.54s p=0.575 n=6
Bind Time 2.79s (± 1.12%) 2.64s (± 5.05%) 🟩-0.15s (- 5.43%) 2.49s 2.78s p=0.020 n=6
Check Time 49.50s (± 0.26%) 49.12s (± 0.56%) -0.38s (- 0.78%) 48.71s 49.46s p=0.020 n=6
Emit Time 4.01s (± 3.80%) 3.96s (± 2.03%) ~ 3.83s 4.05s p=0.810 n=6
Total Time 63.78s (± 0.32%) 63.20s (± 0.36%) -0.58s (- 0.92%) 62.95s 63.61s p=0.013 n=6
self-compiler - node (v18.15.0, x64)
Memory used 416,082k (± 0.00%) 414,769k (± 0.01%) -1,314k (- 0.32%) 414,720k 414,810k p=0.005 n=6
Parse Time 3.32s (± 0.73%) 3.33s (± 0.63%) ~ 3.31s 3.36s p=0.570 n=6
Bind Time 1.30s (± 0.40%) 1.30s (± 0.84%) ~ 1.29s 1.32s p=0.784 n=6
Check Time 18.20s (± 0.32%) 17.93s (± 0.24%) -0.27s (- 1.48%) 17.86s 17.98s p=0.005 n=6
Emit Time 1.36s (± 1.50%) 1.34s (± 1.03%) -0.02s (- 1.72%) 1.32s 1.35s p=0.050 n=6
Total Time 24.18s (± 0.22%) 23.90s (± 0.21%) -0.28s (- 1.17%) 23.81s 23.95s p=0.005 n=6
vscode - node (v18.15.0, x64)
Memory used 2,898,737k (± 0.00%) 2,893,995k (± 0.00%) -4,742k (- 0.16%) 2,893,884k 2,894,059k p=0.005 n=6
Parse Time 12.87s (± 0.38%) 12.92s (± 0.50%) ~ 12.83s 12.99s p=0.297 n=6
Bind Time 4.10s (± 0.50%) 4.15s (± 0.35%) +0.04s (+ 1.06%) 4.13s 4.16s p=0.007 n=6
Check Time 71.41s (± 0.41%) 71.07s (± 0.67%) ~ 70.81s 72.04s p=0.066 n=6
Emit Time 19.90s (± 7.73%) 19.28s (± 0.83%) ~ 19.05s 19.45s p=0.809 n=6
Total Time 108.29s (± 1.64%) 107.42s (± 0.43%) ~ 107.00s 108.28s p=0.149 n=6
webpack - node (v18.15.0, x64)
Memory used 408,871k (± 0.01%) 408,468k (± 0.04%) -403k (- 0.10%) 408,318k 408,680k p=0.005 n=6
Parse Time 3.89s (± 0.42%) 3.89s (± 0.27%) ~ 3.87s 3.90s p=0.743 n=6
Bind Time 1.68s (± 0.82%) 1.68s (± 0.70%) ~ 1.66s 1.69s p=0.461 n=6
Check Time 16.83s (± 0.14%) 16.78s (± 0.52%) ~ 16.68s 16.89s p=0.376 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.41s (± 0.20%) 22.34s (± 0.32%) ~ 22.26s 22.43s p=0.149 n=6
xstate - node (v18.15.0, x64)
Memory used 513,467k (± 0.01%) 513,261k (± 0.01%) -206k (- 0.04%) 513,189k 513,331k p=0.005 n=6
Parse Time 3.95s (± 0.34%) 3.95s (± 0.26%) ~ 3.94s 3.97s p=0.683 n=6
Bind Time 1.88s (± 0.78%) 1.88s (± 0.82%) ~ 1.86s 1.90s p=1.000 n=6
Check Time 3.45s (± 0.46%) 3.41s (± 0.86%) -0.04s (- 1.21%) 3.37s 3.45s p=0.019 n=6
Emit Time 0.08s (± 6.19%) 0.09s (± 4.45%) 🔻+0.01s (+10.00%) 0.09s 0.10s p=0.006 n=6
Total Time 9.36s (± 0.28%) 9.33s (± 0.18%) ~ 9.30s 9.35s p=0.064 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the user tests comparing main and refs/pull/58013/merge:

Something interesting changed - please have a look.

Details

webpack

tsconfig.types.json

@ahejlsberg
Copy link
Member Author

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 31, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started 👀 Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

Hey @ahejlsberg, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the user tests comparing main and refs/pull/58013/merge:

Something interesting changed - please have a look.

Details

webpack

tsconfig.types.json

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,667k (± 0.01%) 295,434k (± 0.01%) -234k (- 0.08%) 295,415k 295,462k p=0.005 n=6
Parse Time 2.66s (± 0.28%) 2.66s (± 0.00%) ~ 2.66s 2.66s p=0.598 n=6
Bind Time 0.83s (± 1.00%) 0.83s (± 1.01%) ~ 0.82s 0.84s p=0.062 n=6
Check Time 8.24s (± 0.23%) 8.22s (± 0.28%) ~ 8.20s 8.25s p=0.373 n=6
Emit Time 7.04s (± 0.23%) 7.04s (± 0.25%) ~ 7.02s 7.07s p=0.802 n=6
Total Time 18.77s (± 0.04%) 18.75s (± 0.14%) ~ 18.71s 18.79s p=0.061 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,338k (± 0.97%) 192,826k (± 0.90%) ~ 191,585k 195,084k p=0.128 n=6
Parse Time 1.36s (± 0.55%) 1.36s (± 1.43%) ~ 1.33s 1.38s p=1.000 n=6
Bind Time 0.73s (± 0.00%) 0.72s (± 0.71%) -0.01s (- 0.91%) 0.72s 0.73s p=0.025 n=6
Check Time 9.53s (± 0.49%) 9.37s (± 0.65%) -0.15s (- 1.61%) 9.31s 9.46s p=0.006 n=6
Emit Time 2.64s (± 0.56%) 2.62s (± 0.54%) ~ 2.60s 2.64s p=0.073 n=6
Total Time 14.25s (± 0.35%) 14.07s (± 0.46%) -0.17s (- 1.20%) 13.99s 14.17s p=0.005 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,406k (± 0.01%) 347,118k (± 0.00%) -288k (- 0.08%) 347,092k 347,137k p=0.005 n=6
Parse Time 2.49s (± 0.59%) 2.48s (± 0.49%) ~ 2.47s 2.50s p=0.503 n=6
Bind Time 0.93s (± 0.00%) 0.89s (± 0.61%) 🟩-0.04s (- 3.76%) 0.89s 0.90s p=0.002 n=6
Check Time 7.03s (± 0.36%) 7.00s (± 0.36%) ~ 6.96s 7.03s p=0.061 n=6
Emit Time 4.06s (± 0.41%) 4.06s (± 0.41%) ~ 4.04s 4.09s p=0.934 n=6
Total Time 14.50s (± 0.34%) 14.44s (± 0.21%) ~ 14.39s 14.47s p=0.053 n=6
TFS - node (v18.15.0, x64)
Memory used 302,780k (± 0.01%) 302,275k (± 0.01%) -505k (- 0.17%) 302,243k 302,320k p=0.005 n=6
Parse Time 2.44s (± 0.73%) 2.40s (± 1.34%) -0.04s (- 1.84%) 2.36s 2.44s p=0.029 n=6
Bind Time 1.20s (± 0.53%) 1.16s (± 0.70%) 🟩-0.04s (- 3.06%) 1.16s 1.18s p=0.003 n=6
Check Time 7.52s (± 0.47%) 7.49s (± 0.50%) ~ 7.45s 7.54s p=0.196 n=6
Emit Time 4.27s (± 0.48%) 4.29s (± 0.62%) ~ 4.25s 4.33s p=0.257 n=6
Total Time 15.43s (± 0.16%) 15.33s (± 0.53%) -0.09s (- 0.61%) 15.23s 15.44s p=0.044 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,052k (± 0.00%) 509,968k (± 0.00%) -84k (- 0.02%) 509,931k 510,007k p=0.005 n=6
Parse Time 2.66s (± 0.19%) 2.66s (± 0.41%) ~ 2.65s 2.68s p=0.784 n=6
Bind Time 1.00s (± 0.55%) 0.99s (± 0.76%) ~ 0.98s 1.00s p=0.476 n=6
Check Time 17.35s (± 0.55%) 17.26s (± 0.63%) ~ 17.14s 17.45s p=0.148 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.01s (± 0.44%) 20.91s (± 0.55%) ~ 20.79s 21.11s p=0.108 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,744,695k (± 0.00%) 1,742,846k (± 0.00%) -1,850k (- 0.11%) 1,742,802k 1,742,893k p=0.005 n=6
Parse Time 9.64s (± 0.54%) 9.73s (± 1.77%) ~ 9.60s 10.07s p=0.261 n=6
Bind Time 3.47s (± 0.92%) 3.46s (± 0.44%) ~ 3.44s 3.48s p=0.681 n=6
Check Time 82.02s (± 0.21%) 81.14s (± 0.21%) -0.89s (- 1.08%) 80.87s 81.38s p=0.005 n=6
Emit Time 0.19s (± 2.13%) 0.19s (± 2.13%) ~ 0.19s 0.20s p=1.000 n=6
Total Time 95.32s (± 0.18%) 94.52s (± 0.27%) -0.80s (- 0.84%) 94.18s 94.89s p=0.005 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,399,680k (± 0.02%) 2,397,413k (± 0.04%) -2,267k (- 0.09%) 2,396,480k 2,398,817k p=0.005 n=6
Parse Time 6.00s (± 0.85%) 5.99s (± 0.35%) ~ 5.97s 6.02s p=0.872 n=6
Bind Time 2.25s (± 1.18%) 2.17s (± 4.21%) ~ 2.05s 2.25s p=0.053 n=6
Check Time 39.87s (± 0.29%) 39.60s (± 0.89%) ~ 39.27s 40.17s p=0.149 n=6
Emit Time 3.19s (± 0.86%) 3.16s (± 1.32%) ~ 3.11s 3.21s p=0.261 n=6
Total Time 51.34s (± 0.24%) 50.94s (± 0.50%) -0.39s (- 0.77%) 50.67s 51.36s p=0.020 n=6
self-compiler - node (v18.15.0, x64)
Memory used 416,097k (± 0.01%) 414,764k (± 0.01%) -1,334k (- 0.32%) 414,731k 414,787k p=0.005 n=6
Parse Time 3.35s (± 0.73%) 3.34s (± 1.02%) ~ 3.29s 3.39s p=0.627 n=6
Bind Time 1.29s (± 0.42%) 1.29s (± 0.94%) ~ 1.28s 1.31s p=0.865 n=6
Check Time 18.22s (± 0.44%) 18.01s (± 0.41%) -0.22s (- 1.19%) 17.92s 18.12s p=0.008 n=6
Emit Time 1.35s (± 1.58%) 1.35s (± 1.21%) ~ 1.33s 1.37s p=0.685 n=6
Total Time 24.21s (± 0.33%) 23.99s (± 0.31%) -0.23s (- 0.94%) 23.92s 24.13s p=0.008 n=6
vscode - node (v18.15.0, x64)
Memory used 2,898,621k (± 0.01%) 2,893,873k (± 0.00%) -4,748k (- 0.16%) 2,893,754k 2,893,970k p=0.005 n=6
Parse Time 12.88s (± 0.31%) 12.86s (± 0.30%) ~ 12.80s 12.92s p=0.517 n=6
Bind Time 4.12s (± 0.56%) 4.11s (± 0.65%) ~ 4.08s 4.15s p=0.333 n=6
Check Time 71.66s (± 0.68%) 70.94s (± 0.71%) -0.72s (- 1.00%) 70.48s 71.89s p=0.045 n=6
Emit Time 21.16s (±10.02%) 19.89s (± 8.12%) ~ 19.12s 23.19s p=0.688 n=6
Total Time 109.82s (± 2.28%) 107.80s (± 1.94%) ~ 106.55s 112.05s p=0.093 n=6
webpack - node (v18.15.0, x64)
Memory used 408,871k (± 0.01%) 408,351k (± 0.02%) -520k (- 0.13%) 408,268k 408,438k p=0.005 n=6
Parse Time 3.89s (± 0.66%) 3.88s (± 0.45%) ~ 3.86s 3.90s p=1.000 n=6
Bind Time 1.69s (± 0.69%) 1.67s (± 0.76%) -0.02s (- 1.09%) 1.66s 1.69s p=0.041 n=6
Check Time 16.84s (± 0.27%) 16.74s (± 0.38%) -0.10s (- 0.61%) 16.64s 16.80s p=0.019 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.42s (± 0.28%) 22.30s (± 0.30%) -0.12s (- 0.53%) 22.18s 22.38s p=0.013 n=6
xstate - node (v18.15.0, x64)
Memory used 513,468k (± 0.01%) 513,258k (± 0.01%) -210k (- 0.04%) 513,185k 513,327k p=0.005 n=6
Parse Time 3.97s (± 0.70%) 3.95s (± 0.39%) ~ 3.93s 3.97s p=0.123 n=6
Bind Time 1.87s (± 0.87%) 1.88s (± 0.65%) ~ 1.86s 1.89s p=0.796 n=6
Check Time 3.42s (± 0.34%) 3.40s (± 0.57%) ~ 3.38s 3.43s p=0.105 n=6
Emit Time 0.09s (±11.12%) 0.10s (± 5.34%) ~ 0.09s 0.10s p=0.120 n=6
Total Time 9.36s (± 0.36%) 9.33s (± 0.32%) ~ 9.29s 9.36s p=0.126 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the top 400 repos comparing main and refs/pull/58013/merge:

Everything looks good!

1 similar comment
@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the top 400 repos comparing main and refs/pull/58013/merge:

Everything looks good!

@ahejlsberg
Copy link
Member Author

@typescript-bot user test this
@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 1, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
user test this ✅ Started 👀 Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the user tests comparing main and refs/pull/58013/merge:

Something interesting changed - please have a look.

Details

webpack

tsconfig.types.json

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,688k (± 0.01%) 295,464k (± 0.01%) -225k (- 0.08%) 295,421k 295,523k p=0.005 n=6
Parse Time 2.65s (± 0.51%) 2.65s (± 0.44%) ~ 2.63s 2.66s p=0.506 n=6
Bind Time 0.83s (± 0.91%) 0.83s (± 0.66%) ~ 0.82s 0.83s p=0.476 n=6
Check Time 8.25s (± 0.21%) 8.22s (± 0.27%) -0.03s (- 0.38%) 8.18s 8.24s p=0.033 n=6
Emit Time 7.04s (± 0.39%) 7.04s (± 0.40%) ~ 7.00s 7.08s p=1.000 n=6
Total Time 18.77s (± 0.14%) 18.73s (± 0.28%) ~ 18.67s 18.79s p=0.514 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,761k (± 0.75%) 192,267k (± 0.77%) -494k (- 0.26%) 191,553k 195,277k p=0.045 n=6
Parse Time 1.36s (± 0.80%) 1.36s (± 1.21%) ~ 1.33s 1.38s p=0.611 n=6
Bind Time 0.73s (± 0.56%) 0.73s (± 0.87%) ~ 0.72s 0.74s p=0.673 n=6
Check Time 9.52s (± 0.33%) 9.40s (± 0.39%) -0.13s (- 1.33%) 9.34s 9.45s p=0.005 n=6
Emit Time 2.63s (± 0.32%) 2.63s (± 0.32%) ~ 2.61s 2.63s p=1.000 n=6
Total Time 14.23s (± 0.24%) 14.11s (± 0.23%) -0.13s (- 0.89%) 14.06s 14.15s p=0.005 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,421k (± 0.01%) 347,106k (± 0.01%) -315k (- 0.09%) 347,086k 347,131k p=0.005 n=6
Parse Time 2.48s (± 0.40%) 2.49s (± 0.30%) ~ 2.48s 2.50s p=0.102 n=6
Bind Time 0.93s (± 0.00%) 0.90s (± 0.45%) 🟩-0.03s (- 3.41%) 0.89s 0.90s p=0.002 n=6
Check Time 7.02s (± 0.18%) 6.99s (± 0.42%) ~ 6.95s 7.02s p=0.073 n=6
Emit Time 4.07s (± 0.26%) 4.07s (± 0.48%) ~ 4.04s 4.09s p=0.625 n=6
Total Time 14.49s (± 0.18%) 14.45s (± 0.18%) ~ 14.42s 14.49s p=0.052 n=6
TFS - node (v18.15.0, x64)
Memory used 302,794k (± 0.01%) 302,267k (± 0.01%) -527k (- 0.17%) 302,248k 302,310k p=0.005 n=6
Parse Time 2.41s (± 0.89%) 2.51s (± 9.80%) ~ 2.38s 3.01s p=1.000 n=6
Bind Time 1.20s (± 0.68%) 1.22s (± 9.40%) ~ 1.16s 1.45s p=0.061 n=6
Check Time 7.51s (± 0.39%) 7.79s (± 9.55%) ~ 7.46s 9.31s p=0.517 n=6
Emit Time 4.25s (± 0.24%) 4.45s (± 9.47%) ~ 4.25s 5.31s p=0.055 n=6
Total Time 15.37s (± 0.25%) 15.96s (± 9.53%) ~ 15.30s 19.07s p=0.574 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,079k (± 0.00%) 509,982k (± 0.00%) -97k (- 0.02%) 509,960k 510,019k p=0.005 n=6
Parse Time 2.66s (± 0.44%) 2.67s (± 0.52%) ~ 2.65s 2.69s p=1.000 n=6
Bind Time 0.99s (± 1.43%) 0.99s (± 0.76%) ~ 0.98s 1.00s p=0.867 n=6
Check Time 17.34s (± 0.26%) 17.20s (± 0.26%) -0.14s (- 0.80%) 17.13s 17.24s p=0.005 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.99s (± 0.23%) 20.86s (± 0.26%) -0.14s (- 0.64%) 20.79s 20.91s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,745,037k (± 0.00%) 1,743,173k (± 0.00%) -1,864k (- 0.11%) 1,743,114k 1,743,220k p=0.005 n=6
Parse Time 9.66s (± 0.40%) 9.67s (± 0.95%) ~ 9.52s 9.79s p=0.747 n=6
Bind Time 3.48s (± 0.47%) 3.46s (± 0.85%) ~ 3.42s 3.49s p=0.183 n=6
Check Time 81.84s (± 0.25%) 81.45s (± 0.74%) ~ 80.98s 82.55s p=0.127 n=6
Emit Time 0.19s (± 2.67%) 0.19s (± 2.67%) ~ 0.19s 0.20s p=1.000 n=6
Total Time 95.18s (± 0.20%) 94.78s (± 0.68%) ~ 94.22s 95.88s p=0.128 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,400,279k (± 0.03%) 2,398,558k (± 0.03%) -1,721k (- 0.07%) 2,397,732k 2,399,674k p=0.008 n=6
Parse Time 6.03s (± 1.45%) 6.01s (± 0.58%) ~ 5.97s 6.06s p=0.127 n=6
Bind Time 2.25s (± 0.83%) 2.16s (± 4.14%) 🟩-0.09s (- 4.07%) 2.04s 2.25s p=0.010 n=6
Check Time 39.95s (± 0.33%) 39.64s (± 0.54%) -0.32s (- 0.79%) 39.37s 39.99s p=0.031 n=6
Emit Time 3.16s (± 1.23%) 3.15s (± 1.02%) ~ 3.12s 3.21s p=0.689 n=6
Total Time 51.43s (± 0.36%) 50.97s (± 0.23%) -0.45s (- 0.88%) 50.78s 51.15s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Memory used 416,186k (± 0.00%) 414,849k (± 0.01%) -1,338k (- 0.32%) 414,810k 414,882k p=0.005 n=6
Parse Time 3.32s (± 0.97%) 3.33s (± 0.97%) ~ 3.30s 3.39s p=0.870 n=6
Bind Time 1.30s (± 0.94%) 1.29s (± 0.58%) ~ 1.28s 1.30s p=0.555 n=6
Check Time 18.18s (± 0.52%) 18.02s (± 0.62%) -0.17s (- 0.93%) 17.83s 18.16s p=0.031 n=6
Emit Time 1.32s (± 1.12%) 1.33s (± 2.37%) ~ 1.28s 1.37s p=0.571 n=6
Total Time 24.12s (± 0.25%) 23.96s (± 0.30%) -0.15s (- 0.64%) 23.84s 24.06s p=0.010 n=6
vscode - node (v18.15.0, x64)
Memory used 2,899,027k (± 0.00%) 2,894,209k (± 0.01%) -4,818k (- 0.17%) 2,893,915k 2,894,332k p=0.005 n=6
Parse Time 12.89s (± 0.15%) 12.88s (± 0.35%) ~ 12.83s 12.95s p=0.872 n=6
Bind Time 4.11s (± 0.72%) 4.12s (± 0.50%) ~ 4.08s 4.14s p=0.627 n=6
Check Time 71.51s (± 0.37%) 70.78s (± 0.55%) -0.73s (- 1.02%) 70.31s 71.33s p=0.013 n=6
Emit Time 19.27s (± 0.61%) 19.83s (± 7.94%) ~ 19.05s 23.04s p=0.688 n=6
Total Time 107.77s (± 0.24%) 107.61s (± 1.74%) ~ 106.45s 111.38s p=0.066 n=6
webpack - node (v18.15.0, x64)
Memory used 408,844k (± 0.01%) 408,298k (± 0.01%) -546k (- 0.13%) 408,223k 408,324k p=0.005 n=6
Parse Time 3.89s (± 0.82%) 3.90s (± 0.66%) ~ 3.87s 3.94s p=0.871 n=6
Bind Time 1.69s (± 0.79%) 1.67s (± 0.96%) ~ 1.66s 1.70s p=0.087 n=6
Check Time 16.82s (± 0.23%) 16.76s (± 0.20%) -0.06s (- 0.35%) 16.72s 16.80s p=0.017 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.40s (± 0.19%) 22.33s (± 0.23%) ~ 22.26s 22.40s p=0.054 n=6
xstate - node (v18.15.0, x64)
Memory used 513,504k (± 0.02%) 513,284k (± 0.01%) -220k (- 0.04%) 513,237k 513,429k p=0.008 n=6
Parse Time 3.96s (± 0.70%) 3.95s (± 0.35%) ~ 3.93s 3.96s p=0.370 n=6
Bind Time 1.89s (± 1.41%) 1.87s (± 1.05%) ~ 1.84s 1.89s p=0.466 n=6
Check Time 3.44s (± 0.77%) 3.39s (± 0.40%) -0.05s (- 1.40%) 3.38s 3.41s p=0.005 n=6
Emit Time 0.08s (± 6.19%) 0.09s (± 8.80%) 🔻+0.01s (+14.00%) 0.08s 0.10s p=0.022 n=6
Total Time 9.38s (± 0.68%) 9.30s (± 0.28%) -0.08s (- 0.84%) 9.27s 9.33s p=0.036 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@ahejlsberg
Copy link
Member Author

ahejlsberg commented Apr 2, 2024

@jakebailey Any idea why we're seeing the test failure here #58013 (comment)? I can't reproduce it locally. Meaning, when I run tsc locally in a cloned repository, I never get the error, be it on main or my branch.

@ahejlsberg
Copy link
Member Author

@jakebailey Ok, I wasn't running with -p tsconfig.types.json, but now when I do, I get the error both in main and in my branch. I just can't reproduce any sort of difference.

@jakebailey
Copy link
Member

I was out yesterday; will take a look at this shortly.

@jakebailey
Copy link
Member

jakebailey commented Apr 2, 2024

Cloning webpack, doing yarn, then running tsc -p ./tsconfig.types.json on main and with this PR does produce a difference:

$ git clone git@github.com:webpack/webpack.git
$ cd webpack
$ yarn
$ node ~/work/TypeScript/built/local-old/tsc.js -p tsconfig.types.json |& tee ~/out-main.txt
$ node ~/work/TypeScript/built/local/tsc.js -p tsconfig.types.json |& tee ~/out-pr.txt
$ wc -l ~/out-main.txt ~/out-pr.txt
  3188 /home/jabaile/out-main.txt
  3187 /home/jabaile/out-pr.txt
  6375 total
$ gdiff ~/out-main.txt ~/out-pr.txt
diff --git a/home/jabaile/out-main.txt b/home/jabaile/out-pr.txt
index c825c4baf..0b70c6f9f 100644
--- a/home/jabaile/out-main.txt
+++ b/home/jabaile/out-pr.txt
@@ -2004,7 +2004,6 @@ lib/optimize/SplitChunksPlugin.js(1011,30): error TS7006: Parameter 'key' implic
 lib/optimize/SplitChunksPlugin.js(1022,37): error TS7006: Parameter 'key' implicitly has an 'any' type.
 lib/optimize/SplitChunksPlugin.js(1095,35): error TS18048: 'cacheGroup.minChunks' is possibly 'undefined'.
 lib/optimize/SplitChunksPlugin.js(1099,9): error TS2722: Cannot invoke an object which is possibly 'undefined'.
-lib/optimize/SplitChunksPlugin.js(1099,9): error TS18048: 'cacheGroup.getName' is possibly 'undefined'.
 lib/optimize/SplitChunksPlugin.js(1254,21): error TS18048: 'cacheGroup.minChunks' is possibly 'undefined'.
 lib/optimize/SplitChunksPlugin.js(1687,37): error TS7006: Parameter 'value' implicitly has an 'any' type.
 lib/optimize/SplitChunksPlugin.js(1687,44): error TS7006: Parameter 'key' implicitly has an 'any' type.

This difference appears no matter what order I run things or the clean-ness of the repo. So, I think the bot is correct here.

@jakebailey
Copy link
Member

jakebailey commented Apr 2, 2024

Also, I modified this PR to force hasFlowEffects to always be true, and the error count went back to the one on main, so it's definitely some behavior change in the PR (somehow).

Trial and error says that it's something to do with the = false in bindConditionalExpressionFlow; forcing that one to true also "fixes" the error, so maybe that's where it's diverging, somehow... I haven't had time to try and cut it down further.

@ahejlsberg
Copy link
Member Author

@jakebailey Yeah, sorry, I see what's going on now. What confused me is that we used to issue two errors, and now we only issue one, but the errors have the exact same location. The missing error is an effect of optimizing away part of the control flow graph, causing us to not call getEffectsSignature, the method that resolves signatures in control flow analysis. For a variety of reasons we can't use the regular (and more complex) signature resolution in CFA, and the CFA resolution reports nullable errors slightly differently. Anyway, the error that went away is just a (differently worded) duplicate, so there's no problem here.

@ahejlsberg
Copy link
Member Author

Now that we've sorted out the missing error, I think this is ready for a review.

@ahejlsberg ahejlsberg merged commit cf4bb58 into main Apr 4, 2024
25 checks passed
@ahejlsberg ahejlsberg deleted the optimizeFlowGraph branch April 4, 2024 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants