Enable Maglev

[kvakil]

What is the problem this feature will solve?

V8 recently introduced their new Maglev compiler which is a mid-tier compiler (blog post, design doc).

We currently do not compile or ship Maglev.

What is the feature you are proposing to solve the problem?

Build and enable Maglev. Proposed plan:

1. Get Maglev to actually compile with our settings.
2. Run some benchmarks and post them here.
3. Add a configuration option to compile with Maglev enabled.
4. Upstream V8 patches to get Maglev compilation working.
5. Cherry-pick upstream patches back to this repo.
6. Flip the configuration flag to default-on and ship Maglev.

Open questions

1. Do we want to ship this?
2. Should turning the configuration flag to default-on be semver major? Or maybe it only needs to be dont-land-on-v20.x, and we can ship it as part of v21?

What alternatives have you considered?

We could also not enable Maglev, which is a valid choice.

Pros of shipping:

• Will likely be faster?
• Gets us closer to Google Chrome which ships Maglev.

Cons of shipping:

• As a mid-tier compiler Maglev may not bring much performance benefit for long-running server-like workloads.
• Enabling Maglev may cause performance regressions for users.
  • Users can disable Maglev with the V8 option --no-maglev but it may be difficult to figure out that Maglev is causing the regression.
• Potentially leads to more maintenance burden from V8 changes.

[kvakil]

I ran a random subset of 82 benchmark files from the repo, once with maglev and once without. Overall the results from this sample look positive for Maglev, of the "statistically significant" changes:

• assert/ suite generally positive.
• async_hooks/ suite generally positive.
• blob/ suite positive.
• buffers/ suite positive.
• events/ suite negative.
• path/ suite positive.
• streams/ suite generally positive.
• url/ suite generally positive.
• validators/ suite positive.

Here are the raw CSVs from benchmark/compare.js: results.csv.tar.gz, if you want to look at specific benchmarks. The benchmark suite is very slow, so I'm loathe to run the entire thing, but let me know if there are other benchmarks people would find informative.

[joyeecheung]

It seems the positive ones are mostly microbenchmarks that probably are not written in a way to eliminate compilation speed out of the measurement. So if it's Maglev kicking in instead of Turbofan, that might make a difference in a micro-benchmarking loop, but then in real world applications users don't usually run one API in a loop so that when or which compiler kicks in makes a significant difference. We should probably look into some more real-world CLIs for the measurement (e.g. #50684), where a mid-tier optimizing compiler could make a meaningful difference (A significant part of CLI startup is compilation. At the same time, Node.js core could also hit many hot paths trying to complete the loading process).

[targos]

Gets us closer to Google Chrome which ships Maglev.

Can you show where it's enabled in Chrome's build config?

[kvakil]

Gets us closer to Google Chrome which ships Maglev.

Can you show where it's enabled in Chrome's build config?

I don't know enough about Chrome's setup to answer that precisely. You can run Chrome with --js-flags=--help and see that --maglev is enabled by default. It likely comes from this default in v8's BUILD.gn:

node/deps/v8/BUILD.gn

Lines 505 to 508 in fc2862b

	if (v8_enable_maglev == "") {
	v8_enable_maglev = v8_enable_turbofan &&
	(v8_current_cpu == "arm" || v8_current_cpu == "x64" ||
	v8_current_cpu == "arm64")

[kvakil]

It seems the positive ones are mostly microbenchmarks that probably are not written in a way to eliminate compilation speed out of the measurement. So if it's Maglev kicking in instead of Turbofan, that might make a difference in a micro-benchmarking loop, but then in real world applications users don't usually run one API in a loop so that when or which compiler kicks in makes a significant difference. We should probably look into some more real-world CLIs for the measurement (e.g. #50684), where a mid-tier optimizing compiler could make a meaningful difference (A significant part of CLI startup is compilation. At the same time, Node.js core could also hit many hot paths trying to complete the loading process).

I tried the startup benchmarks in that PR and there wasn't a significant difference with Maglev on/off unfortunately.

I guess what we need here is more information on "for each benchmark, which compilation tier does it end up in?" It is possible that some of them never reach Turbofan at all. We could a detailed analysis like this one, but it sounds like you think it would be better to try to find representative macro-benchmarks rather than continuing to look at the microbenchmarks.

[kvakil]

On a small Typescript build job, it looks like --maglev is ~7-10% faster than --no-maglev:

--maglev
        2.77 real         5.75 user         0.43 sys
        2.79 real         5.75 user         0.44 sys
        2.78 real         5.80 user         0.47 sys
        2.80 real         5.74 user         0.46 sys
        2.79 real         5.69 user         0.44 sys
        2.81 real         5.76 user         0.45 sys

--no-maglev
        2.98 real         6.30 user         0.39 sys
        3.00 real         6.27 user         0.41 sys
        3.01 real         6.27 user         0.40 sys
        3.02 real         6.26 user         0.41 sys
        3.05 real         6.28 user         0.42 sys
        3.01 real         6.25 user         0.40 sys

I can't share the repo unfortunately, but will try to run typescript on some public repos to see if it generalizes.

[kvakil]

Trying more Typescript projects. I chose xstate which is one the libraries that Typescript benchmarks for performance regressions against. I compiled node with v8-lkgr.

$ git clone https://github.com/statelyai/xstate.git
$ yarn install

and then I timed a typecheck after a clean:

$ node node_modules/.bin/tsc --build --clean && \time node node_modules/.bin/tsc --build

10 iterations of tsc --build with --maglev:

        3.90 real         7.33 user         0.32 sys                                                      
        3.87 real         7.29 user         0.30 sys                                                      
        3.87 real         7.30 user         0.30 sys                                                      
        3.91 real         7.34 user         0.31 sys                                                      
        3.90 real         7.30 user         0.29 sys                                                      
        3.87 real         7.28 user         0.31 sys                                                      
        3.89 real         7.31 user         0.30 sys                                                      
        3.92 real         7.35 user         0.31 sys                                                      
        3.86 real         7.27 user         0.31 sys                                                                                                                                                                
        4.03 real         7.49 user         0.31 sys                                                      

10 iterations of --no-maglev:

        4.79 real         8.52 user         0.29 sys 
        4.80 real         8.53 user         0.30 sys
        4.82 real         8.53 user         0.29 sys 
        4.87 real         8.63 user         0.29 sys                                                                                                                                                                
        4.79 real         8.53 user         0.29 sys                                                                                                                                                                
        4.82 real         8.55 user         0.29 sys                                                                                                                                                                
        4.92 real         8.66 user         0.31 sys                                                                                                                                                                
        4.81 real         8.52 user         0.30 sys                                                                                                                                                                
        4.81 real         8.53 user         0.30 sys                                                                                                                                                                
        4.79 real         8.52 user         0.29 sys                                                                                                                                                                

20% improvement in wall-clock and 14% improvement in CPU-time.

[targos]

Gets us closer to Google Chrome which ships Maglev.

Can you show where it's enabled in Chrome's build config?

I don't know enough about Chrome's setup to answer that precisely. You can run Chrome with --js-flags=--help and see that --maglev is enabled by default. It likely comes from this default in v8's BUILD.gn:

node/deps/v8/BUILD.gn

Lines 505 to 508 in fc2862b

	if (v8_enable_maglev == "") {
	v8_enable_maglev = v8_enable_turbofan &&
	(v8_current_cpu == "arm" || v8_current_cpu == "x64" ||
	v8_current_cpu == "arm64")

OK. I'm fine with the proposal as long as we enable it by default on the same architectures as V8.

[kvakil]

OK. I'm fine with the proposal as long as we enable it by default on the same architectures as V8.

Sure. Do you have an opinion on whether enabling it by default can happen in a minor release or if that should be a semver-major change?

[kvakil]

Another benchmark: eslint on specific directories of the Node.js codebase. Overall marginally (i.e. 2-4%) faster with maglev than without:

$ hyperfine -L maglev --maglev,--no-maglev './node_maglev {maglev} tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives  lib'
Benchmark 1: ./node_maglev --maglev tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives  lib
  Time (mean ± σ):     12.885 s ±  0.271 s    [User: 17.605 s, System: 0.775 s]
  Range (min … max):   12.578 s … 13.384 s    10 runs
 
Benchmark 2: ./node_maglev --no-maglev tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives  lib
  Time (mean ± σ):     13.105 s ±  0.349 s    [User: 18.707 s, System: 0.688 s]
  Range (min … max):   12.762 s … 13.838 s    10 runs
 
Summary
  './node_maglev --maglev tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives  lib' ran
    1.02 ± 0.03 times faster than './node_maglev --no-maglev tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives  lib'
$ hyperfine -L maglev --maglev,--no-maglev './node_maglev {maglev} tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives benchmark'
Benchmark 1: ./node_maglev --maglev tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives benchmark
  Time (mean ± σ):      3.792 s ±  0.050 s    [User: 5.603 s, System: 0.351 s]
  Range (min … max):    3.716 s …  3.891 s    10 runs
 
Benchmark 2: ./node_maglev --no-maglev tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives benchmark
  Time (mean ± σ):      3.943 s ±  0.061 s    [User: 5.899 s, System: 0.297 s]
  Range (min … max):    3.877 s …  4.055 s    10 runs
 
Summary
  './node_maglev --maglev tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives benchmark' ran
    1.04 ± 0.02 times faster than './node_maglev --no-maglev tools/node_modules/eslint/bin/eslint.js --max-warnings=0 --report-unused-disable-directives benchmark'

[Talent30]

As a mid-tier compiler Maglev may not bring much performance benefit for long-running server-like workloads.

The middle-tier compiler definitely has a positive impact on JS development tools(CLI apps).