build, win: make LTCG optional #21186

bzoz · 2018-06-07T09:14:00Z

Add noltcg option to vcbuild that disables Link Time Code Generation. While clean build is only slightly faster (19 min vs 20), rebuild after change in one of the js files now takes 20 seconds instead of 2 minutes.

/cc @nodejs/platform-windows

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows commit guidelines

bzoz · 2018-06-07T09:15:06Z

CI: https://ci.nodejs.org/job/node-test-pull-request/15309/

For reference, there was also similar PR for Linux: #7408

targos · 2018-06-07T09:18:16Z

I can't review the code changes but +1000 on making Windows rebuilds faster!

addaleax

Thank You.

We may want to advertise this in BUILDING.md as well, I think.

seishun · 2018-06-07T12:47:20Z

How does it affect the performance of the resulting binary?

Fishrock123 · 2018-06-07T13:17:17Z

To be clear, our windows releases should use --with-ltcg now, right?

bzoz · 2018-06-07T13:33:15Z

@seishun: I did not check that yet, but surely there is some
@Fishrock123: yes, but it is on by default

refack · 2018-06-07T13:37:45Z

configure

+parser.add_option('--with-ltcg',
+    action='store_true',
+    dest='with_ltcg',
+    help='Use Link Time Code Generation. This feature is only available on Windows.')


I would suggest flipping the switch to --disable-windows-ltcg (and use store-_false) similar to:
https://github.com/nodejs/node/blob/23efa8cc0692e690fd5138f5b93ccfd4e47c2f58/configure#L509-L517

I wanted to keep this in line with Linux - there was PR to add --with-lto, at some point, when it will be ready that can be merged into this option

The difference is that if we want LTCG to be the default, it makes more sense to have an opt-out --without flag so the call to python configure could stay simple.

BTW I meant to refeance: https://github.com/nodejs/node/blob/23efa8cc0692e690fd5138f5b93ccfd4e47c2f58/configure#L525-L528

refack

A very blessed optimization.
But I would suggest flipping the switch semantics.

refack · 2018-06-07T13:38:53Z

common.gypi

@@ -16,6 +16,7 @@
    'node_use_v8_platform%': 'true',
    'node_use_bundled_v8%': 'true',
    'node_module_version%': '',
+    'node_with_ltcg%': '',


IMHO default should be 'true'

refack · 2018-06-07T13:41:59Z

vcbuild.bat

@@ -15,6 +15,7 @@ cd %~dp0
 set config=Release
 set target=Build
 set target_arch=x64
+set ltcg=1


Again IMHO switch should be flipped (since default should stay to enable LTCG).

I mean it should be set no-ltcg=0

richardlau · 2018-06-07T20:25:48Z

Since this touches common.gypi how will this affect addons built with node-gyp? There's a long standing node-gyp issue to move away from using core's common.gypi (nodejs/node-gyp#1118) but for now it will be used.

For example, nodejs/node-gyp#1457 was recently opened regarding compiling addons for a different version of Node.js than was used to run node-gyp.

addaleax · 2018-06-10T20:37:20Z

@refack Setting no LTCG as the default is definitely the more developer-friendly thing to do here … we should just make sure to keep it enabled in CI?

refack · 2018-06-10T22:15:06Z

Setting no LTCG as the default is definitely the more developer-friendly thing to do here … we should just make sure to keep it enabled in CI?

I agree, that's why I prefer to have an opt-out switch (LTCG on by default, switch to turn it off).
I usually call configure manually so it will make a difference, while for those using vcbuild I agree it doesn't really matter.

refack · 2018-06-10T22:22:52Z

Also if we turn this into an opt-out switch, we could keep vcbuild.bat unchanged and use the environment variable config_flags

set config_flags=--without-ltcg
vcbuild.bat

joyeecheung

FWIW, I am also +1 on disabling LTO on Windows by default as long as we make sure it is enabled in CI.

bzoz · 2018-06-11T18:44:05Z

@refack: how about making noltcg the default for vcbuild (but keeping LTCG on for build-release)?

bzoz · 2018-06-12T13:43:07Z

Rebased on master.

bzoz · 2018-06-13T16:36:18Z

Updated: made LTCG disabled by default. It will be enabled for build-release builds.

joaocgreis · 2018-06-13T17:32:49Z

Test CI uses vcbuild release, so LTCG needs to be enabled on that case as well.

refack · 2018-06-13T19:06:02Z

how about making noltcg the default for vcbuild (but keeping LTCG on for build-release)?

I'm Ok with that but it might have higher semver-ity. Definitely minor for a new option, maybe major since we are changing default behaviour.

There is a tradeoff here

Faster link time (which is minutes in 100% CPU)
on the other hand
We are creating two tiers of non-debug binaries: Release and Release+LTCG. Which one should be tested in CI

/CC @nodejs/platform-windows @nodejs/build Any opinions?

richardlau · 2018-06-13T19:10:23Z

We are creating two tiers of non-debug binaries: Release and Release+LTCG. Which one should be tested in CI

Whichever we end up using for the shipped binaries (i.e., Release+LTCG), given that at the moment we don't run tests on the release builds.

bzoz · 2018-06-14T08:57:12Z

Made LTCG enabled for release builds. CI will compile and use Node with LTCG enabled, same as published binary.

bzoz · 2018-06-20T23:20:21Z

@refack, @bnoordhuis, @jasnell and @joyeecheung - this was changed since your reviews. Now, LTCG is off by default, enabled only for CI testing and releases. Is this ok with you?

bzoz · 2018-06-26T08:48:29Z

CI: https://ci.nodejs.org/job/node-test-pull-request/15628/

Disables Link Time Code Generation by default. Adds ‘ltcg’ vcbuild option to enable it. LTCG will be used by default by release and CI builds.

bzoz · 2018-06-26T17:48:16Z

Rebased, new CI: https://ci.nodejs.org/job/node-test-pull-request/15636/

bzoz · 2018-06-27T08:59:38Z

Green CI: https://ci.nodejs.org/job/node-test-commit/19488/

bzoz · 2018-06-27T09:01:26Z

Landed in 7c45284

Disables Link Time Code Generation by default. Adds ‘ltcg’ vcbuild option to enable it. LTCG will be used by default by release and CI builds. PR-URL: #21186 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

nodejs-github-bot added build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform. labels Jun 7, 2018

addaleax approved these changes Jun 7, 2018

View reviewed changes

bnoordhuis approved these changes Jun 7, 2018

View reviewed changes

refack reviewed Jun 7, 2018

View reviewed changes

refack suggested changes Jun 7, 2018

View reviewed changes

jasnell approved these changes Jun 10, 2018

View reviewed changes

joyeecheung approved these changes Jun 11, 2018

View reviewed changes

bzoz force-pushed the bartek-optional-ltcg branch from 23efa8c to d7f9e50 Compare June 12, 2018 13:42

refack approved these changes Jun 13, 2018

View reviewed changes

refack added semver-minor PRs that contain new features and should be released in the next minor version. discuss Issues opened for discussions and feedbacks. blocked PRs that are blocked by other issues or PRs. labels Jun 13, 2018

joyeecheung approved these changes Jun 21, 2018

View reviewed changes

build, win: make LTCG optional

f2a6b28

Disables Link Time Code Generation by default. Adds ‘ltcg’ vcbuild option to enable it. LTCG will be used by default by release and CI builds.

bzoz force-pushed the bartek-optional-ltcg branch from fd6cf18 to f2a6b28 Compare June 26, 2018 17:45

bzoz closed this Jun 27, 2018

targos mentioned this pull request Jul 3, 2018

v10.6.0 proposal #21629

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build, win: make LTCG optional #21186

build, win: make LTCG optional #21186

bzoz commented Jun 7, 2018

bzoz commented Jun 7, 2018

targos commented Jun 7, 2018

addaleax left a comment

seishun commented Jun 7, 2018

Fishrock123 commented Jun 7, 2018

bzoz commented Jun 7, 2018

refack Jun 7, 2018 •

edited

Loading

bzoz Jun 7, 2018

refack Jun 10, 2018

refack left a comment

refack Jun 7, 2018

refack Jun 7, 2018

refack Jun 10, 2018

richardlau commented Jun 7, 2018

addaleax commented Jun 10, 2018 •

edited

Loading

refack commented Jun 10, 2018

refack commented Jun 10, 2018

joyeecheung left a comment

bzoz commented Jun 11, 2018

bzoz commented Jun 12, 2018

bzoz commented Jun 13, 2018

joaocgreis commented Jun 13, 2018

refack commented Jun 13, 2018

richardlau commented Jun 13, 2018

bzoz commented Jun 14, 2018

bzoz commented Jun 20, 2018

bzoz commented Jun 26, 2018

bzoz commented Jun 26, 2018

bzoz commented Jun 27, 2018

bzoz commented Jun 27, 2018

build, win: make LTCG optional #21186

build, win: make LTCG optional #21186

Conversation

bzoz commented Jun 7, 2018

Checklist

bzoz commented Jun 7, 2018

targos commented Jun 7, 2018

addaleax left a comment

Choose a reason for hiding this comment

seishun commented Jun 7, 2018

Fishrock123 commented Jun 7, 2018

bzoz commented Jun 7, 2018

refack Jun 7, 2018 • edited Loading

Choose a reason for hiding this comment

bzoz Jun 7, 2018

Choose a reason for hiding this comment

refack Jun 10, 2018

Choose a reason for hiding this comment

refack left a comment

Choose a reason for hiding this comment

refack Jun 7, 2018

Choose a reason for hiding this comment

refack Jun 7, 2018

Choose a reason for hiding this comment

refack Jun 10, 2018

Choose a reason for hiding this comment

richardlau commented Jun 7, 2018

addaleax commented Jun 10, 2018 • edited Loading

refack commented Jun 10, 2018

refack commented Jun 10, 2018

joyeecheung left a comment

Choose a reason for hiding this comment

bzoz commented Jun 11, 2018

bzoz commented Jun 12, 2018

bzoz commented Jun 13, 2018

joaocgreis commented Jun 13, 2018

refack commented Jun 13, 2018

richardlau commented Jun 13, 2018

bzoz commented Jun 14, 2018

bzoz commented Jun 20, 2018

bzoz commented Jun 26, 2018

bzoz commented Jun 26, 2018

bzoz commented Jun 27, 2018

bzoz commented Jun 27, 2018

refack Jun 7, 2018 •

edited

Loading

addaleax commented Jun 10, 2018 •

edited

Loading