Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: always compile and store code cache for native modules #24950

Closed
wants to merge 4 commits into from

Conversation

@joyeecheung
Copy link
Member

joyeecheung commented Dec 10, 2018

src: remove code cache integrity check

In preparation of sharing code cache among different threads -
we simply rely on v8 to reject invalid cache, since there isn't
any serious consequence when the cache is invalid anyway.

src: always compile and store code cache for native modules

This patch changes the NativeModuleLoader to always try to find
code cache for native modules when it compiles them, and always
produce and store the code cache after compilation. The cache
map is protected by a mutex and can be accessed by different
threads - including the worker threads and the main thread. Hence any
thread can reuse the code cache if the native module has already
been compiled by another thread - in particular the cache of the
bootstrappers and per_context.js will always be hit when a new thread
is spun.

This results in a ~6% startup overhead in the worst case
(when only the main thread is launched without requiring any additional
native module - it now needs to do the extra work of finding and
storing caches), which balances out the recent improvements by moving
the compilation to C++, but it also leads to a ~60% improvement in
the best case (when a worker thread is spun and requires a lot of native
modules thus hitting the cache compiled by the main thread).

Local benchmark results:

                                                                                   confidence improvement accuracy (*)   (**)  (***)
 misc/startup.js mode='process' script='benchmark/fixtures/require-cachable' dur=1        ***     -6.80 %       ±0.77% ±1.02% ±1.34%
 misc/startup.js mode='process' script='test/fixtures/semicolon' dur=1                    ***     -4.90 %       ±0.82% ±1.10% ±1.43%
 misc/startup.js mode='worker' script='benchmark/fixtures/require-cachable' dur=1         ***     63.13 %       ±1.58% ±2.12% ±2.79%
 misc/startup.js mode='worker' script='test/fixtures/semicolon' dur=1                     ***     56.20 %       ±1.33% ±1.78% ±2.33%
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines
joyeecheung added 2 commits Dec 3, 2018
In preparation of sharing code cache among different threads -
we simply rely on v8 to reject invalid cache, since there isn't
any serious consequence when the cache is invalid anyway.
This patch changes the NativeModuleLoader to always try to find
code cache for native modules when it compiles them, and always
produce and store the code cache after compilation. The cache
map is protected by a mutex and can be accessed by different
threads - including the worker threads and the main thread. Hence any
thread can reuse the code cache if the native module has already
been compiled by another thread - in particular the cache of the
bootstrappers and per_context.js will always be hit when a new thread
is spun.

This results in a ~6% startup overhead in the worst case
(when only the main thread is launched without requiring any additional
native module - it now needs to do the extra work of finding and
storing caches), which balances out the recent improvements by moving
the compilation to C++, but it also leads to a ~60% improvement in
the best case (when a worker thread is spun and requires a lot of native
modules thus hitting the cache compiled by the main thread).
src/node_native_module.cc Show resolved Hide resolved
src/node_native_module.cc Show resolved Hide resolved
src/node_native_module.cc Outdated Show resolved Hide resolved
@joyeecheung joyeecheung force-pushed the joyeecheung:cache-worker branch from 022a0b3 to fe44133 Dec 11, 2018
@joyeecheung joyeecheung force-pushed the joyeecheung:cache-worker branch from fe44133 to bdf589b Dec 11, 2018
…ules
@joyeecheung

This comment has been minimized.

Copy link
Member Author

joyeecheung commented Dec 11, 2018

@addaleax Thanks for the review, updated. Also updated the emplace() calls since gcc 4.9.4 needs the pointers to be moved before the call, and updated the test to pass the --worker test suit since compileWithCache is no longer guaranteed to be empty when a worker thread is spun (that's the whole point of this patch :p)

PTAL.

Also cc @nodejs/process @nodejs/build-files

CI: https://ci.nodejs.org/job/node-test-pull-request/19412/
Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/282/

@joyeecheung

This comment has been minimized.

Copy link
Member Author

joyeecheung commented Dec 12, 2018

Removing author-ready because this needs at least one more approval to land before next week.

From the previous benchmark CI:

                                                                                   confidence improvement accuracy (*)   (**)  (***)
 misc/startup.js mode='process' script='benchmark/fixtures/require-cachable' dur=1        ***     -7.38 %       ±1.09% ±1.46% ±1.90%
 misc/startup.js mode='process' script='test/fixtures/semicolon' dur=1                    ***     -5.12 %       ±0.79% ±1.06% ±1.37%
 misc/startup.js mode='worker' script='benchmark/fixtures/require-cachable' dur=1         ***     54.77 %       ±1.42% ±1.91% ±2.50%
 misc/startup.js mode='worker' script='test/fixtures/semicolon' dur=1                     ***     48.22 %       ±3.27% ±4.40% ±5.82%

Resume CI: https://ci.nodejs.org/job/node-test-pull-request/19454/

@BridgeAR

This comment has been minimized.

Copy link
Member

BridgeAR commented Dec 12, 2018

@joyeecheung the label is not meant as ready to land but the author is done. Missing LGs are absolutely fine while having the author ready label applied :-)

@joyeecheung

This comment has been minimized.

Copy link
Member Author

joyeecheung commented Dec 12, 2018

@BridgeAR

the label is not meant as ready to land but the author is done. Missing LGs are absolutely fine while having the author ready label applied :-)

That seems to contradict the label description:

PRs that have at least two approvals, no pending requests for changes, and a CI started.

And the collaborator guide:

at least two Collaborators approved the PR (one Collaborator approval is enough if the pull request has been open for more than 7 days).

@BridgeAR

This comment has been minimized.

Copy link
Member

BridgeAR commented Dec 13, 2018

@joyeecheung

That seems to contradict the label description:

The label was not up to date. It's fixed now.

And the collaborator guide:

Thanks! Seems like I missed that in #24893. I am opening a PR to fix that right away.

@joyeecheung

This comment has been minimized.

Copy link
Member Author

joyeecheung commented Dec 18, 2018

@joyeecheung

This comment has been minimized.

Copy link
Member Author

joyeecheung commented Dec 18, 2018

Landed in ceb6635 and 0858e5d

joyeecheung added a commit that referenced this pull request Dec 18, 2018
In preparation of sharing code cache among different threads -
we simply rely on v8 to reject invalid cache, since there isn't
any serious consequence when the cache is invalid anyway.

PR-URL: #24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
joyeecheung added a commit that referenced this pull request Dec 18, 2018
This patch changes the NativeModuleLoader to always try to find
code cache for native modules when it compiles them, and always
produce and store the code cache after compilation. The cache
map is protected by a mutex and can be accessed by different
threads - including the worker threads and the main thread. Hence any
thread can reuse the code cache if the native module has already
been compiled by another thread - in particular the cache of the
bootstrappers and per_context.js will always be hit when a new thread
is spun.

This results in a ~6% startup overhead in the worst case
(when only the main thread is launched without requiring any additional
native module - it now needs to do the extra work of finding and
storing caches), which balances out the recent improvements by moving
the compilation to C++, but it also leads to a ~60% improvement in
the best case (when a worker thread is spun and requires a lot of native
modules thus hitting the cache compiled by the main thread).

PR-URL: #24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@MylesBorins

This comment has been minimized.

Copy link
Member

MylesBorins commented Dec 25, 2018

This does not land cleanly on v11.x, would someone be willing to backport?

@targos targos added this to Backport requested in v11.x Dec 28, 2018
@BridgeAR

This comment has been minimized.

Copy link
Member

BridgeAR commented Jan 9, 2019

addaleax added a commit that referenced this pull request Jan 14, 2019
In preparation of sharing code cache among different threads -
we simply rely on v8 to reject invalid cache, since there isn't
any serious consequence when the cache is invalid anyway.

PR-URL: #24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
addaleax added a commit that referenced this pull request Jan 14, 2019
This patch changes the NativeModuleLoader to always try to find
code cache for native modules when it compiles them, and always
produce and store the code cache after compilation. The cache
map is protected by a mutex and can be accessed by different
threads - including the worker threads and the main thread. Hence any
thread can reuse the code cache if the native module has already
been compiled by another thread - in particular the cache of the
bootstrappers and per_context.js will always be hit when a new thread
is spun.

This results in a ~6% startup overhead in the worst case
(when only the main thread is launched without requiring any additional
native module - it now needs to do the extra work of finding and
storing caches), which balances out the recent improvements by moving
the compilation to C++, but it also leads to a ~60% improvement in
the best case (when a worker thread is spun and requires a lot of native
modules thus hitting the cache compiled by the main thread).

PR-URL: #24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@addaleax

This comment has been minimized.

Copy link
Member

addaleax commented Jan 14, 2019

Lands cleanly now 👍

refack added a commit to refack/node that referenced this pull request Jan 14, 2019
In preparation of sharing code cache among different threads -
we simply rely on v8 to reject invalid cache, since there isn't
any serious consequence when the cache is invalid anyway.

PR-URL: nodejs#24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
refack added a commit to refack/node that referenced this pull request Jan 14, 2019
This patch changes the NativeModuleLoader to always try to find
code cache for native modules when it compiles them, and always
produce and store the code cache after compilation. The cache
map is protected by a mutex and can be accessed by different
threads - including the worker threads and the main thread. Hence any
thread can reuse the code cache if the native module has already
been compiled by another thread - in particular the cache of the
bootstrappers and per_context.js will always be hit when a new thread
is spun.

This results in a ~6% startup overhead in the worst case
(when only the main thread is launched without requiring any additional
native module - it now needs to do the extra work of finding and
storing caches), which balances out the recent improvements by moving
the compilation to C++, but it also leads to a ~60% improvement in
the best case (when a worker thread is spun and requires a lot of native
modules thus hitting the cache compiled by the main thread).

PR-URL: nodejs#24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@BridgeAR BridgeAR moved this from Backport requested to Backported in v11.x Jan 14, 2019
@BridgeAR BridgeAR mentioned this pull request Jan 16, 2019
BridgeAR added a commit to BridgeAR/node that referenced this pull request Jan 16, 2019
In preparation of sharing code cache among different threads -
we simply rely on v8 to reject invalid cache, since there isn't
any serious consequence when the cache is invalid anyway.

PR-URL: nodejs#24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
BridgeAR added a commit to BridgeAR/node that referenced this pull request Jan 16, 2019
This patch changes the NativeModuleLoader to always try to find
code cache for native modules when it compiles them, and always
produce and store the code cache after compilation. The cache
map is protected by a mutex and can be accessed by different
threads - including the worker threads and the main thread. Hence any
thread can reuse the code cache if the native module has already
been compiled by another thread - in particular the cache of the
bootstrappers and per_context.js will always be hit when a new thread
is spun.

This results in a ~6% startup overhead in the worst case
(when only the main thread is launched without requiring any additional
native module - it now needs to do the extra work of finding and
storing caches), which balances out the recent improvements by moving
the compilation to C++, but it also leads to a ~60% improvement in
the best case (when a worker thread is spun and requires a lot of native
modules thus hitting the cache compiled by the main thread).

PR-URL: nodejs#24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@MylesBorins MylesBorins mentioned this pull request Jan 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
v11.x
  
Backported
6 participants
You can’t perform that action at this time.