Upgrade to OpenSSL-1.0.2l for Node-v4 #13696

shigeki · 2017-06-15T10:43:30Z

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

deps/openssl

This PR is upgrading to OpenSSL-1.0.2l agains Node-v4. It includes a series of cherry-picks as 72785dd..d63ff23 in the master for upgrading OpenSSL-1.0.2l of #13233 and doc fix of #13234. It also fixes #12691.

CC @MylesBorins @gibfahn

* The following function from <unicode/normlzr.h> is used: normalize() * Until ICU 59, <unicode/normlzr.h> is indirectly included, but this changed with the 59 release. Adding this header has been the right thing to do for many years, so it is backwards compatible and fix compilation with recent ICU. Refs: nodejs#13022 PR-URL: nodejs#13040 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

When upgrading OpenSSL, Step 6 in upgrading guide explains the steps that need to be taken if asm files need updating. This might not always be the case and something that needs to be checked from release to release. This commit adds an example of using github to manually compare two tags to see if any changes were made to asm files. PR-URL: nodejs#13234 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

This replaces all sources of openssl-1.0.2l.tar.gz into deps/openssl/openssl Fixes: nodejs#13161 PR-URL: nodejs#13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

All symlink files in deps/openssl/openssl/include/openssl/ are removed and replaced with real header files to avoid issues on Windows. Two files of opensslconf.h in crypto and include dir are replaced to refer config/opensslconf.h. Fixes: nodejs#13161 PR-URL: nodejs#13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

`x86masm.pl` was mistakenly using .486 instruction set, why `cpuid` (and perhaps others) are requiring .686 . Fixes: nodejs#589 PR-URL: nodejs#1389 Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>

See https://mta.openssl.org/pipermail/openssl-dev/2015-February/000651.html iojs needs to stop using masm and move to nasm or yasm on Win32. Fixes: nodejs#589 PR-URL: nodejs#1389 Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Reapply b910613 . Fixes: nodejs#589 PR-URL: nodejs#1389 Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Fixes: nodejs#1461 PR-URL: nodejs#1836 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Regenerate config files for supported platforms with Makefile. Fixes: nodejs#13161 PR-URL: nodejs#13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

Regenerate asm files with Makefile and CC=gcc and ASM=nasm where gcc version was 5.4.0 and nasm version was 2.11.08. Also asm files in asm_obsolete dir to support old compiler and assembler are regenerated without CC and ASM envs. Fixes: nodejs#13161 PR-URL: nodejs#13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

Added the missing make command in steps 6.3 when building asm_obsolete. Also updated the commit message to include the version nasm in addition to the gcc version. Fixes: nodejs#13161 PR-URL: nodejs#13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

shigeki · 2017-06-15T10:44:17Z

CI is running on https://ci.nodejs.org/job/node-test-pull-request/8672/.

jasnell · 2017-06-15T18:02:19Z

@shigeki ... given that v4 is in maintenance mode, would you consider this update to be must-have or just nice-to-have?

jasnell · 2017-06-15T18:02:35Z

ping @nodejs/lts

MylesBorins · 2017-06-15T18:26:31Z

It fixes an experience breaking bug on certain platforms

…

On Jun 15, 2017 12:02 PM, "James M Snell" ***@***.***> wrote: ping @nodejs/lts <https://github.com/orgs/nodejs/teams/lts> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13696 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAecVx4Il4bLg41FygfV6bi_tngJfWWDks5sEXHCgaJpZM4N7COT> .

gibfahn · 2017-06-15T18:58:10Z

@jasnell see also #12691 (comment)

shigeki · 2017-06-16T02:32:38Z

I think this update is needed in Node-v4 because #12591 affects all Node running on Windows which have Intel CPUs with SHA-extension feature.
As of the current time, all Goldmont families of Intel CPUs released in April 2016 have this issue and it would get increased if Cannonlake families are released by the end of this year.

The CI results showed that test-child-process-stdio-big-write-end was failed on linux arm64 but it is nothing to do with openssl and the failure still exists in node-daily-v4.x-staging batch.

BridgeAR · 2017-08-27T01:11:10Z

This needs a rebase. Otherwise I guess this could land?

When upgrading OpenSSL, Step 6 in upgrading guide explains the steps that need to be taken if asm files need updating. This might not always be the case and something that needs to be checked from release to release. This commit adds an example of using github to manually compare two tags to see if any changes were made to asm files. Backport-PR-URL: #13696 PR-URL: #13234 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

This replaces all sources of openssl-1.0.2l.tar.gz into deps/openssl/openssl Fixes: #13161 Backport-PR-URL: #13696 PR-URL: #13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

All symlink files in deps/openssl/openssl/include/openssl/ are removed and replaced with real header files to avoid issues on Windows. Two files of opensslconf.h in crypto and include dir are replaced to refer config/opensslconf.h. Fixes: #13161 Backport-PR-URL: #13696 PR-URL: #13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

`x86masm.pl` was mistakenly using .486 instruction set, why `cpuid` (and perhaps others) are requiring .686 . Fixes: #589 Backport-PR-URL: #13696 PR-URL: #1389 Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>

See https://mta.openssl.org/pipermail/openssl-dev/2015-February/000651.html iojs needs to stop using masm and move to nasm or yasm on Win32. Fixes: #589 Backport-PR-URL: #13696 PR-URL: #1389 Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Reapply b910613 . Fixes: #589 Backport-PR-URL: #13696 PR-URL: #1389 Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Fixes: #1461 Backport-PR-URL: #13696 PR-URL: #1836 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Regenerate config files for supported platforms with Makefile. Fixes: #13161 Backport-PR-URL: #13696 PR-URL: #13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

Regenerate asm files with Makefile and CC=gcc and ASM=nasm where gcc version was 5.4.0 and nasm version was 2.11.08. Also asm files in asm_obsolete dir to support old compiler and assembler are regenerated without CC and ASM envs. Fixes: #13161 Backport-PR-URL: #13696 PR-URL: #13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

Added the missing make command in steps 6.3 when building asm_obsolete. Also updated the commit message to include the version nasm in addition to the gcc version. Fixes: #13161 Backport-PR-URL: #13696 PR-URL: #13233 Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

MylesBorins · 2017-09-19T14:58:41Z

landed in 5d7f68a...273e02d