SIGSEGV in libcrypto.so when https.request is called

[brunohadlich]

• Node.js Version:
  v6.12.3
• OS:
  Linux coordinator 4.9.79-yocto-standard Update README for help #1 SMP PREEMPT Sun Feb 25 00:21:42 PST 2018 x86_64 x86_64 x86_64 GNU/Linux
• Scope (install, code, runtime, meta, other?):
• Module (and version) (if relevant):
  https

I have an application running behind apache and apache only accepts https requests, my problem is that when a request is received I have to make a https request to another server in order to obtain the response and then return such response to the browser, the moment that I execute a https.request call I get a SIGSEGV. I won't be able to provide a piece of code that anyone can run and reproduce because this problem happens only when I run the entire application but I executed node with valgrind and it seems that the problem is coming from libcrypto.so, I think that maybe this is a concurrency issue in libcrypto.so but I'm not sure, the valgrind output is:

==6646== Use of uninitialised value of size 8
==6646== at 0x1B53F4223073: ???
==6646== by 0x1B53F458DA12: ???
==6646== by 0x1B53F421C7D4: ???
==6646== by 0x1B53F458D29B: ???
==6646== by 0x1B53F421C7D4: ???
==6646== by 0x1B53F458C21E: ???
==6646== by 0x1B53F4589EBA: ???
==6646== by 0x1B53F45847E6: ???
==6646== by 0x1B53F421E7B1: ???
==6646== by 0x1B53F45828AC: ???
==6646== by 0x1B53F45820A1: ???
==6646== by 0x1B53F4578F87: ???
==6646==
==6646== Invalid read of size 8
==6646== at 0x3A0E78F4F3: CRYPTO_set_ex_data (in /usr/lib/libcrypto.so.1.0.0)
==6646== by 0xCAF3E4: node::crypto::SecureContext::Init(v8::FunctionCallbackInfov8::Value const&) (in /usr/bin/node)
==6646== by 0x6303C3: v8::internal::FunctionCallbackArguments::Call(void ()(v8::FunctionCallbackInfov8::Value const&)) (in /usr/bin/node)
==6646== by 0x671522: ??? (in /usr/bin/node)
==6646== by 0x671B38: ??? (in /usr/bin/node)
==6646== by 0x671C04: ??? (in /usr/bin/node)
==6646== by 0x1B53F42060C6: ???
==6646== by 0x1B53F4591493: ???
==6646== by 0x1B53F421E7B1: ???
==6646== by 0x1B53F458FB08: ???
==6646== by 0x1B53F421C7D4: ???
==6646== by 0x1B53F458DF5A: ???
==6646== Address 0xd0 is not stack'd, malloc'd or (recently) free'd
==6646==
==6646==
==6646== Process terminating with default action of signal 11 (SIGSEGV)
==6646== Access not within mapped region at address 0xD0
==6646== at 0x3A0E78F4F3: CRYPTO_set_ex_data (in /usr/lib/libcrypto.so.1.0.0)
==6646== by 0xCAF3E4: node::crypto::SecureContext::Init(v8::FunctionCallbackInfov8::Value const&) (in /usr/bin/node)
==6646== by 0x6303C3: v8::internal::FunctionCallbackArguments::Call(void ()(v8::FunctionCallbackInfov8::Value const&)) (in /usr/bin/node)
==6646== by 0x671522: ??? (in /usr/bin/node)
==6646== by 0x671B38: ??? (in /usr/bin/node)
==6646== by 0x671C04: ??? (in /usr/bin/node)
==6646== by 0x1B53F42060C6: ???
==6646== by 0x1B53F4591493: ???
==6646== by 0x1B53F421E7B1: ???
==6646== by 0x1B53F458FB08: ???
==6646== by 0x1B53F421C7D4: ???
==6646== by 0x1B53F458DF5A: ???
==6646== If you believe this happened as a result of a stack
==6646== overflow in your program's main thread (unlikely but
==6646== possible), you can try to increase the size of the
==6646== main thread stack using the --main-stacksize= flag.
==6646== The main thread stack size used in this run was 8388608.

As I said before I could not create a simple script that could reproduce the issue but I'm making a https request like:

var payload = JSON.stringify({"key":"value"});
var options = {
	hostname: '192.168.15.14',
	port: 443,
	path: '/path',
	method: 'POST',
	headers: {
		'Content-Type': 'application/json',
		'Content-Length': Buffer.byteLength(payload)
	}
};
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
var req_remote_outlets = https.request(options, function(res) {
		res.on("error", (error) => {
				console.log(error);
		});
		res.on("data", (chunk) => {
				console.log(chunk.toString("utf8"));
		});
		res.on("end", () => {
		});
});
req_remote_outlets.on("error", (err) => {
		console.log(err);
});
req_remote_outlets.write(payload);
req_remote_outlets.end();

I didn't find any source in the web that could help me with such issue, please tell me if I'm doing something wrong.

[gireeshpunathil]

==6646== at 0x3A0E78F4F3: CRYPTO_set_ex_data (in /usr/lib/libcrypto.so.1.0.0)

the version of node you are using is not the official distribution from https://nodejs.org , can you confirm that?

[brunohadlich]

We are using v6.12.3, the build is done in our own environment but no patch has been applied.

[gireeshpunathil]

right - but how / where did you download / install node.js?
can you please tell me the output of:
node -e 'console.log(process.versions)'
and
node -e 'console.log(process.release)'

[gireeshpunathil]

sorry, I misread your last comment. You meant you built it by yourself? If so, what was the configurations you used? ./configure ... options?

[brunohadlich]

Hi @gireeshpunathil.

node -e 'console.log(process.versions)'
{ http_parser: '2.7.0',
node: '6.12.3',
v8: '5.1.281.111',
uv: '1.15.0',
zlib: '1.2.8',
ares: '1.10.1-DEV',
modules: '48',
openssl: '1.0.2n' }

node -e 'console.log(process.release)'
{ name: 'node',
lts: 'Boron',
sourceUrl: 'https://nodejs.org/download/release/v6.12.3/node-v6.12.3.tar.gz',
headersUrl: 'https://nodejs.org/download/release/v6.12.3/node-v6.12.3-headers.tar.gz' }

We use yocto to build our embedded linux distro. Attached is the configure file:
configure.txt

[gireeshpunathil]

I mean to ask - did you use one (or more) of these options when you did ./configure?

#./configure --help | grep shared-openssl
    --shared-openssl    link to a shared OpenSSl DLL instead of static linking
    --shared-openssl-includes=SHARED_OPENSSL_INCLUDES
    --shared-openssl-libname=SHARED_OPENSSL_LIBNAME
    --shared-openssl-libpath=SHARED_OPENSSL_LIBPATH

[brunohadlich]

Yes the option --shared-openssl is used, should I try to build node with static openssl? I mean, do you think this could solve my issue?

[gireeshpunathil]

Sorry, I have not review reviewed the code or the problem. But we don't support issues coming through / due to the linked ssl library.

So it is worthwhile trying with the static ssl to see if the problem goes away. Having said that, I am not advocating to use static ssl, instead saying that it is option for problem determination.

Hope this helps.

[gireeshpunathil]

@brunohadlich - were your issues resolved?

[gireeshpunathil]

closing as answered, please re-open if it is still outstanding.