Garbage Collection Crash

[sehz]

• Version: v6.4.0:
• Platform: Mac 15.6.0 Darwin Kernel Version 15.6.0::
• ** OS X Version 10.11.6 (15G31)**:

Node crashed with following stack trace:

#
# Fatal error in ../deps/v8/src/execution.cc, line 64
# Check failed: AllowJavascriptExecution::IsAllowed(isolate).
#

==== C stack trace ===============================

 1: V8_Fatal
 2: v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, bool, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*, v8::internal::Handle<v8::internal::Object>)
 3: v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*)
 4: v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*)
 5: node::TLSWrap::~TLSWrap()
 6: node::TLSWrap::~TLSWrap()
 7: v8::internal::GlobalHandles::DispatchPendingPhantomCallbacks(bool)
 8: v8::internal::GlobalHandles::PostGarbageCollectionProcessing(v8::internal::GarbageCollector, v8::GCCallbackFlags)
 9: v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags)
10: v8::internal::Heap::CollectGarbage(v8::internal::GarbageCollector, char const*, char const*, v8::GCCallbackFlags)
11: v8::internal::Factory::NewCodeRaw(int, bool)
12: v8::internal::Factory::NewCode(v8::internal::CodeDesc const&, unsigned int, v8::internal::Handle<v8::internal::Object>, bool, bool, int, bool)
13: v8::internal::CodeGenerator::MakeCodeEpilogue(v8::internal::MacroAssembler*, v8::internal::CompilationInfo*)
14: v8::internal::LChunk::Codegen()
15: v8::internal::OptimizedCompileJob::GenerateCode()
16: v8::internal::Compiler::GetConcurrentlyOptimizedCode(v8::internal::OptimizedCompileJob*)
17: v8::internal::OptimizingCompileDispatcher::InstallOptimizedFunctions()
18: v8::internal::StackGuard::HandleInterrupts()
19: v8::internal::Runtime_StackGuard(int, v8::internal::Object**, v8::internal::Isolate*)
20: 0xe170e00961b
21: 0xe170e6cb2e5

Process finished with exit code 132 (interrupted by signal 4: SIGILL)

[mscdex]

/cc @indutny

[addaleax]

@sehyoc Thanks for reporting this! Is there any chance you can offer more information? Is the problem reproducible? Do you have something like a test script that can demonstrate the issue?

[mscdex]

Also: do you know if there is a node version where the same code was working, or is this the first version of node you tried?

[indutny]

I have no other suspects but the AsyncWrap class. @trevnorris looks like calling JS functions is no longer allowed from the ~AsyncWrap, is it something that we should look into?

[bnoordhuis]

I agree with Fedor that the call into the VM from the AsyncWrap destructor is responsible.

@sehyoc Something in your project (either your own code or a third-party module) is calling process.binding('async_wrap').setupHooks(). Try disabling that for now as a workaround.

[sehz]

This problem is reproducible but not everytime. Crashes happen 1 of 5 times during startup. Before I was running v6.3.1 which also crashed but much less frequency.

We are using babel with async plug-in. Since we are heavily relying on 'async' keyword, there is no way to turn this off.

[addaleax]

@sehyoc This is not about the async keyword, async_wrap is something very different. Are you sure nothing in your project is touching process.binding('async_wrap')? grep -R async_wrap . should be a reasonable way to check for that.

[sehz]

Yes, I found who was using async_wrap. It was async-hook package:

node_modules/async-hook/async-hook.js:const asyncWrap = process.binding('async_wrap'); Binary file node_modules/electron-prebuilt/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Libraries/libnode.dylib matches Binary file node_modules/electron-prebuilt/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib matches Binary file node_modules/electron-prebuilt/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/Current/Libraries/libnode.dylib matches

And furthermore, async-hook was used by:
node_modules/trace/package.json: "async-hook": "^1.0.0", node_modules/trace/trace.js:const asyncHook = require('async-hook');

I was using to get long stack trace. Current version of trace still using async-hook. I removed the package

[gibfahn]

cc/ @nodejs/diagnostics

[nakedcity]

any resolution so far?

[indutny]

ping @trevnorris

[nakedcity]

well just to add I see this is still happening in versions 7 too. Easy way for me to reproduce is to install any package using async hooks and fire few requests to make it crash, let me know if you need any help

[trevnorris]

I have a fix ready, but don't have a reproducible test case for it. There a minimal one I can include?

[nakedcity]

I easily reproduce using this package https://github.com/AndreasMadsen/trace/ (Uses async-hooks)
and firing more than 5 request with express.

If you give me the branch you are working on I can test my current scenario upon it

[trevnorris]

@nakedcity link is #9753

[nakedcity]

Thanks @trevnorris I take a look this weekend

[surajwy]

I am also experiencing a similar crash in PostGarbageCollectionProcessing call
I do not have any reproducible case. But it happens 1-2 times a day in production.
Whenever I have got this crash, we were connecting to AWS's endpoint.
I do not have any module which uses async_wrap

Node.js version: v6.11.0
OpenSSL version: OpenSSL 1.0.1k-fips
Platform: AWS EC2
OS: Amazon linux 4.9.27-14.31.amzn1.x86_64

(lldb) v8 bt
 * SBThread: tid = 0x0000  * frame #0: 0x000000000111b346 node`node::TLSWrap::~TLSWrap() + 358
    frame #1: 0x000000000111b421 node`node::TLSWrap::~TLSWrap() + 17
    frame #2: 0x0000000000c85e5c node`v8::internal::GlobalHandles::DispatchPendingPhantomCallbacks(bool) + 252
    frame #3: 0x0000000000c860ba node`v8::internal::GlobalHandles::PostGarbageCollectionProcessing(v8::internal::GarbageCollector, v8::GCCallbackFlags) + 42
    frame #4: 0x0000000000ca6e37 node`v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) + 535
    frame #5: 0x0000000000ca771a node`v8::internal::Heap::CollectGarbage(v8::internal::GarbageCollector, char const*, char const*, v8::GCCallbackFlags) + 330
    frame #6: 0x0000000000ca932e node`v8::internal::Heap::HandleGCRequest() + 222
    frame #7: 0x0000000000c55bac node`v8::internal::StackGuard::HandleInterrupts() + 796
    frame #8: 0x0000000000eb75fc node`v8::internal::Runtime_StackGuard(int, v8::internal::Object**, v8::internal::Isolate*) + 380
    frame #9: 0x00001ee4063079a7 <exit>

[bnoordhuis]

@surajwy Can you open a new issue and include the contents of the registers and a disassembly of the top frame? Yours is probably a different issue than this bug report.

edit: by the way, is that with a binary from https://nodejs.org/?

[surajwy]

@bnoordhuis I have opened a new issue here: #17097
Thanks.