Intermittent crashes in CI

[mhdawson]

There have been a few recent crashes (maybe 3 in last 15 or so days) in the CI https://ci.nodejs.org/view/x%20-%20Abi%20stable%20module%20API/job/node-test-node-addon-api-LTS%20versions/

A number of the failures look like the following just before the crash

Running test 'objectwrap_constructor_exception'
Running test 'objectwrap_multiple_inheritance'
Running test 'objectwrap_worker_thread'
Running test 'promise'
Running test 'reference'
Running test 'run_script'
Running test 'symbol'
Running test 'threadsafe_function/threadsafe_function'
Running test 'threadsafe_function/threadsafe_function_ctx'
Tests aborted with SIGSEGV
napiVersion:8
Testing with Node-API Version '8'.
Starting test suite

[mhdawson]

I think I've managed to recreate a similar instance and the stack trace looks like:

sing host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/home/midawson/newpull/land/node/out/Release/node --expose-gc --no-concurrent-a'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000e9ffe1 in v8::internal::GlobalHandles::Destroy(unsigned long*) ()
[Current thread is 1 (Thread 0x7fda22e537c0 (LWP 2218825))]
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-127.el8.x86_64 libgcc-8.3.1-5.1.el8.x86_64 libstdc++-8.3.1-5.1.el8.x86_64
(gdb) bt
#0  0x0000000000e9ffe1 in v8::internal::GlobalHandles::Destroy(unsigned long*) ()
#1  0x0000000000ad98dd in napi_resolve_deferred ()
#2  0x00007fda20085ebd in Napi::Promise::Deferred::Resolve (this=0x5935dc0, value=0x5945090) at ../../napi-inl.h:2170
#3  0x00007fda200b272c in (anonymous namespace)::TSFNWrap::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>::operator()(Napi::Env, Napi::Reference<Napi::Value> *) const (__closure=0x5939dc8, env=..., ctx=0x5a09dc0)
    at ../threadsafe_function/threadsafe_function_ctx.cc:51
#4  0x00007fda200b386b in Napi::details::ThreadSafeFinalize<Napi::Reference<Napi::Value>, (anonymous namespace)::TSFNWrap::TSFNWrap(const Napi::CallbackInfo&)::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>, void>::FinalizeWrapperWithContext(napi_env, void *, void *) (env=0x5979870, rawFinalizeData=0x5939dc0, rawContext=0x5a09dc0) at ../../napi-inl.h:245
#5  0x0000000000aeb97d in node::Environment::CloseHandle<uv_handle_s, v8impl::(anonymous namespace)::ThreadSafeFunction::CloseHandlesAndMaybeDelete(bool)::{lambda(uv_handle_s*)#1}>(uv_handle_s*, v8impl::(anonymous namespace)::ThreadSafeFunction::CloseHandlesAndMaybeDelete(bool)::{lambda(uv_handle_s*)#1})::{lambda(uv_handle_s*)#1}::_FUN(uv_handle_s*) ()
#6  0x00000000015b4155 in uv__finish_close (handle=0x596ed78) at ../deps/uv/src/unix/core.c:303
#7  uv__run_closing_handles (loop=0x487e080 <default_loop_struct>) at ../deps/uv/src/unix/core.c:317
#8  uv_run (loop=0x487e080 <default_loop_struct>, mode=UV_RUN_DEFAULT) at ../deps/uv/src/unix/core.c:395
#9  0x0000000000a63ed5 in node::SpinEventLoop(node::Environment*) () at ../deps/uv/src/unix/core.c:854
#10 0x0000000000b64546 in node::NodeMainInstance::Run(node::EnvSerializeInfo const*) () at ../deps/uv/src/unix/core.c:854
#11 0x0000000000ae9152 in node::Start(int, char**) () at ../deps/uv/src/unix/core.c:854
#12 0x00007fda21b4a7b3 in __libc_start_main () from /lib64/libc.so.6
#13 0x0000000000a619ce in _start () at ../deps/uv/src/unix/core.c:854
(gdb) Quit

[mhdawson]

And another one:

warning: Loadable section ".note.gnu.property" outside of ELF segments
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/home/midawson/newpull/land/node/out/Release/node --expose-gc --no-concurrent-a'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000ea21e8 in v8::internal::GlobalHandles::Create(v8::internal::Object) ()
[Current thread is 1 (Thread 0x7f7c0579a7c0 (LWP 374115))]
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-127.el8.x86_64 libgcc-8.3.1-5.1.el8.x86_64 libstdc++-8.3.1-5.1.el8.x86_64
(gdb) bt
#0  0x0000000000ea21e8 in v8::internal::GlobalHandles::Create(v8::internal::Object) ()
#1  0x0000000000d0c2d4 in v8::V8::GlobalizeReference(v8::internal::Isolate*, unsigned long*) ()
#2  0x0000000000aeed03 in napi_create_threadsafe_function ()
#3  0x00007f7c01703ead in Napi::ThreadSafeFunction::New<char const*, Napi::Reference<Napi::Value>, (anonymous namespace)::TSFNWrap::TSFNWrap(const Napi::CallbackInfo&)::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>, void>(napi_env, const Napi::Function &, const Napi::Object &, const char *, size_t, size_t, Napi::Reference<Napi::Value> *, (anonymous namespace)::TSFNWrap::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>, void *, napi_finalize) (env=0x5378ef0, 
    callback=..., resource=..., resourceName=0x7f7c017380c3 "Test", maxQueueSize=1, initialThreadCount=1, context=0x532ffa0, finalizeCallback=..., data=0x0, 
    wrapper=0x7f7c01703d45 <Napi::details::ThreadSafeFinalize<Napi::Reference<Napi::Value>, (anonymous namespace)::TSFNWrap::TSFNWrap(const Napi::CallbackInfo&)::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>, void>::FinalizeWrapperWithContext(napi_env, void *, void *)>) at ../../napi-inl.h:5261
#4  0x00007f7c017033b9 in Napi::ThreadSafeFunction::New<char const*, Napi::Reference<Napi::Value>, (anonymous namespace)::TSFNWrap::TSFNWrap(const Napi::CallbackInfo&)::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)> >(napi_env, const Napi::Function &, const Napi::Object &, const char *, size_t, size_t, Napi::Reference<Napi::Value> *, (anonymous namespace)::TSFNWrap::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>) (env=0x5378ef0, callback=..., resource=..., 
    resourceName=0x7f7c017380c3 "Test", maxQueueSize=1, initialThreadCount=1, context=0x532ffa0, finalizeCallback=...) at ../../napi-inl.h:5119
#5  0x00007f7c01702e78 in (anonymous namespace)::TSFNWrap::TSFNWrap (this=0x53fc120, info=...) at ../threadsafe_function/threadsafe_function_ctx.cc:54
#6  0x00007f7c01704335 in Napi::ObjectWrap<(anonymous namespace)::TSFNWrap>::<lambda()>::operator()(void) const (this=0x7ffdd8d6e370)
    at ../../napi-inl.h:4034
#7  0x00007f7c01704c86 in Napi::details::WrapCallback<Napi::ObjectWrap<T>::ConstructorCallbackWrapper(napi_env, napi_callback_info) [with T = (anonymous namespace)::TSFNWrap]::<lambda()> >(Napi::ObjectWrap<(anonymous namespace)::TSFNWrap>::<lambda()>) (callback=...) at ../../napi-inl.h:82
#8  0x00007f7c0170449f in Napi::ObjectWrap<(anonymous namespace)::TSFNWrap>::ConstructorCallbackWrapper (env=0x5378ef0, info=0x7ffdd8d6e400)
    at ../../napi-inl.h:4032
#9  0x0000000000acfdaf in v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) ()
#10 0x0000000000d72ac5 in v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<true>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) ()
#11 0x0000000000d730af in v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) ()
#12 0x0000000000d736d6 in v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) ()
#13 0x0000000001638539 in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit ()
#14 0x00000000015ceff8 in Builtins_JSBuiltinsConstructStub ()

[mhdawson]

Related to the last crash

41 TSFNWrap::TSFNWrap(const CallbackInfo &info) : ObjectWrap<TSFNWrap>(info) {
 42   Napi::Env env = info.Env();
 43 
 44   Reference<Napi::Value> *_ctx = new Reference<Napi::Value>;
 45   *_ctx = Persistent(info[0]);
 46 
 47   _tsfn = ThreadSafeFunction::New(
 48       info.Env(), Function::New(env, [](const CallbackInfo & /*info*/) {}),
 49       Object::New(env), "Test", 1, 1, _ctx,
 50       [this](Napi::Env env, Reference<Napi::Value> *ctx) {
 51         _deferred->Resolve(env.Undefined());
 52         ctx->Reset();
 53         delete ctx;
 54       });
 55 }
 56 
 57 } // namespace

[mhdawson]

And

#0  0x0000000000ea21e8 in v8::internal::GlobalHandles::Create(v8::internal::Object) ()
[Current thread is 1 (Thread 0x7fd9b2a657c0 (LWP 900741))]
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-127.el8.x86_64 libgcc-8.3.1-5.1.el8.x86_64 libstdc++-8.3.1-5.1.el8.x86_64
(gdb) bt
#0  0x0000000000ea21e8 in v8::internal::GlobalHandles::Create(v8::internal::Object) ()
#1  0x0000000000d0c2d4 in v8::V8::GlobalizeReference(v8::internal::Isolate*, unsigned long*) ()
#2  0x0000000000ad96d3 in napi_create_promise ()
#3  0x00007fd9b01a8e41 in Napi::Promise::Deferred::Deferred (this=0x51c1790, env=0x5208ef0) at ../../napi-inl.h:2157
#4  0x00007fd9b01d0adc in (anonymous namespace)::TSFNWrap::Release (this=0x534da80, info=...) at ../threadsafe_function/threadsafe_function_ctx.cc:21
#5  0x00007fd9b01d14ce in Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::<lambda()>::operator()(void) const (this=0x7fffef3ba650)
    at ../../napi-inl.h:3617
#6  0x00007fd9b01d2001 in Napi::details::WrapCallback<Napi::InstanceWrap<T>::InstanceMethodCallbackWrapper(napi_env, napi_callback_info) [with T = (anonymous namespace)::TSFNWrap]::<lambda()> >(Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::<lambda()>) (callback=...) at ../../napi-inl.h:82
#7  0x00007fd9b01d152b in Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::InstanceMethodCallbackWrapper (env=0x5208ef0, info=0x7fffef3ba6b0)
    at ../../napi-inl.h:3610
#8  0x0000000000acfdaf in v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) ()
#9  0x0000000000d71bab in v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) ()
#10 0x0000000000d7305c in v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) ()
#11 0x0000000000d736d6 in v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) ()
#12 0x0000000001638539 in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit ()
#13 0x00000000015d17eb in Builtins_InterpreterEntryTrampoline ()

[mhdawson]

Going to recompile node in debug mode to see if we get any more info as to what is going on in the GlobalHandles code.

[mhdawson]

Of note I am testing with current master, we've seen the similar failure in master. From the CI this is what we have seen lately:

[mhdawson]

Jul 4 - AIX - Node.js 14.x - crash after

Running test 'threadsafe_function/threadsafe_function'
Running test 'threadsafe_function/threadsafe_function_ctx'
Tests aborted with SIGSEGV 

June 28 - OSX 15 - master - crash after

Running test 'threadsafe_function/threadsafe_function'
Running test 'threadsafe_function/threadsafe_function_ctx'
Tests aborted with SIGSEGV

Jun 25 - OSX 14 - 16.x - 
Running test 'threadsafe_function/threadsafe_function'
Running test 'threadsafe_function/threadsafe_function_ctx'
Mismatched noop function calls. Expected exactly 1, actual 0.
    at Object.exports.mustCall (/Users/iojs/build/workspace/node-test-node-addon-api-new/no

From other issues believe this might have been OSX specific issue

June 22 - Fedora - 14.x - Crash after

Running test 'threadsafe_function/threadsafe_function'
Running test 'threadsafe_function/threadsafe_function_ctx'
Tests aborted with SIGSEGV
npm ERR! Test failed.  See

[mhdawson]

Seems like a debug check fires in the same place

warning: Loadable section ".note.gnu.property" outside of ELF segments
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/home/midawson/newpull/land/node/out/Debug/node --expose-gc --no-concurrent-arr'.
Program terminated with signal SIGTRAP, Trace/breakpoint trap.
#0  v8::base::OS::<lambda()>::operator() (__closure=<optimized out>) at ../deps/v8/src/base/platform/platform-posix.cc:502
502	    IMMEDIATE_CRASH();
[Current thread is 1 (Thread 0x7efe8ee2f7c0 (LWP 1991247))]
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-127.el8.x86_64 libgcc-8.3.1-5.1.el8.x86_64 libstdc++-8.3.1-5.1.el8.x86_64
(gdb) bt
#0  v8::base::OS::<lambda()>::operator() (__closure=<optimized out>) at ../deps/v8/src/base/platform/platform-posix.cc:502
#1  v8::base::OS::Abort () at ../deps/v8/src/base/platform/platform-posix.cc:502
#2  0x0000000002d81368 in V8_Fatal (file=0x36a6a20 "../deps/v8/src/handles/global-handles.cc", line=497, 
    format=format@entry=0x59240d2 "Debug check failed: %s.") at ../deps/v8/src/base/logging.cc:167
#3  0x0000000002d81383 in v8::base::(anonymous namespace)::DefaultDcheckHandler (file=<optimized out>, line=<optimized out>, 
    message=<optimized out>) at ../deps/v8/src/base/logging.cc:57
#4  0x000000000166cbfe in v8::internal::GlobalHandles::Node::next_free (this=<optimized out>) at /usr/include/c++/8/bits/basic_string.h:2290
#5  v8::internal::GlobalHandles::NodeSpace<v8::internal::GlobalHandles::Node>::Acquire (object=..., this=0x748d4f0)
    at ../deps/v8/src/handles/global-handles.cc:224
#6  v8::internal::GlobalHandles::Create (this=0x748d440, value=...) at ../deps/v8/src/handles/global-handles.cc:937
#7  0x000000000166cd65 in v8::internal::GlobalHandles::Create (this=<optimized out>, value=<optimized out>)
    at ../deps/v8/src/objects/tagged-impl.h:38
#8  0x00000000013ca5fd in v8::V8::GlobalizeReference (isolate=0x7467280, obj=0x74d7ef8) at ../deps/v8/src/execution/isolate.h:1125
#9  0x00000000010be9c5 in v8::PersistentBase<v8::Value>::New (isolate=0x7467280, that=0x74d7ef8) at ../deps/v8/include/v8.h:10906
#10 0x00000000010d81d4 in v8::PersistentBase<v8::Value>::Reset<v8::Promise::Resolver> (this=0x74576a0, isolate=0x7467280, other=...)
    at ../deps/v8/include/v8.h:10945
#11 0x00000000010d5b40 in napi_create_promise (env=0x75e63c0, deferred=0x74587d8, promise=0x74587e0) at ../src/js_native_api_v8.cc:3042
#12 0x00007efe7ed5fdf2 in Napi::Promise::Deferred::Deferred (this=0x74587d0, env=0x75e63c0) at ../../napi-inl.h:2157
#13 0x00007efe7ed8c520 in (anonymous namespace)::TSFNWrap::Release (this=0x75dc050, info=...)
    at ../threadsafe_function/threadsafe_function_ctx.cc:21
#14 0x00007efe7ed8cfac in Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::<lambda()>::operator()(void) const (this=0x7ffe1c6fbd30)
    at ../../napi-inl.h:3617
#15 0x00007efe7ed8dadb in Napi::details::WrapCallback<Napi::InstanceWrap<T>::InstanceMethodCallbackWrapper(napi_env, napi_callback_info) [with T = (anonymous namespace)::TSFNWrap]::<lambda()> >(Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::<lambda()>) (callback=...)
    at ../../napi-inl.h:74
#16 0x00007efe7ed8d029 in Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::InstanceMethodCallbackWrapper (env=0x75e63c0, 
    info=0x7ffe1c6fbe90) at ../../napi-inl.h:3610
#17 0x00000000010c9662 in v8impl::(anonymous namespace)::CallbackWrapperBase::<lambda(napi_env)>::operator()(napi_env) const (
    __closure=0x7ffe1c6fbe50, env=0x75e63c0) at ../src/js_native_api_v8.cc:311
#18 0x00000000010d67c9 in napi_env__::CallIntoModule<v8impl::(anonymous namespace)::CallbackWrapperBase::InvokeCallback()::<lambda(napi_env)> >(v8impl::(anonymous namespace)::CallbackWrapperBase::<lambda(napi_env)> &&, void (&&)(napi_env__ *, v8::Local<v8::Value>)) (this=0x75e63c0, 
    call=..., handle_exception=
    @0x10d73da: {void (napi_env__ *, v8::Local<v8::Value>)} 0x10d73da <napi_env__::HandleThrow(napi_env__*, v8::Local<v8::Value>)>)
    at ../src/js_native_api_v8.h:95
#19 0x00000000010c96d1 in v8impl::(anonymous namespace)::CallbackWrapperBase::InvokeCallback (this=0x7ffe1c6fbe90)
    at ../src/js_native_api_v8.cc:310
#20 0x00000000010c9726 in v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke (info=...) at ../src/js_native_api_v8.cc:328
--Type <RET> for more, q to quit, c to continue without paging-- 
#21 0x000000000146bce0 in v8::internal::FunctionCallbackArguments::Call (this=this@entry=0x7ffe1c6fc000, handler=handler@entry=...)
    at ../deps/v8/src/api/api-arguments-inl.h:158
#22 0x000000000146cb07 in v8::internal::(anonymous namespace)::HandleApiCallHelper<false> (isolate=isolate@entry=0x7467280, function=..., 
    function@entry=..., new_target=..., new_target@entry=..., fun_data=..., receiver=..., receiver@entry=..., args=...)
    at ../deps/v8/src/base/platform/wrappers.h:37
#23 0x0000000001471762 in v8::internal::Builtin_Impl_HandleApiCall (args=..., isolate=isolate@entry=0x7467280)
    at ../deps/v8/src/handles/handles.h:132
#24 0x00000000014724e7 in v8::internal::Builtin_HandleApiCall (args_length=5, args_object=0x7ffe1c6fc1a8, isolate=0x7467280)
    at ../deps/v8/src/builtins/builtins-api.cc:131
#25 0x00000000023156a0 in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit () at ../../deps/v8/src/builtins/torque-internal.tq:84
#26 0x000000000209102b in Builtins_InterpreterEntryTrampoline () at ../../deps/v8/src/objects/string.tq:183

[mhdawson]

I'm wondering if this is related to the issue we see in the ci or specific to main. Going to run with 16.x to see if we get the same debug checks there.

[mhdawson]

16.x seems to have less, just 2 in over 10000 runs so far. This is one of the stack traces:

sing host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/home/midawson/newpull/land/node/out/Debug/node --expose-gc --no-concurrent-arr'.
Program terminated with signal SIGTRAP, Trace/breakpoint trap.
#0  v8::base::OS::<lambda()>::operator() (__closure=<optimized out>) at ../deps/v8/src/base/platform/platform-posix.cc:502
502	    IMMEDIATE_CRASH();
[Current thread is 1 (Thread 0x7f79e4f687c0 (LWP 3797863))]
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-127.el8.x86_64 libgcc-8.3.1-5.1.el8.x86_64 libstdc++-8.3.1-5.1.el8.x86_64
(gdb) bt
#0  v8::base::OS::<lambda()>::operator() (__closure=<optimized out>) at ../deps/v8/src/base/platform/platform-posix.cc:502
#1  v8::base::OS::Abort () at ../deps/v8/src/base/platform/platform-posix.cc:502
#2  0x0000000002d82068 in V8_Fatal (file=0x3698260 "../deps/v8/src/handles/global-handles.cc", line=497, 
    format=format@entry=0x590eef2 "Debug check failed: %s.") at ../deps/v8/src/base/logging.cc:167
#3  0x0000000002d82083 in v8::base::(anonymous namespace)::DefaultDcheckHandler (file=<optimized out>, line=<optimized out>, 
    message=<optimized out>) at ../deps/v8/src/base/logging.cc:57
#4  0x000000000166e0ae in v8::internal::GlobalHandles::Node::next_free (this=<optimized out>) at /usr/include/c++/8/bits/basic_string.h:2290
#5  v8::internal::GlobalHandles::NodeSpace<v8::internal::GlobalHandles::Node>::Acquire (object=..., this=0x7be9170)
    at ../deps/v8/src/handles/global-handles.cc:224
#6  v8::internal::GlobalHandles::Create (this=0x7be90c0, value=...) at ../deps/v8/src/handles/global-handles.cc:937
#7  0x000000000166e215 in v8::internal::GlobalHandles::Create (this=<optimized out>, value=<optimized out>)
    at ../deps/v8/src/objects/tagged-impl.h:38
#8  0x00000000013ca55d in v8::V8::GlobalizeReference (isolate=0x7bc2fb0, obj=0x7c33b78) at ../deps/v8/src/execution/isolate.h:1125
#9  0x00000000010be995 in v8::PersistentBase<v8::Value>::New (isolate=0x7bc2fb0, that=0x7c33b78) at ../deps/v8/include/v8.h:11146
#10 0x00000000010d81d4 in v8::PersistentBase<v8::Value>::Reset<v8::Promise::Resolver> (this=0x7c7a0e0, isolate=0x7bc2fb0, other=...)
    at ../deps/v8/include/v8.h:11185
#11 0x00000000010d5b38 in napi_create_promise (env=0x7d30b10, deferred=0x7bb4628, promise=0x7bb4630) at ../src/js_native_api_v8.cc:3045
#12 0x00007f79e1198df2 in Napi::Promise::Deferred::Deferred (this=0x7bb4620, env=0x7d30b10) at ../../napi-inl.h:2157
#13 0x00007f79e11c5520 in (anonymous namespace)::TSFNWrap::Release (this=0x7cc87a0, info=...)
    at ../threadsafe_function/threadsafe_function_ctx.cc:21
#14 0x00007f79e11c5fac in Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::<lambda()>::operator()(void) const (this=0x7ffcbde96e40)
    at ../../napi-inl.h:3617
#15 0x00007f79e11c6adb in Napi::details::WrapCallback<Napi::InstanceWrap<T>::InstanceMethodCallbackWrapper(napi_env, napi_callback_info) [with T = (anonymous namespace)::TSFNWrap]::<lambda()> >(Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::<lambda()>) (callback=...)
    at ../../napi-inl.h:74
#16 0x00007f79e11c6029 in Napi::InstanceWrap<(anonymous namespace)::TSFNWrap>::InstanceMethodCallbackWrapper (env=0x7d30b10, 
    info=0x7ffcbde96fa0) at ../../napi-inl.h:3610
#17 0x00000000010c9632 in v8impl::(anonymous namespace)::CallbackWrapperBase::<lambda(napi_env)>::operator()(napi_env) const (
    __closure=0x7ffcbde96f60, env=0x7d30b10) at ../src/js_native_api_v8.cc:311
#18 0x00000000010d67c1 in napi_env__::CallIntoModule<v8impl::(anonymous namespace)::CallbackWrapperBase::InvokeCallback()::<lambda(napi_env)> >(v8impl::(anonymous namespace)::CallbackWrapperBase::<lambda(napi_env)> &&, void (&&)(napi_env__ *, v8::Local<v8::Value>)) (this=0x7d30b10, 
    call=..., handle_exception=
    @0x10d73d2: {void (napi_env__ *, v8::Local<v8::Value>)} 0x10d73d2 <napi_env__::HandleThrow(napi_env__*, v8::Local<v8::Value>)>)
    at ../src/js_native_api_v8.h:95
#19 0x00000000010c96a1 in v8impl::(anonymous namespace)::CallbackWrapperBase::InvokeCallback (this=0x7ffcbde96fa0)
    at ../src/js_native_api_v8.cc:310
#20 0x00000000010c96f6 in v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke (info=...) at ../src/js_native_api_v8.cc:328
#21 0x000000000146d190 in v8::internal::FunctionCallbackArguments::Call (this=this@entry=0x7ffcbde97110, handler=handler@entry=...)
    at ../deps/v8/src/api/api-arguments-inl.h:158
#22 0x000000000146dfb7 in v8::internal::(anonymous namespace)::HandleApiCallHelper<false> (isolate=isolate@entry=0x7bc2fb0, function=..., 
    function@entry=..., new_target=..., new_target@entry=..., fun_data=..., receiver=..., receiver@entry=..., args=...)
    at ../deps/v8/src/base/platform/wrappers.h:37
#23 0x0000000001472c12 in v8::internal::Builtin_Impl_HandleApiCall (args=..., isolate=isolate@entry=0x7bc2fb0)
    at ../deps/v8/src/handles/handles.h:132
#24 0x0000000001473997 in v8::internal::Builtin_HandleApiCall (args_length=5, args_object=0x7ffcbde972b8, isolate=0x7bc2fb0)
    at ../deps/v8/src/builtins/builtins-api.cc:131
#25 0x0000000002317020 in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit () at ../../deps/v8/src/builtins/torque-internal.tq:84
#26 0x00000000020929ab in Builtins_InterpreterEntryTrampoline () at ../../deps/v8/src/objects/string.tq:183

[KevinEady]

I'm not sure but all of the stack traces have GlobalHandles Create and Destroy involved... Is there something wrong with the test's code? Should I create handles? For example, I don't create a handle in Finalize because there wasn't one done in the original Node-API tsfn test https://github.com/nodejs/node/blob/4b0776ab64596d1bbd98a93e3de7cee82a7e8130/test/node-api/test_threadsafe_function/binding.c#L167-L188

But this one is odd:

And another one:

warning: Loadable section ".note.gnu.property" outside of ELF segments
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/home/midawson/newpull/land/node/out/Release/node --expose-gc --no-concurrent-a'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000ea21e8 in v8::internal::GlobalHandles::Create(v8::internal::Object) ()
[Current thread is 1 (Thread 0x7f7c0579a7c0 (LWP 374115))]
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-127.el8.x86_64 libgcc-8.3.1-5.1.el8.x86_64 libstdc++-8.3.1-5.1.el8.x86_64
(gdb) bt
#0  0x0000000000ea21e8 in v8::internal::GlobalHandles::Create(v8::internal::Object) ()
#1  0x0000000000d0c2d4 in v8::V8::GlobalizeReference(v8::internal::Isolate*, unsigned long*) ()
#2  0x0000000000aeed03 in napi_create_threadsafe_function ()
#3  0x00007f7c01703ead in Napi::ThreadSafeFunction::New<char const*, Napi::Reference<Napi::Value>, (anonymous namespace)::TSFNWrap::TSFNWrap(const Napi::CallbackInfo&)::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>, void>(napi_env, const Napi::Function &, const Napi::Object &, const char *, size_t, size_t, Napi::Reference<Napi::Value> *, (anonymous namespace)::TSFNWrap::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>, void *, napi_finalize) (env=0x5378ef0, 
    callback=..., resource=..., resourceName=0x7f7c017380c3 "Test", maxQueueSize=1, initialThreadCount=1, context=0x532ffa0, finalizeCallback=..., data=0x0, 
    wrapper=0x7f7c01703d45 <Napi::details::ThreadSafeFinalize<Napi::Reference<Napi::Value>, (anonymous namespace)::TSFNWrap::TSFNWrap(const Napi::CallbackInfo&)::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>, void>::FinalizeWrapperWithContext(napi_env, void *, void *)>) at ../../napi-inl.h:5261
#4  0x00007f7c017033b9 in Napi::ThreadSafeFunction::New<char const*, Napi::Reference<Napi::Value>, (anonymous namespace)::TSFNWrap::TSFNWrap(const Napi::CallbackInfo&)::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)> >(napi_env, const Napi::Function &, const Napi::Object &, const char *, size_t, size_t, Napi::Reference<Napi::Value> *, (anonymous namespace)::TSFNWrap::<lambda(Napi::Env, Napi::Reference<Napi::Value>*)>) (env=0x5378ef0, callback=..., resource=..., 
    resourceName=0x7f7c017380c3 "Test", maxQueueSize=1, initialThreadCount=1, context=0x532ffa0, finalizeCallback=...) at ../../napi-inl.h:5119
#5  0x00007f7c01702e78 in (anonymous namespace)::TSFNWrap::TSFNWrap (this=0x53fc120, info=...) at ../threadsafe_function/threadsafe_function_ctx.cc:54
#6  0x00007f7c01704335 in Napi::ObjectWrap<(anonymous namespace)::TSFNWrap>::<lambda()>::operator()(void) const (this=0x7ffdd8d6e370)
    at ../../napi-inl.h:4034
#7  0x00007f7c01704c86 in Napi::details::WrapCallback<Napi::ObjectWrap<T>::ConstructorCallbackWrapper(napi_env, napi_callback_info) [with T = (anonymous namespace)::TSFNWrap]::<lambda()> >(Napi::ObjectWrap<(anonymous namespace)::TSFNWrap>::<lambda()>) (callback=...) at ../../napi-inl.h:82
#8  0x00007f7c0170449f in Napi::ObjectWrap<(anonymous namespace)::TSFNWrap>::ConstructorCallbackWrapper (env=0x5378ef0, info=0x7ffdd8d6e400)
    at ../../napi-inl.h:4032
#9  0x0000000000acfdaf in v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) ()
#10 0x0000000000d72ac5 in v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<true>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) ()
#11 0x0000000000d730af in v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) ()
#12 0x0000000000d736d6 in v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) ()
#13 0x0000000001638539 in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit ()
#14 0x00000000015ceff8 in Builtins_JSBuiltinsConstructStub ()

This one is part of the synchronous stack from JavaScript -> C++ binding

node-addon-api/test/threadsafe_function/threadsafe_function_ctx.cc

Line 47 in e02e8a4

_tsfn = ThreadSafeFunction::New(

No clue how this could fail...?

[mhdawson]

I tried running just the one test and 30000 iterations led to no recreates so trying with larger set of tests.

[mhdawson]

@KevinEady still trying to narrow down, I agree it's not obvious why some of those failures could/would happen. I'm thinking maybe we are getting memory corruption which then leads to those asserts. Trying to narrow down recreate but seems like it running a single test does not result in the problem.