Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segfault when freeing recognizer in Node js #1034

Closed
sidor926 opened this issue Jun 21, 2022 · 7 comments
Closed

Segfault when freeing recognizer in Node js #1034

sidor926 opened this issue Jun 21, 2022 · 7 comments

Comments

@sidor926
Copy link

Hi,

I'm getting a segfault when I exit the node.js test_microphone example.
I'm on aarch64, and was getting a dynamic linking error when I was using the binary that came with npm, but I replaced that with the python3.8 binary for aarch64 as mentioned in #620 and #479. Specifically, I don't get the seg fault when I comment out recognizer.free(). I ran a memcheck using valgrind and have attached the output. I've tried to figure out if and which deallocation causes the problem, and am leaning towards the idea that this is might be a vosk implementation on aarch64 bug. Are there any quick fixes? What might I be missing?

memcheck:

root@mymachine:/home/vosk_tests/node_modules/vosk/demo# valgrind node test_microphone.js
==31112== Memcheck, a memory error detector
==31112== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==31112== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==31112== Command: node test_microphone.js
==31112==
ARM64 front end: branch_etc
disInstr(arm64): unhandled instruction 0xD50B7E23
disInstr(arm64): 1101'0101 0000'1011 0111'1110 0010'0011
==31112== valgrind: Unrecognised instruction at address 0x13eaa74.
==31112== at 0x13EAA74: v8::internal::CpuFeatures::FlushICache(void*, unsigned long) (in /usr/local/bin/node)
==31112== by 0xD7D863: v8::internal::RelocInfo::set_target_address(unsigned long, v8::internal::WriteBarrierMode, v8::internal::ICacheFlushMode) (in /usr/local/bin/node)
==31112== by 0x11D07DB: int v8::internal::Deserializerv8::internal::Isolate::ReadSingleBytecodeDatav8::internal::SlotAccessorForHeapObject(unsigned char, v8::internal::SlotAccessorForHeapObject) (in /usr/local/bin/node)
==31112== by 0x11D0D0B: v8::internal::Deserializerv8::internal::Isolate::ReadData(v8::internal::Handlev8::internal::HeapObject, int, int) (in /usr/local/bin/node)
==31112== by 0x11D0E77: v8::internal::Deserializerv8::internal::Isolate::ReadObject(v8::internal::SnapshotSpace) (in /usr/local/bin/node)
==31112== by 0x11D156B: int v8::internal::Deserializerv8::internal::Isolate::ReadSingleBytecodeDatav8::internal::SlotAccessorForRootSlots(unsigned char, v8::internal::SlotAccessorForRootSlots) (in /usr/local/bin/node)
==31112== by 0x11D1B17: v8::internal::Deserializerv8::internal::Isolate::ReadData(v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (in /usr/local/bin/node)
==31112== by 0xE70463: v8::internal::Heap::IterateRoots(v8::internal::RootVisitor*, v8::base::EnumSet<v8::internal::SkipRoot, int>) (in /usr/local/bin/node)
==31112== by 0x11EFB47: v8::internal::StartupDeserializer::DeserializeIntoIsolate() (in /usr/local/bin/node)
==31112== by 0xE10277: v8::internal::Isolate::Init(v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) (in /usr/local/bin/node)
==31112== by 0x11ED313: v8::internal::Snapshot::Initialize(v8::internal::Isolate*) [clone .part.31] (in /usr/local/bin/node)
==31112== by 0xCEA367: v8::Isolate::Initialize(v8::Isolate*, v8::Isolate::CreateParams const&) (in /usr/local/bin/node)
==31112== Your program just tried to execute an instruction that Valgrind
==31112== did not recognise. There are two possible reasons for this.
==31112== 1. Your program has a bug and erroneously jumped to a non-code
==31112== location. If you are running Memcheck and you just saw a
==31112== warning about a bad jump, it's probably your program's fault.
==31112== 2. The instruction is legitimate but Valgrind doesn't handle it,
==31112== i.e. it's Valgrind's fault. If you think this is the case or
==31112== you are not sure, please let us know and we'll try to fix it.
==31112== Either way, Valgrind will now raise a SIGILL signal which will
==31112== probably kill your program.
==31112==
==31112== Process terminating with default action of signal 4 (SIGILL)
==31112== Illegal opcode at address 0x13EAA74
==31112== at 0x13EAA74: v8::internal::CpuFeatures::FlushICache(void*, unsigned long) (in /usr/local/bin/node)
==31112== by 0xD7D863: v8::internal::RelocInfo::set_target_address(unsigned long, v8::internal::WriteBarrierMode, v8::internal::ICacheFlushMode) (in /usr/local/bin/node)
==31112== by 0x11D07DB: int v8::internal::Deserializerv8::internal::Isolate::ReadSingleBytecodeDatav8::internal::SlotAccessorForHeapObject(unsigned char, v8::internal::SlotAccessorForHeapObject) (in /usr/local/bin/node)
==31112== by 0x11D0D0B: v8::internal::Deserializerv8::internal::Isolate::ReadData(v8::internal::Handlev8::internal::HeapObject, int, int) (in /usr/local/bin/node)
==31112== by 0x11D0E77: v8::internal::Deserializerv8::internal::Isolate::ReadObject(v8::internal::SnapshotSpace) (in /usr/local/bin/node)
==31112== by 0x11D156B: int v8::internal::Deserializerv8::internal::Isolate::ReadSingleBytecodeDatav8::internal::SlotAccessorForRootSlots(unsigned char, v8::internal::SlotAccessorForRootSlots) (in /usr/local/bin/node)
==31112== by 0x11D1B17: v8::internal::Deserializerv8::internal::Isolate::ReadData(v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (in /usr/local/bin/node)
==31112== by 0xE70463: v8::internal::Heap::IterateRoots(v8::internal::RootVisitor*, v8::base::EnumSet<v8::internal::SkipRoot, int>) (in /usr/local/bin/node)
==31112== by 0x11EFB47: v8::internal::StartupDeserializer::DeserializeIntoIsolate() (in /usr/local/bin/node)
==31112== by 0xE10277: v8::internal::Isolate::Init(v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) (in /usr/local/bin/node)
==31112== by 0x11ED313: v8::internal::Snapshot::Initialize(v8::internal::Isolate*) [clone .part.31] (in /usr/local/bin/node)
==31112== by 0xCEA367: v8::Isolate::Initialize(v8::Isolate*, v8::Isolate::CreateParams const&) (in /usr/local/bin/node)
==31112==
==31112== HEAP SUMMARY:
==31112== in use at exit: 2,438,023 bytes in 3,882 blocks
==31112== total heap usage: 4,037 allocs, 155 frees, 2,678,283 bytes allocated
==31112==
==31112== LEAK SUMMARY:
==31112== definitely lost: 0 bytes in 0 blocks
==31112== indirectly lost: 0 bytes in 0 blocks
==31112== possibly lost: 1,520 bytes in 5 blocks
==31112== still reachable: 2,436,503 bytes in 3,877 blocks
==31112== of which reachable via heuristic:
==31112== stdstring : 48,225 bytes in 980 blocks
==31112== suppressed: 0 bytes in 0 blocks
==31112== Rerun with --leak-check=full to see details of leaked memory
==31112==
==31112== For counts of detected and suppressed errors, rerun with: -v
==31112== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Illegal instruction (core dumped)
@nshmyrev
Copy link
Collaborator

Can you please collect backtrace with gdb? Not just valgrind. Valgrind seem to have problems handling this particular architecture.

@sidor926
Copy link
Author

Any recommended way to use GDB with node?

@nshmyrev
Copy link
Collaborator

gdb --args node test_microphone.js then type run, when crashes type backtrace.

@sidor926
Copy link
Author

sidor926 commented Jun 27, 2022 •
edited by nshmyrev
Loading

Here's the backtrace using gdb:

(gdb) backtrace
#0  0x0000007fb6b000b4 in long fst::LabelReachable<fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::FastLogAccumulator<fst::ArcTpl<fst::TropicalWeightTpl<float> > >, fst::LabelReachableData<int> >::LowerBound<fst::ArcIterator<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > > >(fst::ArcIterator<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >*, long, long, int) const () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#1  0x0000007fb6b00578 in bool fst::LabelReachable<fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::FastLogAccumulator<fst::ArcTpl<fst::TropicalWeightTpl<float> > >, fst::LabelReachableData<int> >::Reach<fst::ArcIterator<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > > >(fst::ArcIterator<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >*, long, long, bool) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#2  0x0000007fb6b009f8 in bool fst::LabelLookAheadMatcher<fst::SortedMatcher<fst::ConstFst<fst::ArcTpl<fst::TropicalWeightTpl<float> >, unsigned int> >, 1760u, fst::FastLogAccumulator<fst::ArcTpl<fst::TropicalWeightTpl<float> > >, fst::LabelReachable<fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::FastLogAccumulator<fst::ArcTpl<fst::TropicalWeightTpl<float> > >, fst::LabelReachableData<int> > >::LookAheadFst<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >(fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > const&, int) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#3  0x0000007fb6ab9cac in fst::PushWeightsComposeFilter<fst::LookAheadComposeFilter<fst::AltSequenceComposeFilter<fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, (fst::MatchType)3>, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, (fst::MatchType)3>::FilterArc(fst::ArcTpl<fst::TropicalWeightTpl<float> >*, fst::ArcTpl<fst::TropicalWeightTpl<float> >*) const () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#4  0x0000007fb6ad07cc in void fst::internal::ComposeFstImpl<fst::DefaultCacheStore<fst::ArcTpl<fst::TropicalWeightTpl<float> > >, fst::PushLabelsComposeFilter<fst::PushWeightsComposeFilter<fst::LookAheadComposeFilter<fst::AltSequenceComposeFilter<fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, (fst::MatchType)3>, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, (fst::MatchType)3>, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, (fst::MatchType)3>, fst::GenericComposeStateTable<fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::PairFilterState<fst::PairFilterState<fst::IntegerFilterState<signed char>, fst::WeightFilterState<fst::TropicalWeightTpl<float> > >, fst::IntegerFilterState<int> >, fst::DefaultComposeStateTuple<int, fst::PairFilterState<fst::PairFilterState<fst::IntegerFilterState<signed char>, fst::WeightFilterState<fst::TropicalWeightTpl<float> > >, fst::IntegerFilterState<int> > >, fst::CompactHashStateTable<fst::DefaultComposeStateTuple<int, fst::PairFilterState<fst::PairFilterState<fst::IntegerFilterState<signed char>, fst::WeightFilterState<fst::TropicalWeightTpl<float> > >, fst::IntegerFilterState<int> > >, fst::ComposeHash<fst::DefaultComposeStateTuple<int, fst::PairFilterState<fst::PairFilterState<fst::IntegerFilterState<signed char>, fst::WeightFilterState<fst::TropicalWeightTpl<float> > >, fst::IntegerFilterState<int> > > > > > >::MatchArc<fst::MultiEpsMatcher<fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > > > >(int, fst::MultiEpsMatcher<fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > > >*, fst::ArcTpl<fst::TropicalWeightTpl<float> > const&, bool) ()
   from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#5  0x0000007fb6ad0ba4 in ?? () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#6  0x0000007fb6ad0e8c in fst::internal::ComposeFstImpl<fst::DefaultCacheStore<fst::ArcTpl<fst::TropicalWeightTpl<float> > >, fst::PushLabelsComposeFilter<fst::PushWeightsComposeFilter<fst::LookAheadComposeFilter<fst::AltSequenceComposeFilter<fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, (fst::MatchType)3>, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, (fst::MatchType)3>, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, fst::LookAheadMatcher<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >, (fst::MatchType)3>, fst::GenericComposeStateTable<fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::PairFilterState<fst::PairFilterState<fst::IntegerFilterState<signed char>, fst::WeightFilterState<fst::TropicalWeightTpl<float> > >, fst::IntegerFilterState<int> >, fst::DefaultComposeStateTuple<int, fst::PairFilterState<fst::PairFilterState<fst::IntegerFilterState<signed char>, fst::WeightFilterState<fst::TropicalWeightTpl<float> > >, fst::IntegerFilterState<int> > >, fst::CompactHashStateTable<fst::DefaultComposeStateTuple<int, fst::PairFilterState<fst::PairFilterState<fst::IntegerFilterState<signed char>, fst::WeightFilterState<fst::TropicalWeightTpl<float> > >, fst::IntegerFilterState<int> > >, fst::ComposeHash<fst::DefaultComposeStateTuple<int, fst::PairFilterState<fst::PairFilterState<fst::IntegerFilterState<signed char>, fst::WeightFilterState<fst::TropicalWeightTpl<float> > >, fst::IntegerFilterState<int> > > > > > >::Expand(int) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#7  0x0000007fb6a825a8 in fst::ComposeFst<fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::DefaultCacheStore<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >::InitArcIterator(int, fst::ArcIteratorData<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*) const () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#8  0x0000007fb6a8ac10 in fst::ArcIterator<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >::ArcIterator(fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > const&, int) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#9  0x0000007fb6acee64 in fst::internal::ArcMapFstImpl<fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::RemoveSomeInputSymbolsMapper<fst::ArcTpl<fst::TropicalWeightTpl<float> >, int> >::Expand(int) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#10 0x0000007fb6acf3f0 in fst::ImplToFst<fst::internal::ArcMapFstImpl<fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::ArcTpl<fst::TropicalWeightTpl<float> >, fst::RemoveSomeInputSymbolsMapper<fst::ArcTpl<fst::TropicalWeightTpl<float> >, int> >, fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >::NumInputEpsilons(int) const
    () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#11 0x0000007fb6b71ea0 in kaldi::LatticeIncrementalDecoderTpl<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >, kaldi::decoder::BackpointerToken>::ProcessNonemit
---Type <return> to continue, or q <return> to quit---
ting(float) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#12 0x0000007fb6b86c90 in kaldi::LatticeIncrementalDecoderTpl<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >, kaldi::decoder::BackpointerToken>::AdvanceDecoding(kaldi::DecodableInterface*, int) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#13 0x0000007fb6b0de5c in kaldi::SingleUtteranceNnet3IncrementalDecoderTpl<fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > >::AdvanceDecoding() ()
   from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#14 0x0000007fb6a6c268 in Recognizer::AcceptWaveform(kaldi::Vector<float>&) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#15 0x0000007fb6a6c440 in Recognizer::AcceptWaveform(char const*, int) () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#16 0x0000007fb6b0304c in vosk_recognizer_accept_waveform () from /home/vosk_tests/node_modules/vosk/lib/linux-x86_64/libvosk.so
#17 0x0000007fb7508a28 in ffi_call_SYSV () from /home/vosk_tests/node_modules/ffi-napi/build/Release/ffi_bindings.node
#18 0x0000007fb7507bdc in ffi_call_int () from /home/vosk_tests/node_modules/ffi-napi/build/Release/ffi_bindings.node
#19 0x0000007fb74fc558 in FFI::FFI::FFICall(Napi::CallbackInfo const&) () from /home/vosk_tests/node_modules/ffi-napi/build/Release/ffi_bindings.node
#20 0x0000007fb7500830 in Napi::details::CallbackData<void (*)(Napi::CallbackInfo const&), void>::Wrapper(napi_env__*, napi_callback_info__*) ()
   from /home/vosk_tests/node_modules/ffi-napi/build/Release/ffi_bindings.node
#21 0x0000000000aa5a74 in v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) ()
#22 0x0000000000d28550 in v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) ()
#23 0x0000000000d295ac in v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) ()
#24 0x00000000015474ac in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

nshmyrev added a commit that referenced this issue Jun 27, 2022
@nshmyrev
Wait for microphone to finalize. Fixes crash in issue #1034
58fa98c
@nshmyrev
Copy link
Collaborator

This is helpful, thank you. The crash must be fixed now with last update when we wait for microphone o finalize.

@sidor926
Copy link
Author

sidor926 commented Jul 1, 2022

Thank you for the resolution! One follow up though on commit #1034: Unless I call process.exit() explicitly on audioProcessExitComplete, I get recording audioProcess has exited with code = 1. I'm fine with calling process.exit() explicitly- and gdb doesn't show any problems- but is this desirable? Is it potentially something to do with the mic package not propagating the SIGTERM/SIGINT upwards correctly?

micInputStream.on('audioProcessExitComplete', function(args) {
console.log('audio process exit complete info: ', args);
console.log("Cleaning up");
console.log(rec.finalResult());
rec.free();
model.free();
// process.exit();
});

@nshmyrev
Copy link
Collaborator

nshmyrev commented Jul 2, 2022

It is a minor thing. Honestly I'd better use different microphone recording library that doesn't spawn processes. Maybe https://www.npmjs.com/package/node-portaudio

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nshmyrev @sidor926