SIGSEGV on simulation exit

[virtuald]

Happening in 2024.2.1, seen in CI. Really hard to reproduce locally, but I caught this:

(gdb) bt
#0  ___pthread_mutex_lock (mutex=0x7f5cac038) at pthread_mutex_lock.c:80
#1  0x00007f5cb569a7e0 in __gthread_mutex_lock (__mutex=0x7f5cac038)
    at /usr/include/x86_64-linux-gnu/c++/11/bits/gthr-default.h:749
#2  std::mutex::lock (this=0x7f5cac038) at /usr/include/c++/11/bits/std_mutex.h:100
#3  std::scoped_lock<std::mutex>::scoped_lock (__m=..., this=<synthetic pointer>)
    at /usr/include/c++/11/mutex:655
#4  HAL_StopNotifier (notifierHandle=<optimized out>, status=<optimized out>)
    at /work/hal/src/main/native/sim/Notifier.cpp:200
#5  0x00007f5cb122f541 in (anonymous namespace)::HeartbeatDaemon::Main() ()
   from /home/virtuald/src/frc/robotpy-rev/rev/_rev.cpython-38-x86_64-linux-gnu.so
#6  0x00007f5cb5939c4b in operator() (__closure=0x5587df615988)
    at /work/wpiutil/src/main/native/cpp/SafeThread.cpp:79
#7  std::__invoke_impl<void, wpi::detail::SafeThreadOwnerBase::Start(std::shared_ptr<wpi::SafeThreadBase>)::<lambda()> > (__f=...) at /usr/include/c++/11/bits/invoke.h:61
#8  std::__invoke<wpi::detail::SafeThreadOwnerBase::Start(std::shared_ptr<wpi::SafeThreadBase>)::<lambda()> > (__fn=...) at /usr/include/c++/11/bits/invoke.h:96
#9  std::thread::_Invoker<std::tuple<wpi::detail::SafeThreadOwnerBase::Start(std::shared_ptr<wpi::SafeThreadBase>)::<lambda()> > >::_M_invoke<0> (this=0x5587df615988) at /usr/include/c++/11/bits/std_thread.h:259
#10 std::thread::_Invoker<std::tuple<wpi::detail::SafeThreadOwnerBase::Start(std::shared_ptr<wpi::SafeThreadBase>)::<lambda()> > >::operator() (this=0x5587df615988) at /usr/include/c++/11/bits/std_thread.h:266
#11 std::thread::_State_impl<std::thread::_Invoker<std::tuple<wpi::detail::SafeThreadOwnerBase::Start(std::shared_ptr<wpi::SafeThreadBase>)::<lambda()> > > >::_M_run(void) (this=0x5587df615980)
    at /usr/include/c++/11/bits/std_thread.h:211
#12 0x00007f5cb54e31e3 in std::execute_native_thread_routine (__p=0x5587df615980)
    at ../../../../../libstdc++-v3/src/c++11/thread.cc:104
#13 0x00007f5cc36ac897 in start_thread (arg=<optimized out>) at pthread_create.c:444
#14 0x00007f5cc373380c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

[virtuald]

If rev had a shutdown function we could just call it as everything is being torn down, and that would likely address this issue.

[auscompgeek]

It's been a while since I saw something like this, but I just saw this abort when exiting sim on macOS:

libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument

Thread 23 Crashed:
0   libsystem_kernel.dylib        	       0x18bba3720 __pthread_kill + 8
1   libsystem_pthread.dylib       	       0x18bbdbf70 pthread_kill + 288
2   libsystem_c.dylib             	       0x18bae8908 abort + 128
3   libc++abi.dylib               	       0x18bb9244c abort_message + 132
4   libc++abi.dylib               	       0x18bb80a24 demangling_terminate_handler() + 320
5   libobjc.A.dylib               	       0x18b8293f4 _objc_terminate() + 172
6   libc++abi.dylib               	       0x18bb91710 std::__terminate(void (*)()) + 16
7   libc++abi.dylib               	       0x18bb94cdc __cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*) + 88
8   libc++abi.dylib               	       0x18bb94c84 __cxa_throw + 92
9   libc++.1.dylib                	       0x18bb11a44 std::__1::__throw_system_error[abi:ne180100](std::__1::error_code, char const*) + 92
10  libc++.1.dylib                	       0x18bb119e8 std::__1::__throw_system_error(int, char const*) + 48
11  libc++.1.dylib                	       0x18bb13770 std::__1::mutex::lock() + 40
12  _rev.cpython-313-darwin.so    	       0x107b63cc8 c_REVLib_ErrorSize + 48
13  _rev.cpython-313-darwin.so    	       0x107b62984 (anonymous namespace)::REVLibDaemon::Main() + 132
14  libwpiutil.dylib              	       0x102633cf8 void* std::__1::__thread_proxy[abi:ue170006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, wpi::detail::SafeThreadOwnerBase::Start(std::__1::shared_ptr<wpi::SafeThreadBase>)::$_0>>(void*) + 100
15  libsystem_pthread.dylib       	       0x18bbdc2e4 _pthread_start + 136
16  libsystem_pthread.dylib       	       0x18bbd70fc thread_start + 8

I'm guessing HeartbeatDaemon got renamed to REVLibDaemon in the 2025 rewrite. Probably related given this stack is also in a mutex lock.