New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stopping a framework while bundle threads are still running may deadlock #210

Closed
saschazelzer opened this Issue May 26, 2017 · 0 comments

Comments

Projects
1 participant
@saschazelzer
Member

saschazelzer commented May 26, 2017

The valgrind memcheck test for the BundleRegistryPerformanceTest sometimes times out on the Travis CI infrastructure. I have also experienced this lately on my local system. valgrind forces the code to run on a single processor inside its VM, so the issue is more likely to appear:

Thread #0: the main thread implicitly waits on the stopped framework (by waiting for a stop event from the shutdown thread #1), because the framework variable goes out of scope.

#0  0x0000000005f70460 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x000000000577050c in std::condition_variable::wait(std::unique_lock<std::mutex>&) () from /lib64/libstdc++.so.6
#2  0x0000000004f1675f in std::condition_variable::wait<cppmicroservices::FrameworkPrivate::WaitForStop(const milliseconds&)::<lambda()> >(std::unique_lock<std::mutex> &, cppmicroservices::FrameworkPrivate::<lambda()>) (this=0x655e480, 
    __lock=..., __p=...) at /usr/include/c++/6.3.1/condition_variable:99
#3  0x0000000004f1666a in cppmicroservices::detail::WaitCondition<cppmicroservices::detail::MutexLockingStrategy<std::mutex> >::Wait<cppmicroservices::FrameworkPrivate::WaitForStop(const milliseconds&)::<lambda()> >(cppmicroservices::detail::MutexLockingStrategy<std::mutex>::UniqueLock &, cppmicroservices::FrameworkPrivate::<lambda()>) (this=0x655e480, 
    lock=..., pred=...)
    at /home/sascha/git/code/CppMicroServices/framework/include/cppmicroservices/detail/WaitCondition.h:53
#4  0x0000000004f14b21 in cppmicroservices::FrameworkPrivate::WaitForStop (this=0x655e450, timeout=...)
    at /home/sascha/git/code/CppMicroServices/framework/src/util/FrameworkPrivate.cpp:118
#5  0x0000000004f124c2 in cppmicroservices::CoreBundleContextHolder::~CoreBundleContextHolder (this=0x655e8b0, 
    __in_chrg=<optimized out>) at /home/sascha/git/code/CppMicroServices/framework/src/util/FrameworkFactory.cpp:58
#6  0x0000000004f140d7 in __gnu_cxx::new_allocator<cppmicroservices::CoreBundleContextHolder>::destroy<cppmicroservices::CoreBundleContextHolder> (this=0x655e8b0, __p=0x655e8b0) at /usr/include/c++/6.3.1/ext/new_allocator.h:124
#7  0x0000000004f1405f in std::allocator_traits<std::allocator<cppmicroservices::CoreBundleContextHolder> >::destroy<cppmicroservices::CoreBundleContextHolder> (__a=..., __p=0x655e8b0) at /usr/include/c++/6.3.1/bits/alloc_traits.h:467
#8  0x0000000004f13ee2 in std::_Sp_counted_ptr_inplace<cppmicroservices::CoreBundleContextHolder, std::allocator<cppmicroservices::CoreBundleContextHolder>, (__gnu_cxx::_Lock_policy)2>::_M_dispose (this=0x655e8a0)
    at /usr/include/c++/6.3.1/bits/shared_ptr_base.h:529
#9  0x0000000000500141 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x655e8a0)
    at /usr/include/c++/6.3.1/bits/shared_ptr_base.h:150
#10 0x00000000004ff77c in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=0xffefffa00, 
    __in_chrg=<optimized out>) at /usr/include/c++/6.3.1/bits/shared_ptr_base.h:662
#11 0x0000000004f122e3 in std::__shared_ptr<cppmicroservices::CoreBundleContext, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0xffefff9f8, __in_chrg=<optimized out>) at /usr/include/c++/6.3.1/bits/shared_ptr_base.h:928
#12 0x0000000004f12321 in std::shared_ptr<cppmicroservices::CoreBundleContext>::~shared_ptr (this=0xffefff9f8, 
    __in_chrg=<optimized out>) at /usr/include/c++/6.3.1/bits/shared_ptr.h:93
#13 0x0000000004fa6d51 in cppmicroservices::Bundle::~Bundle (this=0xffefff9e0, __in_chrg=<optimized out>)
    at /home/sascha/git/code/CppMicroServices/framework/src/bundle/Bundle.cpp:108
#14 0x0000000000502767 in cppmicroservices::Framework::~Framework (this=0xffefff9e0, __in_chrg=<optimized out>)
    at /home/sascha/git/code/CppMicroServices/framework/include/cppmicroservices/Framework.h:63
#15 0x00000000004fe08f in BundleRegistryPerformanceTest ()
    at /home/sascha/git/code/CppMicroServices/framework/test/BundleRegistryPerformanceTest.cpp:140
#16 0x00000000004d7c99 in main (ac=1, av=0xffefffc00)
    at /home/sascha/builds/CppMicroServices-dev-shared-debug/framework/test/usFrameworkTestDriver.cpp:315

Thread #1: the framework shutdown thread joins all bundle threads, while holding the CoreBundleContext::bundleThreads lock.

#0  0x0000000005f6b96d in pthread_join () from /lib64/libpthread.so.0
#1  0x0000000005776877 in std::thread::join() () from /lib64/libstdc++.so.6
#2  0x0000000004ff1cb5 in cppmicroservices::BundleThread::Quit (this=0x65b61a0)
    at /home/sascha/git/code/CppMicroServices/framework/src/bundle/BundleThread.cpp:63
#3  0x0000000005000502 in cppmicroservices::CoreBundleContext::Uninit1 (this=0x655d5c0)
    at /home/sascha/git/code/CppMicroServices/framework/src/bundle/CoreBundleContext.cpp:176
#4  0x0000000004f15eca in cppmicroservices::FrameworkPrivate::Shutdown0 (this=0x655e450, restart=false, 
    wasActive=true) at /home/sascha/git/code/CppMicroServices/framework/src/util/FrameworkPrivate.cpp:287
#5  0x0000000004f1e0a0 in std::__invoke_impl<void, void (cppmicroservices::FrameworkPrivate::* const&)(bool, bool), cppmicroservices::FrameworkPrivate*&, bool&, bool&> (__f=
    @0x37a5b068: (void (cppmicroservices::FrameworkPrivate::*)(cppmicroservices::FrameworkPrivate * const, bool, bool)) 0x4f15c88 <cppmicroservices::FrameworkPrivate::Shutdown0(bool, bool)>, __t=@0x37a5b080: 0x655e450, 
    __args#0=@0x37a5b079: false, __args#1=@0x37a5b078: true) at /usr/include/c++/6.3.1/functional:227
#6  0x0000000004f1dfc7 in std::__invoke<void (cppmicroservices::FrameworkPrivate::* const&)(bool, bool), cppmicroservices::FrameworkPrivate*&, bool&, bool&> (__fn=
    @0x37a5b068: (void (cppmicroservices::FrameworkPrivate::*)(cppmicroservices::FrameworkPrivate * const, bool, bool)) 0x4f15c88 <cppmicroservices::FrameworkPrivate::Shutdown0(bool, bool)>, __args#0=@0x37a5b080: 0x655e450, 
    __args#1=@0x37a5b079: false, __args#2=@0x37a5b078: true) at /usr/include/c++/6.3.1/functional:251
#7  0x0000000004f1def7 in std::_Mem_fn_base<void (cppmicroservices::FrameworkPrivate::*)(bool, bool), true>::operator()<cppmicroservices::FrameworkPrivate*&, bool&, bool&> (this=0x37a5b068, __args#0=@0x37a5b080: 0x655e450, 
    __args#1=@0x37a5b079: false, __args#2=@0x37a5b078: true) at /usr/include/c++/6.3.1/functional:604
#8  0x0000000004f1dde4 in std::_Bind<std::_Mem_fn<void (cppmicroservices::FrameworkPrivate::*)(bool, bool)> (cppmicroservices::FrameworkPrivate*, bool, bool)>::__call<void, , 0ul, 1ul, 2ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul, 2ul>) (this=0x37a5b068, 
    __args=<unknown type in /home/sascha/builds/CppMicroServices-dev-shared-debug/lib/libCppMicroServicesd.so.3.1.0, CU 0x714a5, DIE 0xb98bc>) at /usr/include/c++/6.3.1/functional:934
#9  0x0000000004f1dba1 in std::_Bind<std::_Mem_fn<void (cppmicroservices::FrameworkPrivate::*)(bool, bool)> (cppmicroservices::FrameworkPrivate*, bool, bool)>::operator()<, void>() (this=0x37a5b068)
    at /usr/include/c++/6.3.1/functional:993
#10 0x0000000004f1da66 in std::_Bind_simple<std::_Bind<std::_Mem_fn<void (cppmicroservices::FrameworkPrivate::*)(bool, bool)> (cppmicroservices::FrameworkPrivate*, bool, bool)> ()>::_M_invoke<>(std::_Index_tuple<>) (this=0x37a5b068)
    at /usr/include/c++/6.3.1/functional:1391
#11 0x0000000004f1d8da in std::_Bind_simple<std::_Bind<std::_Mem_fn<void (cppmicroservices::FrameworkPrivate::*)(bool, bool)> (cppmicroservices::FrameworkPrivate*, bool, bool)> ()>::operator()() (this=0x37a5b068)
    at /usr/include/c++/6.3.1/functional:1380
#12 0x0000000004f1d7c5 in std::thread::_State_impl<std::_Bind_simple<std::_Bind<std::_Mem_fn<void (cppmicroservices::FrameworkPrivate::*)(bool, bool)> (cppmicroservices::FrameworkPrivate*, bool, bool)> ()> >::_M_run() (this=0x37a5b060)
    at /usr/include/c++/6.3.1/thread:196
#13 0x00000000057765cf in ?? () from /lib64/libstdc++.so.6
#14 0x0000000005f6a6ca in start_thread () from /lib64/libpthread.so.0
#15 0x0000000006288f7f in clone () from /lib64/libc.so.6

Thread #2: a bundle thread, while being stopped, tries to lock CoreBundleContext::bundleThreads, but the lock is acquired already in thread #1.

#0  0x0000000005f7338d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x0000000005f6ce4d in pthread_mutex_lock () from /lib64/libpthread.so.0
#2  0x00000000004fc7eb in __gthread_mutex_lock (__mutex=0x655d610)
    at /usr/include/c++/6.3.1/x86_64-redhat-linux/bits/gthr-default.h:748
#3  0x00000000004ff31f in std::mutex::lock (this=0x655d610) at /usr/include/c++/6.3.1/bits/std_mutex.h:103
#4  0x000000000050021a in std::unique_lock<std::mutex>::lock (this=0x7146d40)
    at /usr/include/c++/6.3.1/bits/std_mutex.h:267
#5  0x00000000004ff88d in std::unique_lock<std::mutex>::unique_lock (this=0x7146d40, __m=...)
    at /usr/include/c++/6.3.1/bits/std_mutex.h:197
#6  0x0000000004f18cd6 in cppmicroservices::detail::MutexLockingStrategy<std::mutex>::UniqueLock::UniqueLock (
    this=0x7146d40, host=0x655d610)
    at /home/sascha/git/code/CppMicroServices/framework/include/cppmicroservices/detail/Threads.h:86
#7  0x0000000004f17af8 in cppmicroservices::detail::MutexLockingStrategy<std::mutex>::Lock (this=0x655d610)
    at /home/sascha/git/code/CppMicroServices/framework/include/cppmicroservices/detail/Threads.h:147
#8  0x0000000004ff1f7f in cppmicroservices::BundleThread::Run (this=0x65b61a0)
    at /home/sascha/git/code/CppMicroServices/framework/src/bundle/BundleThread.cpp:94
#9  0x0000000004ffb3cd in std::__invoke_impl<void, void (cppmicroservices::BundleThread::* const&)(), cppmicroservices::BundleThread*>(std::__invoke_memfun_deref, void (cppmicroservices::BundleThread::* const&)(), cppmicroservices::BundleThread*&&) (
    __f=@0x65b63e0: (void (cppmicroservices::BundleThread::*)(cppmicroservices::BundleThread * const)) 0x4ff1cfa <cppmicroservices::BundleThread::Run()>, 
    __t=<unknown type in /home/sascha/builds/CppMicroServices-dev-shared-debug/lib/libCppMicroServicesd.so.3.1.0, CU 0x649eba, DIE 0x68bbad>) at /usr/include/c++/6.3.1/functional:227
#10 0x0000000004ffb330 in std::__invoke<void (cppmicroservices::BundleThread::* const&)(), cppmicroservices::BundleThread*>(void (cppmicroservices::BundleThread::* const&)(), cppmicroservices::BundleThread*&&) (
    __fn=@0x65b63e0: (void (cppmicroservices::BundleThread::*)(cppmicroservices::BundleThread * const)) 0x4ff1cfa <cppmicroservices::BundleThread::Run()>, 
    __args#0=<unknown type in /home/sascha/builds/CppMicroServices-dev-shared-debug/lib/libCppMicroServicesd.so.3.1.0, CU 0x649eba, DIE 0x68bbad>) at /usr/include/c++/6.3.1/functional:251
#11 0x0000000004ffb297 in std::_Mem_fn_base<void (cppmicroservices::BundleThread::*)(), true>::operator()<cppmicroservices::BundleThread*>(cppmicroservices::BundleThread*&&) const (this=0x65b63e0, 
    __args#0=<unknown type in /home/sascha/builds/CppMicroServices-dev-shared-debug/lib/libCppMicroServicesd.so.3.1.0, CU 0x649eba, DIE 0x68bbad>) at /usr/include/c++/6.3.1/functional:604
#12 0x0000000004ffb23f in std::_Bind_simple<std::_Mem_fn<void (cppmicroservices::BundleThread::*)()> (cppmicroservices::BundleThread*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x65b63d8) at /usr/include/c++/6.3.1/functional:1391
#13 0x0000000004ffb058 in std::_Bind_simple<std::_Mem_fn<void (cppmicroservices::BundleThread::*)()> (cppmicroservices::BundleThread*)>::operator()() (this=0x65b63d8) at /usr/include/c++/6.3.1/functional:1380
#14 0x0000000004ffae8d in std::thread::_State_impl<std::_Bind_simple<std::_Mem_fn<void (cppmicroservices::BundleThread::*)()> (cppmicroservices::BundleThread*)> >::_M_run() (this=0x65b63d0) at /usr/include/c++/6.3.1/thread:196
#15 0x00000000057765cf in ?? () from /lib64/libstdc++.so.6
#16 0x0000000005f6a6ca in start_thread () from /lib64/libpthread.so.0
#17 0x0000000006288f7f in clone () from /lib64/libc.so.6

@saschazelzer saschazelzer self-assigned this May 26, 2017

@saschazelzer saschazelzer changed the title from Stopping a framework while bundle threads a still running may deadlock to Stopping a framework while bundle threads are still running may deadlock May 26, 2017

saschazelzer added a commit that referenced this issue May 26, 2017

@saschazelzer saschazelzer added this to In Progress in Release 3.1 May 26, 2017

@jeffdiclemente jeffdiclemente moved this from In Progress to Done in Release 3.1 Jun 1, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment