Change wrong package name. #10
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
eos-fuse-sysv package's name was changed 2 month ago.
apeters1971
pushed a commit
that referenced
this pull request
Oct 13, 2017
Since making cap_dec atomic, the lock is no longer required.
Fixes the following:
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=14178)
Cycle in lock order graph: M158887504752680392 (0x000000000000) => M335 (0x0000007dae70) => M158887504752680392
Mutex M335 acquired here while holding mutex M158887504752680392 in thread T41:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /usr/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x0000004ea2c9)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex&) /usr/include/xrootd/XrdSys/XrdSysPthread.hh:208 (eosxd+0x0000004ea2c9)
#3 cap::get(fuse_req*, unsigned long, bool) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:187 (eosxd+0x0000004ea2c9)
#4 cap::acquire(fuse_req*, unsigned long, unsigned int, bool) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:334 (eosxd+0x0000004ee1ec)
#5 EosFuse::lookup(fuse_req*, unsigned long, char const*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:1177 (eosxd+0x00000045ef1b)
#6 <null> <null> (libfuse.so.2+0x000000016042)
Mutex M158887504752680392 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /usr/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x00000045ee2e)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex&) /usr/include/xrootd/XrdSys/XrdSysPthread.hh:208 (eosxd+0x00000045ee2e)
#3 EosFuse::lookup(fuse_req*, unsigned long, char const*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:1170 (eosxd+0x00000045ee2e)
#4 <null> <null> (libfuse.so.2+0x000000016042)
Mutex M158887504752680392 acquired here while holding mutex M335 in thread T31:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /usr/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x0000004eb942)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex&) /usr/include/xrootd/XrdSys/XrdSysPthread.hh:208 (eosxd+0x0000004eb942)
#3 metad::decrease_cap(unsigned long) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./md/md.hh:604 (eosxd+0x0000004eb942)
#4 cap::capflush() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:494 (eosxd+0x0000004eb942)
#5 void std::__invoke_impl<void, void (cap::*)(), cap*>(std::__invoke_memfun_deref, void (cap::*&&)(), cap*&&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x00000047c250)
#6 std::__invoke_result<void (cap::*)(), cap*>::type std::__invoke<void (cap::*)(), cap*>(void (cap::*&&)(), cap*&&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x00000047c250)
#7 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)())) std::thread::_Invoker<std::tuple<void (cap::*)(), cap*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/include/c++/7/thread:234 (eosxd+0x00000047c250)
#8 std::thread::_Invoker<std::tuple<void (cap::*)(), cap*> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x00000047c250)
#9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (cap::*)(), cap*> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x00000047c250)
#10 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Mutex M335 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /usr/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x0000004eb428)
#2 cap::capflush() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:485 (eosxd+0x0000004eb428)
#3 void std::__invoke_impl<void, void (cap::*)(), cap*>(std::__invoke_memfun_deref, void (cap::*&&)(), cap*&&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x00000047c250)
#4 std::__invoke_result<void (cap::*)(), cap*>::type std::__invoke<void (cap::*)(), cap*>(void (cap::*&&)(), cap*&&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x00000047c250)
#5 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)())) std::thread::_Invoker<std::tuple<void (cap::*)(), cap*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/include/c++/7/thread:234 (eosxd+0x00000047c250)
#6 std::thread::_Invoker<std::tuple<void (cap::*)(), cap*> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x00000047c250)
#7 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (cap::*)(), cap*> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x00000047c250)
#8 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Thread T41 (tid=14455, running) created by thread T40 at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 <null> <null> (libfuse.so.2+0x000000012984)
Thread T31 (tid=14229, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0x0000000bc304)
#2 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:31 (eosxd+0x000000457f38)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/lib64/libtsan.so.0+0x3b62e) in pthread_mutex_lock
apeters1971
pushed a commit
that referenced
this pull request
Feb 6, 2018
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=30041)
Cycle in lock order graph: M1223 (0x000000b11310) => M1243 (0x000000b11700) => M1223
Mutex M1243 acquired here while holding mutex M1223 in thread T30:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x000000623d40)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex&) /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:208 (eosxd+0x000000623d40)
#3 cap::get(unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:221 (eosxd+0x000000623d40)
#4 metad::mdcommunicate(ThreadAssistant&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/md/md.cc:2294 (eosxd+0x000000617c59)
#5 void std::__invoke_impl<void, void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>(std::__invoke_memfun_deref, void (metad::*&&)(ThreadAssistant&), metad*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x0000005a94a6)
#6 std::__invoke_result<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>::type std::__invoke<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>(void (metad::*&&)(ThreadAssistant&), metad*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x0000005a94a6)
#7 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/thread:234 (eosxd+0x0000005a94a6)
#8 std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x0000005a94a6)
#9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x0000005a94a6)
#10 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Mutex M1223 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x000000617942)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex&) /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:208 (eosxd+0x000000617942)
#3 metad::mdcommunicate(ThreadAssistant&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/md/md.cc:2271 (eosxd+0x000000617942)
#4 void std::__invoke_impl<void, void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>(std::__invoke_memfun_deref, void (metad::*&&)(ThreadAssistant&), metad*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x0000005a94a6)
#5 std::__invoke_result<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>::type std::__invoke<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>(void (metad::*&&)(ThreadAssistant&), metad*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x0000005a94a6)
#6 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/thread:234 (eosxd+0x0000005a94a6)
#7 std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x0000005a94a6)
#8 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x0000005a94a6)
#9 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Mutex M1223 acquired here while holding mutex M1243 in thread T31:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x000000622946)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex*) /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:204 (eosxd+0x000000622946)
#3 metad::pmap::retrieveTS(unsigned long, std::shared_ptr<metad::mdx>&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./md/md.hh:322 (eosxd+0x000000622946)
#4 metad::decrease_cap(unsigned long) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./md/md.hh:539 (eosxd+0x000000622946)
#5 cap::capflush(ThreadAssistant&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:537 (eosxd+0x000000622946)
#6 void std::__invoke_impl<void, void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&>(std::__invoke_memfun_deref, void (cap::*&&)(ThreadAssistant&), cap*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x0000005a9406)
#7 std::__invoke_result<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&>::type std::__invoke<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&>(void (cap::*&&)(ThreadAssistant&), cap*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x0000005a9406)
#8 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/thread:234 (eosxd+0x0000005a9406)
#9 std::thread::_Invoker<std::tuple<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x0000005a9406)
#10 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x0000005a9406)
#11 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Mutex M1243 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x000000622582)
#2 cap::capflush(ThreadAssistant&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:528 (eosxd+0x000000622582)
#3 void std::__invoke_impl<void, void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&>(std::__invoke_memfun_deref, void (cap::*&&)(ThreadAssistant&), cap*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x0000005a9406)
#4 std::__invoke_result<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&>::type std::__invoke<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&>(void (cap::*&&)(ThreadAssistant&), cap*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x0000005a9406)
#5 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/thread:234 (eosxd+0x0000005a9406)
#6 std::thread::_Invoker<std::tuple<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x0000005a9406)
#7 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (cap::*)(ThreadAssistant&), cap*, ThreadAssistant&> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x0000005a9406)
#8 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Thread T30 (tid=30083, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0x0000000bc304)
#2 EosFuse::run(int, char**, void*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:631 (eosxd+0x00000058b69e)
#3 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:30 (eosxd+0x0000004d8db8)
Thread T31 (tid=30084, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0x0000000bc304)
#2 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:30 (eosxd+0x0000004d8db8)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/lib64/libtsan.so.0+0x3b62e) in pthread_mutex_lock
apeters1971
pushed a commit
that referenced
this pull request
Feb 6, 2018
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=5361)
Cycle in lock order graph: M495531576898552008 (0x000000000000) => M1224 (0x000000b1d370) => M495531576898552008
Mutex M1224 acquired here while holding mutex M495531576898552008 in thread T39:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x00000060da11)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex*) /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:204 (eosxd+0x00000060da11)
#3 metad::pmap::insertTS(unsigned long, std::shared_ptr<metad::mdx>&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./md/md.hh:339 (eosxd+0x00000060da11)
#4 metad::insert(fuse_req*, std::shared_ptr<metad::mdx>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/md/md.cc:778 (eosxd+0x00000060da11)
#5 EosFuse::mkdir(fuse_req*, unsigned long, char const*, unsigned int) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:2094 (eosxd+0x000000598743)
#6 <null> <null> (libfuse.so.2+0x000000016042)
Mutex M495531576898552008 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x0000005982e4)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex&) /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:208 (eosxd+0x0000005982e4)
#3 EosFuse::mkdir(fuse_req*, unsigned long, char const*, unsigned int) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:2062 (eosxd+0x0000005982e4)
#4 <null> <null> (libfuse.so.2+0x000000016042)
Mutex M495531576898552008 acquired here while holding mutex M1224 in thread T30:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x00000061451d)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex&) /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:208 (eosxd+0x00000061451d)
#3 metad::add(fuse_req*, std::shared_ptr<metad::mdx>, std::shared_ptr<metad::mdx>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/md/md.cc:866 (eosxd+0x00000061451d)
#4 metad::mdcommunicate(ThreadAssistant&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/md/md.cc:2264 (eosxd+0x00000062014f)
#5 void std::__invoke_impl<void, void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>(std::__invoke_memfun_deref, void (metad::*&&)(ThreadAssistant&), metad*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x0000005b1ef6)
#6 std::__invoke_result<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>::type std::__invoke<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>(void (metad::*&&)(ThreadAssistant&), metad*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x0000005b1ef6)
#7 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/thread:234 (eosxd+0x0000005b1ef6)
#8 std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x0000005b1ef6)
#9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x0000005b1ef6)
#10 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Mutex M1224 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x00000003b62e)
#1 XrdSysMutex::Lock() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:149 (eosxd+0x00000061f8d2)
#2 XrdSysMutexHelper::XrdSysMutexHelper(XrdSysMutex&) /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:208 (eosxd+0x00000061f8d2)
#3 metad::mdcommunicate(ThreadAssistant&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/md/md.cc:2251 (eosxd+0x00000061f8d2)
#4 void std::__invoke_impl<void, void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>(std::__invoke_memfun_deref, void (metad::*&&)(ThreadAssistant&), metad*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x0000005b1ef6)
#5 std::__invoke_result<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>::type std::__invoke<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&>(void (metad::*&&)(ThreadAssistant&), metad*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x0000005b1ef6)
#6 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/thread:234 (eosxd+0x0000005b1ef6)
#7 std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x0000005b1ef6)
#8 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (metad::*)(ThreadAssistant&), metad*, ThreadAssistant&> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x0000005b1ef6)
#9 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Thread T39 (tid=5501, running) created by thread T33 at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 <null> <null> (libfuse.so.2+0x000000012984)
Thread T30 (tid=5401, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0x0000000bc304)
#2 EosFuse::run(int, char**, void*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:914 (eosxd+0x000000593299)
#3 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:30 (eosxd+0x0000004dab58)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/lib64/libtsan.so.0+0x3b62e) in pthread_mutex_lock
apeters1971
pushed a commit
that referenced
this pull request
Feb 6, 2018
Fixes the following, among many others. (Observed when trying to
compile quarkdb on /eos)
WARNING: ThreadSanitizer: data race (pid=10251)
Write of size 8 at 0x7b380018d780 by thread T23 (mutexes: write M115258917723780808):
#0 operator delete(void*) <null> (libtsan.so.0+0x00000006fa94)
171115 09:35:10 t=1510734910.299846 f=loginurl l=NOTE tid=00007fae866fd700 s=fusexrdlogin:85 rc=00 uid=84760 gid=02763 pid=11044 ino=0000000000000000 fh=00000000 name= uid=84760 gid=2763 rc=0 user-name=AAAAAAAI
#1 __gnu_cxx::new_allocator<char>::deallocate(char*, unsigned long) /usr/include/c++/7/ext/new_allocator.h:125 (eosxd+0x000000644a55)
#2 std::allocator_traits<std::allocator<char> >::deallocate(std::allocator<char>&, char*, unsigned long) /usr/include/c++/7/bits/alloc_traits.h:462 (eosxd+0x000000644a55)
#3 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy(unsigned long) /usr/include/c++/7/bits/basic_string.h:225 (eosxd+0x000000644a55)
#4 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose() /usr/include/c++/7/bits/basic_string.h:220 (eosxd+0x000000644a55)
#5 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() /usr/include/c++/7/bits/basic_string.h:646 (eosxd+0x000000644a55)
#6 XrdCl::Proxy::~Proxy() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/xrdclproxy.hh:255 (eosxd+0x000000644a55)
#7 XrdCl::Proxy::~Proxy() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./data/xrdclproxy.hh:257 (eosxd+0x00000063ceda)
#8 data::dmap::ioflush(ThreadAssistant&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/data.cc:1284 (eosxd+0x00000063ceda)
171115 09:35:10 t=1510734910.299981 f=forget l=NOTE tid=00007fae7c0ff700 s=eosfuse:3240 t(ms)=0.075 rc=11 uid=00000 gid=00000 pid=00000 ino=0000000000000c26 fh=00000000 name= nlookup=1
#9 void std::__invoke_impl<void, void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&>(std::__invoke_memfun_deref, void (data::dmap::*&&)(ThreadAssistant&), data::dmap*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x00000063feb6)
#10 std::__invoke_result<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&>::type std::__invoke<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&>(void (data::dmap::*&&)(ThreadAssistant&), data::dmap*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x00000063feb6)
#11 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/thread:234 (eosxd+0x00000063feb6)
#12 std::thread::_Invoker<std::tuple<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x00000063feb6)
#13 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x00000063feb6)
#14 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Previous read of size 8 at 0x7b380018d780 by thread T39 (mutexes: write M71911651052264600, write M113007083552541384, write M1243):
#0 memcpy <null> (libtsan.so.0+0x000000030101)
#1 std::char_traits<char>::copy(char*, char const*, unsigned long) /usr/include/c++/7/bits/char_traits.h:350 (eosxd+0x0000006327b6)
#2 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy(char*, char const*, unsigned long) /usr/include/c++/7/bits/basic_string.h:339 (eosxd+0x0000006327b6)
#3 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy_chars(char*, char const*, char const*) /usr/include/c++/7/bits/basic_string.h:386 (eosxd+0x0000006327b6)
#4 void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag) /usr/include/c++/7/bits/basic_string.tcc:225 (eosxd+0x0000006327b6)
#5 void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct_aux<char*>(char*, char*, std::__false_type) /usr/include/c++/7/bits/basic_string.h:235 (eosxd+0x000000632e63)
#6 void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*) /usr/include/c++/7/bits/basic_string.h:254 (eosxd+0x000000632e63)
#7 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/include/c++/7/bits/basic_string.h:439 (eosxd+0x000000632e63)
#8 XrdCl::Proxy::url[abi:cxx11]() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./data/xrdclproxy.hh:589 (eosxd+0x000000632e63)
#9 data::datax::WaitOpen() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/data.cc:498 (eosxd+0x000000632e63)
171115 09:35:10 t=1510734910.300394 f=xrd_login l=NOTE tid=00007fae764fd700 s=fusexrdlogin:108 uid=84760 gid=2763 xrd-login=AAAAAAAI
#10 data::unlink(fuse_req*, unsigned long) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/data.cc:151 (eosxd+0x00000063f7b2)
#11 EosFuse::unlink(fuse_req*, unsigned long, char const*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:2267 (eosxd+0x000000599fc0)
#12 <null> <null> (libfuse.so.2+0x000000016042)
Mutex M115258917723780808 is already destroyed.
Mutex M71911651052264600 is already destroyed.
Mutex M113007083552541384 is already destroyed.
Mutex M1243 (0x000000b1d680) created at:
#0 pthread_mutex_init <null> (libtsan.so.0+0x00000002971e)
#1 XrdSysMutex::XrdSysMutex() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:153 (eosxd+0x000000632870)
#2 data::dmap::dmap() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./data/data.hh:265 (eosxd+0x000000632870)
#3 data::data() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/data.cc:38 (eosxd+0x000000632870)
#4 EosFuse::EosFuse() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:84 (eosxd+0x0000005afb3a)
#5 EosFuse::instance() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.hh:56 (eosxd+0x00000056beb7)
#6 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:29 (eosxd+0x0000004dab99)
Thread T23 (tid=10293, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0x0000000bc304)
171115 09:35:10 t=1510734910.300638 f=xrd_login l=NOTE tid=00007fae764fd700 s=fusexrdlogin:108 uid=84760 gid=2763 xrd-login=AAAAAAAI
#2 EosFuse::run(int, char**, void*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:874 (eosxd+0x000000592bf3)
#3 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:30 (eosxd+0x0000004daba8)
Thread T39 (tid=10416, running) created by thread T33 at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
171115 09:35:10 t=1510734910.300908 f=xrd_login l=NOTE tid=00007fae764fd700 s=fusexrdlogin:108 uid=84760 gid=2763 xrd-login=AAAAAAAI
#1 <null> <null> (libfuse.so.2+0x000000012984)
SUMMARY: ThreadSanitizer: data race (/lib64/libtsan.so.0+0x6fa94) in operator delete(void*)
apeters1971
pushed a commit
that referenced
this pull request
Feb 6, 2018
Well, well, well... look what I found... https://github.com/linux-test-project/ltp/tree/master/testcases/kernel/fs/racer WARNING: ThreadSanitizer: data race (pid=3472) Write of size 4 at 0x7b2c000270a4 by thread T23 (mutexes: read M2586, write M2569): #0 eos::fusex::cap::set_errc(unsigned int) /home/gbitzes/eos-fusex-tsan/fusex/fusex.pb.h:4387 (eosxd+0x000000632e8f) #1 cap::acquire(fuse_req*, unsigned long, unsigned int, bool) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:359 (eosxd+0x000000632e8f) #2 EosFuse::opendir(fuse_req*, unsigned long, fuse_file_info*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:1768 (eosxd+0x000000598bd5) #3 <null> <null> (libfuse.so.2+0x000000016c32) Previous read of size 4 at 0x7b2c000270a4 by thread T16 (mutexes: read M2586): #0 EosFuse::opendir(fuse_req*, unsigned long, fuse_file_info*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:1768 (eosxd+0x000000598bf8) #1 <null> <null> (libfuse.so.2+0x000000016c32) Location is heap block of size 176 at 0x7b2c00027040 allocated by thread T15: #0 operator new(unsigned long) <null> (libtsan.so.0+0x00000006f766) #1 __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<cap::capx, std::allocator<cap::capx>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /usr/include/c++/7/ext/new_allocator.h:111 (eosxd+0x00000062e649) #2 std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<cap::capx, std::allocator<cap::capx>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<cap::capx, std::allocator<cap::capx>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /usr/include/c++/7/bits/alloc_traits.h:436 (eosxd+0x00000062e649) #3 std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<cap::capx, std::allocator<cap::capx>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<cap::capx, std::allocator<cap::capx>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<cap::capx, std::allocator<cap::capx>, (__gnu_cxx::_Lock_policy)2> >&) /usr/include/c++/7/bits/allocated_ptr.h:104 (eosxd+0x00000062e649) #4 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<cap::capx, std::allocator<cap::capx>>(std::_Sp_make_shared_tag, cap::capx*, std::allocator<cap::capx> const&) /usr/include/c++/7/bits/shared_ptr_base.h:635 (eosxd+0x00000062e649) #5 std::__shared_ptr<cap::capx, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<cap::capx>>(std::_Sp_make_shared_tag, std::allocator<cap::capx> const&) /usr/include/c++/7/bits/shared_ptr_base.h:1295 (eosxd+0x00000062e649) #6 std::shared_ptr<cap::capx>::shared_ptr<std::allocator<cap::capx>>(std::_Sp_make_shared_tag, std::allocator<cap::capx> const&) /usr/include/c++/7/bits/shared_ptr.h:344 (eosxd+0x00000062e649) #7 std::shared_ptr<cap::capx> std::allocate_shared<cap::capx, std::allocator<cap::capx>>(std::allocator<cap::capx> const&) /usr/include/c++/7/bits/shared_ptr.h:691 (eosxd+0x00000062e649) #8 std::shared_ptr<cap::capx> std::make_shared<cap::capx>() /usr/include/c++/7/bits/shared_ptr.h:707 (eosxd+0x00000062e649) #9 cap::imply(std::shared_ptr<cap::capx>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned int, unsigned long) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:309 (eosxd+0x00000062e649) #10 EosFuse::mkdir(fuse_req*, unsigned long, char const*, unsigned int) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:2208 (eosxd+0x00000059a963) #11 <null> <null> (libfuse.so.2+0x000000016042) Mutex M2586 (0x7b4400080218) created at: #0 pthread_rwlock_init <null> (libtsan.so.0+0x00000002a140) #1 eos::common::RWMutex::RWMutex() /afs/cern.ch/user/g/gbitzes/dev/eos-master/common/RWMutex.cc:154 (libeosCommon.so.4+0x00000003f455) #2 EosFuse::create(fuse_req*, unsigned long, char const*, unsigned int, fuse_file_info*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:2910 (eosxd+0x0000005a0739) #3 <null> <null> (libfuse.so.2+0x0000000149f6) Mutex M2569 (0x7b2c000270c0) created at: #0 pthread_mutex_init <null> (libtsan.so.0+0x00000002971e) #1 XrdSysMutex::XrdSysMutex() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:153 (eosxd+0x00000062e6bf) #2 void __gnu_cxx::new_allocator<cap::capx>::construct<cap::capx>(cap::capx*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./cap/cap.hh:104 (eosxd+0x00000062e6bf) #3 void std::allocator_traits<std::allocator<cap::capx> >::construct<cap::capx>(std::allocator<cap::capx>&, cap::capx*) /usr/include/c++/7/bits/alloc_traits.h:475 (eosxd+0x00000062e6bf) #4 std::_Sp_counted_ptr_inplace<cap::capx, std::allocator<cap::capx>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<>(std::allocator<cap::capx>) /usr/include/c++/7/bits/shared_ptr_base.h:526 (eosxd+0x00000062e6bf) #5 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<cap::capx, std::allocator<cap::capx>>(std::_Sp_make_shared_tag, cap::capx*, std::allocator<cap::capx> const&) /usr/include/c++/7/bits/shared_ptr_base.h:637 (eosxd+0x00000062e6bf) #6 std::__shared_ptr<cap::capx, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<cap::capx>>(std::_Sp_make_shared_tag, std::allocator<cap::capx> const&) /usr/include/c++/7/bits/shared_ptr_base.h:1295 (eosxd+0x00000062e6bf) #7 std::shared_ptr<cap::capx>::shared_ptr<std::allocator<cap::capx>>(std::_Sp_make_shared_tag, std::allocator<cap::capx> const&) /usr/include/c++/7/bits/shared_ptr.h:344 (eosxd+0x00000062e6bf) #8 std::shared_ptr<cap::capx> std::allocate_shared<cap::capx, std::allocator<cap::capx>>(std::allocator<cap::capx> const&) /usr/include/c++/7/bits/shared_ptr.h:691 (eosxd+0x00000062e6bf) #9 std::shared_ptr<cap::capx> std::make_shared<cap::capx>() /usr/include/c++/7/bits/shared_ptr.h:707 (eosxd+0x00000062e6bf) #10 cap::imply(std::shared_ptr<cap::capx>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned int, unsigned long) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/cap/cap.cc:309 (eosxd+0x00000062e6bf) #11 EosFuse::mkdir(fuse_req*, unsigned long, char const*, unsigned int) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:2208 (eosxd+0x00000059a963) #12 <null> <null> (libfuse.so.2+0x000000016042) Thread T23 (tid=3578, running) created by thread T16 at: #0 pthread_create <null> (libtsan.so.0+0x000000028e03) #1 <null> <null> (libfuse.so.2+0x000000012984) Thread T16 (tid=3499, running) created by thread T15 at: #0 pthread_create <null> (libtsan.so.0+0x000000028e03) #1 <null> <null> (libfuse.so.2+0x000000012984) Thread T15 (tid=3498, running) created by main thread at: #0 pthread_create <null> (libtsan.so.0+0x000000028e03) #1 <null> <null> (libfuse.so.2+0x000000012984) #2 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:30 (eosxd+0x0000004db118) SUMMARY: ThreadSanitizer: data race /home/gbitzes/eos-fusex-tsan/fusex/fusex.pb.h:4387 in eos::fusex::cap::set_errc(unsigned int)
apeters1971
pushed a commit
that referenced
this pull request
Feb 6, 2018
Fixes the following, among many others. (Observed when trying to
compile quarkdb on /eos)
WARNING: ThreadSanitizer: data race (pid=10251)
Write of size 8 at 0x7b380018d780 by thread T23 (mutexes: write M115258917723780808):
#0 operator delete(void*) <null> (libtsan.so.0+0x00000006fa94)
171115 09:35:10 t=1510734910.299846 f=loginurl l=NOTE tid=00007fae866fd700 s=fusexrdlogin:85 rc=00 uid=84760 gid=02763 pid=11044 ino=0000000000000000 fh=00000000 name= uid=84760 gid=2763 rc=0 user-name=AAAAAAAI
#1 __gnu_cxx::new_allocator<char>::deallocate(char*, unsigned long) /usr/include/c++/7/ext/new_allocator.h:125 (eosxd+0x000000644a55)
#2 std::allocator_traits<std::allocator<char> >::deallocate(std::allocator<char>&, char*, unsigned long) /usr/include/c++/7/bits/alloc_traits.h:462 (eosxd+0x000000644a55)
#3 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy(unsigned long) /usr/include/c++/7/bits/basic_string.h:225 (eosxd+0x000000644a55)
#4 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose() /usr/include/c++/7/bits/basic_string.h:220 (eosxd+0x000000644a55)
#5 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() /usr/include/c++/7/bits/basic_string.h:646 (eosxd+0x000000644a55)
#6 XrdCl::Proxy::~Proxy() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/xrdclproxy.hh:255 (eosxd+0x000000644a55)
#7 XrdCl::Proxy::~Proxy() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./data/xrdclproxy.hh:257 (eosxd+0x00000063ceda)
#8 data::dmap::ioflush(ThreadAssistant&) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/data.cc:1284 (eosxd+0x00000063ceda)
171115 09:35:10 t=1510734910.299981 f=forget l=NOTE tid=00007fae7c0ff700 s=eosfuse:3240 t(ms)=0.075 rc=11 uid=00000 gid=00000 pid=00000 ino=0000000000000c26 fh=00000000 name= nlookup=1
#9 void std::__invoke_impl<void, void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&>(std::__invoke_memfun_deref, void (data::dmap::*&&)(ThreadAssistant&), data::dmap*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:73 (eosxd+0x00000063feb6)
#10 std::__invoke_result<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&>::type std::__invoke<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&>(void (data::dmap::*&&)(ThreadAssistant&), data::dmap*&&, ThreadAssistant&) /usr/include/c++/7/bits/invoke.h:95 (eosxd+0x00000063feb6)
#11 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/thread:234 (eosxd+0x00000063feb6)
#12 std::thread::_Invoker<std::tuple<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&> >::operator()() /usr/include/c++/7/thread:243 (eosxd+0x00000063feb6)
#13 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (data::dmap::*)(ThreadAssistant&), data::dmap*, ThreadAssistant&> > >::_M_run() /usr/include/c++/7/thread:186 (eosxd+0x00000063feb6)
#14 <null> <null> (libstdc++.so.6+0x0000000bc01e)
Previous read of size 8 at 0x7b380018d780 by thread T39 (mutexes: write M71911651052264600, write M113007083552541384, write M1243):
#0 memcpy <null> (libtsan.so.0+0x000000030101)
#1 std::char_traits<char>::copy(char*, char const*, unsigned long) /usr/include/c++/7/bits/char_traits.h:350 (eosxd+0x0000006327b6)
#2 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy(char*, char const*, unsigned long) /usr/include/c++/7/bits/basic_string.h:339 (eosxd+0x0000006327b6)
#3 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy_chars(char*, char const*, char const*) /usr/include/c++/7/bits/basic_string.h:386 (eosxd+0x0000006327b6)
#4 void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag) /usr/include/c++/7/bits/basic_string.tcc:225 (eosxd+0x0000006327b6)
#5 void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct_aux<char*>(char*, char*, std::__false_type) /usr/include/c++/7/bits/basic_string.h:235 (eosxd+0x000000632e63)
#6 void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*) /usr/include/c++/7/bits/basic_string.h:254 (eosxd+0x000000632e63)
#7 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/include/c++/7/bits/basic_string.h:439 (eosxd+0x000000632e63)
#8 XrdCl::Proxy::url[abi:cxx11]() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./data/xrdclproxy.hh:589 (eosxd+0x000000632e63)
#9 data::datax::WaitOpen() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/data.cc:498 (eosxd+0x000000632e63)
171115 09:35:10 t=1510734910.300394 f=xrd_login l=NOTE tid=00007fae764fd700 s=fusexrdlogin:108 uid=84760 gid=2763 xrd-login=AAAAAAAI
#10 data::unlink(fuse_req*, unsigned long) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/data.cc:151 (eosxd+0x00000063f7b2)
#11 EosFuse::unlink(fuse_req*, unsigned long, char const*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:2267 (eosxd+0x000000599fc0)
#12 <null> <null> (libfuse.so.2+0x000000016042)
Mutex M115258917723780808 is already destroyed.
Mutex M71911651052264600 is already destroyed.
Mutex M113007083552541384 is already destroyed.
Mutex M1243 (0x000000b1d680) created at:
#0 pthread_mutex_init <null> (libtsan.so.0+0x00000002971e)
#1 XrdSysMutex::XrdSysMutex() /home/gbitzes/xrootd/build-tsan/install/include/xrootd/XrdSys/XrdSysPthread.hh:153 (eosxd+0x000000632870)
#2 data::dmap::dmap() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/./data/data.hh:265 (eosxd+0x000000632870)
#3 data::data() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/data/data.cc:38 (eosxd+0x000000632870)
#4 EosFuse::EosFuse() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:84 (eosxd+0x0000005afb3a)
#5 EosFuse::instance() /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.hh:56 (eosxd+0x00000056beb7)
#6 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:29 (eosxd+0x0000004dab99)
Thread T23 (tid=10293, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0x0000000bc304)
171115 09:35:10 t=1510734910.300638 f=xrd_login l=NOTE tid=00007fae764fd700 s=fusexrdlogin:108 uid=84760 gid=2763 xrd-login=AAAAAAAI
#2 EosFuse::run(int, char**, void*) /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/eosfuse.cc:874 (eosxd+0x000000592bf3)
#3 main /afs/cern.ch/user/g/gbitzes/dev/eos/fusex/main.cc:30 (eosxd+0x0000004daba8)
Thread T39 (tid=10416, running) created by thread T33 at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
171115 09:35:10 t=1510734910.300908 f=xrd_login l=NOTE tid=00007fae764fd700 s=fusexrdlogin:108 uid=84760 gid=2763 xrd-login=AAAAAAAI
#1 <null> <null> (libfuse.so.2+0x000000012984)
SUMMARY: ThreadSanitizer: data race (/lib64/libtsan.so.0+0x6fa94) in operator delete(void*)
|
Sorry, I overlooked that. It has been corrected since then. |
apeters1971
pushed a commit
that referenced
this pull request
Jul 4, 2018
WARNING: ThreadSanitizer: data race (pid=11483)
Write of size 8 at 0x7b4c000036b0 by thread T105 (mutexes: read M2044, read M2202):
#0 eos::common::FileSystem::SetLongLong(char const*, long long, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/common/FileSystem.hh:390 (libXrdEosMgm.so+0x00000027d722)
#1 eos::mgm::Messaging::Update(XrdAdvisoryMqMessage*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:163 (libXrdEosMgm.so+0x00000027d722)
#2 eos::mgm::Messaging::Process(XrdMqMessage*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:220 (libXrdEosMgm.so+0x00000027bfab)
#3 eos::mgm::Messaging::Listen() /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:187 (libXrdEosMgm.so+0x00000027ab4e)
#4 XrdMqMessaging::Start(void*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mq/XrdMqMessaging.cc:35 (libXrdMqClient.so.4+0x0000000272ac)
#5 XrdSysThread_Xeq <null> (libXrdUtils.so.2+0x00000002d75e)
#6 <null> <null> (libtsan.so.0+0x0000000257eb)
Previous write of size 8 at 0x7b4c000036b0 by main thread (mutexes: read M2044, read M2202):
#0 eos::common::FileSystem::SetString(char const*, char const*, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/common/FileSystem.hh:358 (libXrdEosMgm.so+0x0000005421e3)
#1 eos::mgm::FileSystem::SetConfigStatus(int) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FileSystem.cc:166 (libXrdEosMgm.so+0x0000005421e3)
#2 eos::mgm::FsView::ReapplyConfigStatus() /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FsView.cc:4153 (libXrdEosMgm.so+0x00000041b7bb)
#3 eos::mgm::Master::Activate(XrdOucString&, XrdOucString&, int) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Master.cc:1145 (libXrdEosMgm.so+0x0000006940d8)
#4 eos::mgm::Master::ApplyMasterConfig(XrdOucString&, XrdOucString&, eos::mgm::Master::Transition::Type) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Master.cc:1056 (libXrdEosMgm.so+0x00000069483b)
#5 XrdMgmOfs::Configure(XrdSysError&) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/XrdMgmOfsConfigure.cc:1899 (libXrdEosMgm.so+0x000000455222)
#6 XrdSfsGetFileSystem /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/XrdMgmOfs.cc:137 (libXrdEosMgm.so+0x0000004cc72f)
#7 XrdXrootdloadFileSystem(XrdSysError*, XrdSfsFileSystem*, char*, int, char const*, XrdOucEnv*) <null> (libXrdServer.so.2+0x00000004be05)
#8 XrdXrootdProtocol::Configure(char*, XrdProtocol_Config*) <null> (libXrdServer.so.2+0x000000045671)
#9 XrdgetProtocol <null> (libXrdServer.so.2+0x000000055233)
#10 XrdProtLoad::getProtocol(char const*, char const*, char*, XrdProtocol_Config*) <null> (xrootd+0x00000040de08)
#11 XrdProtLoad::Load(char const*, char const*, char*, XrdProtocol_Config*) <null> (xrootd+0x00000040def2)
#12 XrdConfig::Setup(char*) <null> (xrootd+0x0000004079ea)
#13 XrdConfig::Configure(int, char**) <null> (xrootd+0x00000040c4be)
#14 main <null> (xrootd+0x000000406465)
apeters1971
pushed a commit
that referenced
this pull request
Jul 9, 2018
WARNING: ThreadSanitizer: data race (pid=11483)
Write of size 8 at 0x7b4c000036b0 by thread T105 (mutexes: read M2044, read M2202):
#0 eos::common::FileSystem::SetLongLong(char const*, long long, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/common/FileSystem.hh:390 (libXrdEosMgm.so+0x00000027d722)
#1 eos::mgm::Messaging::Update(XrdAdvisoryMqMessage*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:163 (libXrdEosMgm.so+0x00000027d722)
#2 eos::mgm::Messaging::Process(XrdMqMessage*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:220 (libXrdEosMgm.so+0x00000027bfab)
#3 eos::mgm::Messaging::Listen() /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:187 (libXrdEosMgm.so+0x00000027ab4e)
#4 XrdMqMessaging::Start(void*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mq/XrdMqMessaging.cc:35 (libXrdMqClient.so.4+0x0000000272ac)
#5 XrdSysThread_Xeq <null> (libXrdUtils.so.2+0x00000002d75e)
#6 <null> <null> (libtsan.so.0+0x0000000257eb)
Previous write of size 8 at 0x7b4c000036b0 by main thread (mutexes: read M2044, read M2202):
#0 eos::common::FileSystem::SetString(char const*, char const*, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/common/FileSystem.hh:358 (libXrdEosMgm.so+0x0000005421e3)
#1 eos::mgm::FileSystem::SetConfigStatus(int) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FileSystem.cc:166 (libXrdEosMgm.so+0x0000005421e3)
#2 eos::mgm::FsView::ReapplyConfigStatus() /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FsView.cc:4153 (libXrdEosMgm.so+0x00000041b7bb)
#3 eos::mgm::Master::Activate(XrdOucString&, XrdOucString&, int) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Master.cc:1145 (libXrdEosMgm.so+0x0000006940d8)
#4 eos::mgm::Master::ApplyMasterConfig(XrdOucString&, XrdOucString&, eos::mgm::Master::Transition::Type) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Master.cc:1056 (libXrdEosMgm.so+0x00000069483b)
#5 XrdMgmOfs::Configure(XrdSysError&) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/XrdMgmOfsConfigure.cc:1899 (libXrdEosMgm.so+0x000000455222)
#6 XrdSfsGetFileSystem /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/XrdMgmOfs.cc:137 (libXrdEosMgm.so+0x0000004cc72f)
#7 XrdXrootdloadFileSystem(XrdSysError*, XrdSfsFileSystem*, char*, int, char const*, XrdOucEnv*) <null> (libXrdServer.so.2+0x00000004be05)
#8 XrdXrootdProtocol::Configure(char*, XrdProtocol_Config*) <null> (libXrdServer.so.2+0x000000045671)
#9 XrdgetProtocol <null> (libXrdServer.so.2+0x000000055233)
#10 XrdProtLoad::getProtocol(char const*, char const*, char*, XrdProtocol_Config*) <null> (xrootd+0x00000040de08)
#11 XrdProtLoad::Load(char const*, char const*, char*, XrdProtocol_Config*) <null> (xrootd+0x00000040def2)
#12 XrdConfig::Setup(char*) <null> (xrootd+0x0000004079ea)
#13 XrdConfig::Configure(int, char**) <null> (xrootd+0x00000040c4be)
#14 main <null> (xrootd+0x000000406465)
apeters1971
pushed a commit
that referenced
this pull request
Jul 26, 2018
WARNING: ThreadSanitizer: data race (pid=27407)
Write of size 1 at 0x7b44001717c0 by thread T105 (mutexes: read M2220, read M2444, write M1123783398499600256):
#0 XrdMqSharedHash::CloseTransaction() /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mq/XrdMqSharedObject.cc:429 (libXrdMqClient.so.4+0x00000002d19f)
#1 XrdMqSharedHash::SetImpl(char const*, char const*, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mq/XrdMqSharedObject.cc:758 (libXrdMqClient.so.4+0x000000038015)
#2 bool XrdMqSharedHash::Set<char const*>(char const*, char const*&&, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mq/XrdMqSharedObject.hh:1115 (libXrdEosMgm.so+0x0000001d28a8)
#3 eos::mgm::BaseView::SetConfigMember(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FsView.cc:2587 (libXrdEosMgm.so+0x00000042618c)
#4 eos::mgm::FsNode::SetActiveStatus(int) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FsView.cc:2551 (libXrdEosMgm.so+0x00000041a5ec)
#5 eos::mgm::Messaging::Update(XrdAdvisoryMqMessage*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:141 (libXrdEosMgm.so+0x00000027d4f8)
#6 eos::mgm::Messaging::Process(XrdMqMessage*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:219 (libXrdEosMgm.so+0x00000027c51b)
#7 eos::mgm::Messaging::Listen() /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/Messaging.cc:187 (libXrdEosMgm.so+0x00000027b07c)
#8 XrdMqMessaging::Start(void*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mq/XrdMqMessaging.cc:35 (libXrdMqClient.so.4+0x0000000272ac)
#9 XrdSysThread_Xeq <null> (libXrdUtils.so.2+0x00000002d75e)
#10 <null> <null> (libtsan.so.0+0x0000000257eb)
Previous read of size 1 at 0x7b44001717c0 by thread T40 (mutexes: read M2220, read M2444):
#0 XrdMqSharedHash::SetImpl(char const*, char const*, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mq/XrdMqSharedObject.cc:749 (libXrdMqClient.so.4+0x000000037e77)
#1 bool XrdMqSharedHash::Set<char const*>(char const*, char const*&&, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mq/XrdMqSharedObject.hh:1115 (libXrdEosMgm.so+0x0000001d28a8)
#2 eos::mgm::BaseView::SetConfigMember(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FsView.cc:2587 (libXrdEosMgm.so+0x00000042618c)
#3 eos::mgm::FsNode::SetActiveStatus(int) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FsView.cc:2553 (libXrdEosMgm.so+0x00000041a539)
#4 eos::mgm::FsView::HeartBeatCheck() /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FsView.cc:2292 (libXrdEosMgm.so+0x0000004280b9)
#5 eos::mgm::FsView::StaticHeartBeatCheck(void*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/mgm/FsView.cc:2223 (libXrdEosMgm.so+0x000000428f25)
#6 XrdSysThread_Xeq <null> (libXrdUtils.so.2+0x00000002d75e)
#7 <null> <null> (libtsan.so.0+0x0000000257eb)
apeters1971
pushed a commit
that referenced
this pull request
Sep 5, 2018
WARNING: ThreadSanitizer: unlock of an unlocked mutex (or by a wrong thread) (pid=18781)
#0 pthread_mutex_unlock <null> (libtsan.so.0+0x00000003b79b)
#1 XrdSysMutex::UnLock() /usr/include/xrootd/XrdSys/XrdSysPthread.hh:151 (eosxd+0x000000607851)
#2 XrdSysMutexHelper::UnLock() /usr/include/xrootd/XrdSys/XrdSysPthread.hh:201 (eosxd+0x000000607851)
#3 XrdSysMutexHelper::~XrdSysMutexHelper() /usr/include/xrootd/XrdSys/XrdSysPthread.hh:212 (eosxd+0x000000607851)
#4 EosFuse::link(fuse_req*, unsigned long, unsigned long, char const*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/eosfuse.cc:4612 (eosxd+0x000000607851)
#5 <null> <null> (libfuse.so.2+0x0000000167e2)
Location is heap block of size 768 at 0x7b58000e9700 allocated by thread T27:
#0 operator new(unsigned long) <null> (libtsan.so.0+0x00000006f766)
#1 __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<metad::mdx, std::allocator<metad::mdx>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /usr/include/c++/7/ext/new_allocator.h:111 (eosxd+0x000000681ab4)
#2 std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<metad::mdx, std::allocator<metad::mdx>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<metad::mdx, std::allocator<metad::mdx>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /usr/include/c++/7/bits/alloc_traits.h:436 (eosxd+0x000000681ab4)
#3 std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<metad::mdx, std::allocator<metad::mdx>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<metad::mdx, std::allocator<metad::mdx>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<metad::mdx, std::allocator<metad::mdx>, (__gnu_cxx::_Lock_policy)2> >&) /usr/include/c++/7/bits/allocated_ptr.h:104 (eosxd+0x000000681ab4)
#4 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<metad::mdx, std::allocator<metad::mdx>>(std::_Sp_make_shared_tag, metad::mdx*, std::allocator<metad::mdx> const&) /usr/include/c++/7/bits/shared_ptr_base.h:635 (eosxd+0x000000681ab4)
#5 std::__shared_ptr<metad::mdx, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<metad::mdx>>(std::_Sp_make_shared_tag, std::allocator<metad::mdx> const&) /usr/include/c++/7/bits/shared_ptr_base.h:1295 (eosxd+0x000000681ab4)
#6 std::shared_ptr<metad::mdx>::shared_ptr<std::allocator<metad::mdx>>(std::_Sp_make_shared_tag, std::allocator<metad::mdx> const&) /usr/include/c++/7/bits/shared_ptr.h:344 (eosxd+0x000000678905)
#7 std::shared_ptr<metad::mdx> std::allocate_shared<metad::mdx, std::allocator<metad::mdx>>(std::allocator<metad::mdx> const&) /usr/include/c++/7/bits/shared_ptr.h:691 (eosxd+0x000000678905)
#8 std::shared_ptr<metad::mdx> std::make_shared<metad::mdx>() /usr/include/c++/7/bits/shared_ptr.h:707 (eosxd+0x000000678905)
#9 metad::lookup(fuse_req*, unsigned long, char const*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/md/md.cc:197 (eosxd+0x000000678905)
#10 EosFuse::link(fuse_req*, unsigned long, unsigned long, char const*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/eosfuse.cc:4610 (eosxd+0x0000006072f8)
#11 <null> <null> (libfuse.so.2+0x0000000167e2)
Mutex M330868715523315928 is already destroyed.
Thread T27 (tid=19301, running) created by thread T23 at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 <null> <null> (libfuse.so.2+0x000000013134)
SUMMARY: ThreadSanitizer: unlock of an unlocked mutex (or by a wrong thread) (/usr/lib64/libtsan.so.0+0x3b79b) in pthread_mutex_unlock
apeters1971
pushed a commit
that referenced
this pull request
Sep 5, 2018
WARNING: ThreadSanitizer: data race (pid=27564)
Write of size 8 at 0x7b6c000464d8 by thread T3 (mutexes: write M69597):
#0 std::_Rb_tree<unsigned long, std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> >, std::_Select1st<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > >, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > > >::_M_erase_aux(std::_Rb_tree_const_iterator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > >) /usr/include/c++/7/bits/stl_tree.h:2478 (eosxd+0x0000006f35a6)
#1 std::_Rb_tree<unsigned long, std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> >, std::_Select1st<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > >, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > > >::_M_erase_aux(std::_Rb_tree_const_iterator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > >, std::_Rb_tree_const_iterator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > >) /usr/include/c++/7/bits/stl_tree.h:2491 (eosxd+0x0000006f35a6)
#2 std::_Rb_tree<unsigned long, std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> >, std::_Select1st<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > >, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > > >::erase(unsigned long const&) /usr/include/c++/7/bits/stl_tree.h:2502 (eosxd+0x0000006f35a6)
#3 std::map<unsigned long, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler>, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > > >::erase(unsigned long const&) /usr/include/c++/7/bits/stl_map.h:1063 (eosxd+0x0000006ef7cf)
#4 XrdCl::Proxy::WriteAsyncHandler::HandleResponse(XrdCl::XRootDStatus*, XrdCl::AnyObject*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/data/xrdclproxy.cc:799 (eosxd+0x0000006ef7cf)
#5 XrdCl::ResponseHandler::HandleResponseWithHosts(XrdCl::XRootDStatus*, XrdCl::AnyObject*, std::vector<XrdCl::HostInfo, std::allocator<XrdCl::HostInfo> >*) /usr/include/xrootd/XrdCl/XrdClXRootDResponses.hh:852 (eosxd+0x0000006e3f05)
#6 <null> <null> (libXrdCl.so.2+0x0000000d03f7)
#7 XrdCl::XRootDMsgHandler::HandleResponse() <null> (libXrdCl.so.2+0x000000098b19)
#8 XrdCl::XRootDMsgHandler::Process(XrdCl::Message*) <null> (libXrdCl.so.2+0x00000009d3a1)
#9 XrdCl::Stream::HandleIncMsgJob::Run(void*) <null> (libXrdCl.so.2+0x00000006d5ff)
#10 XrdCl::JobManager::RunJobs() <null> (libXrdCl.so.2+0x00000011f67f)
#11 <null> <null> (libXrdCl.so.2+0x00000011f775)
#12 <null> <null> (libtsan.so.0+0x0000000257eb)
Previous read of size 8 at 0x7b6c000464d8 by thread T4:
#0 std::map<unsigned long, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler>, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > > >::size() const /usr/include/c++/7/bits/stl_map.h:463 (eosxd+0x0000006ef4bb)
#1 XrdCl::Proxy::WriteAsyncHandler::HandleResponse(XrdCl::XRootDStatus*, XrdCl::AnyObject*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/data/xrdclproxy.cc:782 (eosxd+0x0000006ef4bb)
#2 XrdCl::ResponseHandler::HandleResponseWithHosts(XrdCl::XRootDStatus*, XrdCl::AnyObject*, std::vector<XrdCl::HostInfo, std::allocator<XrdCl::HostInfo> >*) /usr/include/xrootd/XrdCl/XrdClXRootDResponses.hh:852 (eosxd+0x0000006e3f05)
#3 <null> <null> (libXrdCl.so.2+0x0000000d03f7)
#4 XrdCl::XRootDMsgHandler::HandleResponse() <null> (libXrdCl.so.2+0x000000098b19)
#5 XrdCl::XRootDMsgHandler::Process(XrdCl::Message*) <null> (libXrdCl.so.2+0x00000009d3a1)
#6 XrdCl::Stream::HandleIncMsgJob::Run(void*) <null> (libXrdCl.so.2+0x00000006d5ff)
#7 XrdCl::JobManager::RunJobs() <null> (libXrdCl.so.2+0x00000011f67f)
#8 <null> <null> (libXrdCl.so.2+0x00000011f775)
#9 <null> <null> (libtsan.so.0+0x0000000257eb)
Location is heap block of size 1712 at 0x7b6c00046000 allocated by thread T27:
#0 operator new(unsigned long) <null> (libtsan.so.0+0x00000006f766)
#1 data::datax::attach(fuse_req*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/data/data.cc:577 (eosxd+0x0000006a656e)
#2 EosFuse::create(fuse_req*, unsigned long, char const*, unsigned int, fuse_file_info*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/eosfuse.cc:3410 (eosxd+0x0000005fa699)
#3 <null> <null> (libfuse.so.2+0x0000000151a6)
Mutex M69597 (0x7b6c00046410) created at:
#0 pthread_mutex_init <null> (libtsan.so.0+0x00000002971e)
#1 XrdSysCondVar::XrdSysCondVar(int, char const*) /usr/include/xrootd/XrdSys/XrdSysPthread.hh:78 (eosxd+0x0000006b12b6)
#2 XrdCl::Proxy::Proxy() /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/./data/xrdclproxy.hh:546 (eosxd+0x0000006b12b6)
#3 data::datax::attach(fuse_req*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/data/data.cc:577 (eosxd+0x0000006a657d)
#4 EosFuse::create(fuse_req*, unsigned long, char const*, unsigned int, fuse_file_info*) /afs/cern.ch/user/g/gbitzes/dev/eos-dev/fusex/eosfuse.cc:3410 (eosxd+0x0000005fa699)
#5 <null> <null> (libfuse.so.2+0x0000000151a6)
Thread T3 (tid=27572, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 XrdCl::JobManager::Start() <null> (libXrdCl.so.2+0x00000011f45a)
#2 XrdCl::PostMaster::Start() <null> (libXrdCl.so.2+0x000000061e31)
#3 XrdCl::DefaultEnv::GetPostMaster() <null> (libXrdCl.so.2+0x000000044354)
#4 <null> <null> (libXrdCl.so.2+0x000000032365)
#5 call_init.part.0 <null> (ld-linux-x86-64.so.2+0x00000000f529)
Thread T4 (tid=27573, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 XrdCl::JobManager::Start() <null> (libXrdCl.so.2+0x00000011f45a)
#2 XrdCl::PostMaster::Start() <null> (libXrdCl.so.2+0x000000061e31)
#3 XrdCl::DefaultEnv::GetPostMaster() <null> (libXrdCl.so.2+0x000000044354)
#4 <null> <null> (libXrdCl.so.2+0x000000032365)
#5 call_init.part.0 <null> (ld-linux-x86-64.so.2+0x00000000f529)
Thread T27 (tid=28416, running) created by thread T24 at:
#0 pthread_create <null> (libtsan.so.0+0x000000028e03)
#1 <null> <null> (libfuse.so.2+0x000000013134)
SUMMARY: ThreadSanitizer: data race /usr/include/c++/7/bits/stl_tree.h:2478 in std::_Rb_tree<unsigned long, std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> >, std::_Select1st<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > >, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > > >::_M_erase_aux(std::_Rb_tree_const_iterator<std::pair<unsigned long const, std::shared_ptr<XrdCl::Proxy::WriteAsyncHandler> > >)
apeters1971
pushed a commit
that referenced
this pull request
Sep 24, 2018
==15952==ERROR: AddressSanitizer: heap-use-after-free on address 0x6060000fa2f8 at pc 0x7f9e634b8185 bp 0x7ffca6d90e80 sp 0x7ffca6d90628
#0 0x7f9e634b8184 (/lib64/libasan.so.3+0x46184)
#1 0x699c14 in testing::internal::String::CStringEquals(char const*, char const*) /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:927
#2 0x699c14 in testing::internal::CmpHelperSTREQ(char const*, char const*, char const*, char const*) /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:1520
#3 0x499e44 in ParseComment_CommentExtraction_Test::TestBody() /builds/dss/eos/unit_tests/console/ParseCommentTest.cc:89
#4 0x6a73a3 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2432
#5 0x6a73a3 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2468
#6 0x694fbd in testing::Test::Run() /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2507
#7 0x695563 in testing::Test::Run() /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2498
#8 0x695563 in testing::TestInfo::Run() /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2682
#9 0x695af4 in testing::TestInfo::Run() /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2657
#10 0x695af4 in testing::TestCase::Run() /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2800
#11 0x6965ec in testing::TestCase::Run() /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:5176
#12 0x6965ec in testing::internal::UnitTestImpl::RunAllTests() /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:5124
#13 0x6a7c73 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2432
#14 0x6a7c73 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:2468
#15 0x696fa5 in testing::UnitTest::Run() /builds/dss/eos/unit_tests/googletest/googletest/src/gtest.cc:4733
#16 0x47221f in RUN_ALL_TESTS() /builds/dss/eos/unit_tests/googletest/googletest/include/gtest/gtest.h:2329
#17 0x47221f in main /builds/dss/eos/unit_tests/googletest/googletest/src/gtest_main.cc:37
#18 0x7f9e5bce3444 in __libc_start_main (/lib64/libc.so.6+0x22444)
#19 0x475279 (/builds/dss/eos/build/unit_tests/eos-unit-tests+0x475279)
0x6060000fa2f8 is located 24 bytes inside of 62-byte region [0x6060000fa2e0,0x6060000fa31e)
freed by thread T0 here:
#0 0x7f9e6353a540 in operator delete(void*) (/lib64/libasan.so.3+0xc8540)
#1 0x50feba in std::string::_Rep::_M_dispose(std::allocator<char> const&) /opt/rh/devtoolset-6/root/usr/include/c++/6.3.1/bits/basic_string.h:2780
#2 0x50feba in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() /opt/rh/devtoolset-6/root/usr/include/c++/6.3.1/bits/basic_string.h:3077
#3 0x50feba in parse_comment(char*, std::string&) /builds/dss/eos/console/ConsoleMain.cc:1214
previously allocated by thread T0 here:
#0 0x7f9e63539ec0 in operator new(unsigned long) (/lib64/libasan.so.3+0xc7ec0)
#1 0x7f9e5c87fab8 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (/lib64/libstdc++.so.6+0xbdab8)
#2 0x1 (<unknown module>)
apeters1971
pushed a commit
that referenced
this pull request
Aug 16, 2022
…eadPoolExecutor object and the local storage of the threads in this pool will lead to a segv due to a bug in the folly library. Therefore, we keep the FmdConverter object around until the FST daemon is shutdown. Referenced folly bug: facebook/folly#1252 The ASAN trace of such a crash: AddressSanitizer:DEADLYSIGNAL ================================================================= ==8066==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000038 (pc 0x7ff5370a3e11 bp 0x7ff521df0bf0 sp 0x7ff521df0bb0 T60) ==8066==The signal is caused by a READ memory access. ==8066==Hint: address points to the zero page. #0 0x7ff5370a3e10 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::get() const /root/rpmbuild/BUILD/eos-folly /folly-2019.11.11.00/folly/ThreadLocal.h:159 #1 0x7ff53709e9d9 in folly::ThreadLocal<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::get() const /root/rpmbuild/BUILD/eos-folly/fo lly-2019.11.11.00/folly/ThreadLocal.h:68 #2 0x7ff53709e9d9 in folly::ThreadLocal<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::operator*() const /root/rpmbuild/BUILD/eos-fo lly/folly-2019.11.11.00/folly/ThreadLocal.h:77 #3 0x7ff53709c0fe in folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::getWrapper() /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/folly/SingletonThreadL ocal.h:148 #4 0x7ff53709c112 in folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::LocalLifetime::~LocalLifetime() /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/fol ly/SingletonThreadLocal.h:120 #5 0x7ff547b80b18 in run ../../../../libstdc++-v3/libsupc++/atexit_thread.cc:64 #6 0x7ff54803aca1 in __nptl_deallocate_tsd /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:155 #7 0x7ff54803aeb2 in start_thread /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:314 #8 0x7ff54733c8dc in __clone (/lib64/libc.so.6+0xfe8dc) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/folly/ThreadLocal.h:159 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::Reque stContext> >, void>::Wrapper, void, void>::get() const Thread T60 created by T40 here: #0 0x7ff548ea7471 in pthread_create (/usr/lib64/libasan.so.5+0x3a471) #1 0x7ff539339c34 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/usr/lib64/libEosCommon.so.4+0x8f2c34) #2 0x7ff53e5b1e50 in void AssistedThread::reset<void (eos::fst::ScanDir::*)(ThreadAssistant&) noexcept, eos::fst::ScanDir*>(void (eos::fst::ScanDir::*&&)(ThreadAssistant&) noexcept, eos::fst::ScanDir*&&) (/lib64/libXrdEosFst-4.so+0x41be50) #3 0x7ff53e587b13 in eos::fst::ScanDir::SetConfig(std::string const&, long long) ../fst/ScanDir.cc:136 #4 0x7ff53e72dba2 in eos::fst::FileSystem::ConfigScanner(eos::fst::Load*, std::string const&, long long) ../fst/storage/FileSystem.cc:165 #5 0x7ff53e710721 in eos::fst::Storage::ProcessFsConfigChange(eos::fst::FileSystem*, std::string const&, std::string const&, std::string const&) ../fst/storage/Communicator.cc:321 #6 0x7ff53e711b67 in eos::fst::Storage::ProcessFsConfigChange(std::string const&, std::string const&) ../fst/storage/Communicator.cc:400 #7 0x7ff53e713e13 in eos::fst::Storage::Communicator(ThreadAssistant&) ../fst/storage/Communicator.cc:513 #8 0x7ff53e76f00f in void std::__invoke_impl<void, void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >(std::__invoke_memfun_deref, void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&, std::reference_wrap per<ThreadAssistant>&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/invoke.h:73 #9 0x7ff53e76a4cd in std::__invoke_result<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >::type std::__invoke<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >(void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&, std::reference_wrapper<ThreadAssistant>&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/invoke.h:95 #10 0x7ff53e7786c4 in decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > >::_M_invoke<0ul, 1ul, 2ul>(std ::_Index_tuple<0ul, 1ul, 2ul>) /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:244 #11 0x7ff53e778663 in std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > >::operator()() /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:253 #12 0x7ff53e778647 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > > >::_M_run() /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:196 #13 0x7ff539339bce in execute_native_thread_routine (/usr/lib64/libEosCommon.so.4+0x8f2bce) Thread T40 created by T0 here: #0 0x7ff548ea7471 in pthread_create (/usr/lib64/libasan.so.5+0x3a471) #1 0x7ff539339c34 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/usr/lib64/libEosCommon.so.4+0x8f2c34) #2 0x7ff53e766d3c in void AssistedThread::reset<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*>(void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&) ../common/AssistedThread.hh:217 #3 0x7ff53e758042 in eos::fst::Storage::Storage(char const*) ../fst/storage/Storage.cc:184 #4 0x7ff53e7569be in eos::fst::Storage::Create(char const*) ../fst/storage/Storage.cc:56 #5 0x7ff53e5e2f63 in eos::fst::XrdFstOfs::Configure(XrdSysError&, XrdOucEnv*) ../fst/XrdFstOfs.cc:843 #6 0x7ff53e5dc668 in XrdSfsGetFileSystem2 ../fst/XrdFstOfs.cc:128 #7 0x7ff54889633a in XrdXrootdloadFileSystem(XrdSysError*, XrdSfsFileSystem*, char*, int, char const*, XrdOucEnv*) /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdLoadLib.cc:71 ==8066==ABORTING
apeters1971
pushed a commit
that referenced
this pull request
Aug 17, 2022
…eadPoolExecutor object and the local storage of the threads in this pool will lead to a segv due to a bug in the folly library. Therefore, we keep the FmdConverter object around until the FST daemon is shutdown. Referenced folly bug: facebook/folly#1252 The ASAN trace of such a crash: AddressSanitizer:DEADLYSIGNAL ================================================================= ==8066==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000038 (pc 0x7ff5370a3e11 bp 0x7ff521df0bf0 sp 0x7ff521df0bb0 T60) ==8066==The signal is caused by a READ memory access. ==8066==Hint: address points to the zero page. #0 0x7ff5370a3e10 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::get() const /root/rpmbuild/BUILD/eos-folly /folly-2019.11.11.00/folly/ThreadLocal.h:159 #1 0x7ff53709e9d9 in folly::ThreadLocal<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::get() const /root/rpmbuild/BUILD/eos-folly/fo lly-2019.11.11.00/folly/ThreadLocal.h:68 #2 0x7ff53709e9d9 in folly::ThreadLocal<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::operator*() const /root/rpmbuild/BUILD/eos-fo lly/folly-2019.11.11.00/folly/ThreadLocal.h:77 #3 0x7ff53709c0fe in folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::getWrapper() /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/folly/SingletonThreadL ocal.h:148 #4 0x7ff53709c112 in folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::LocalLifetime::~LocalLifetime() /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/fol ly/SingletonThreadLocal.h:120 #5 0x7ff547b80b18 in run ../../../../libstdc++-v3/libsupc++/atexit_thread.cc:64 #6 0x7ff54803aca1 in __nptl_deallocate_tsd /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:155 #7 0x7ff54803aeb2 in start_thread /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:314 #8 0x7ff54733c8dc in __clone (/lib64/libc.so.6+0xfe8dc) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/folly/ThreadLocal.h:159 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::Reque stContext> >, void>::Wrapper, void, void>::get() const Thread T60 created by T40 here: #0 0x7ff548ea7471 in pthread_create (/usr/lib64/libasan.so.5+0x3a471) #1 0x7ff539339c34 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/usr/lib64/libEosCommon.so.4+0x8f2c34) #2 0x7ff53e5b1e50 in void AssistedThread::reset<void (eos::fst::ScanDir::*)(ThreadAssistant&) noexcept, eos::fst::ScanDir*>(void (eos::fst::ScanDir::*&&)(ThreadAssistant&) noexcept, eos::fst::ScanDir*&&) (/lib64/libXrdEosFst-4.so+0x41be50) #3 0x7ff53e587b13 in eos::fst::ScanDir::SetConfig(std::string const&, long long) ../fst/ScanDir.cc:136 #4 0x7ff53e72dba2 in eos::fst::FileSystem::ConfigScanner(eos::fst::Load*, std::string const&, long long) ../fst/storage/FileSystem.cc:165 #5 0x7ff53e710721 in eos::fst::Storage::ProcessFsConfigChange(eos::fst::FileSystem*, std::string const&, std::string const&, std::string const&) ../fst/storage/Communicator.cc:321 #6 0x7ff53e711b67 in eos::fst::Storage::ProcessFsConfigChange(std::string const&, std::string const&) ../fst/storage/Communicator.cc:400 #7 0x7ff53e713e13 in eos::fst::Storage::Communicator(ThreadAssistant&) ../fst/storage/Communicator.cc:513 #8 0x7ff53e76f00f in void std::__invoke_impl<void, void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >(std::__invoke_memfun_deref, void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&, std::reference_wrap per<ThreadAssistant>&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/invoke.h:73 #9 0x7ff53e76a4cd in std::__invoke_result<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >::type std::__invoke<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >(void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&, std::reference_wrapper<ThreadAssistant>&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/invoke.h:95 #10 0x7ff53e7786c4 in decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > >::_M_invoke<0ul, 1ul, 2ul>(std ::_Index_tuple<0ul, 1ul, 2ul>) /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:244 #11 0x7ff53e778663 in std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > >::operator()() /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:253 #12 0x7ff53e778647 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > > >::_M_run() /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:196 #13 0x7ff539339bce in execute_native_thread_routine (/usr/lib64/libEosCommon.so.4+0x8f2bce) Thread T40 created by T0 here: #0 0x7ff548ea7471 in pthread_create (/usr/lib64/libasan.so.5+0x3a471) #1 0x7ff539339c34 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/usr/lib64/libEosCommon.so.4+0x8f2c34) #2 0x7ff53e766d3c in void AssistedThread::reset<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*>(void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&) ../common/AssistedThread.hh:217 #3 0x7ff53e758042 in eos::fst::Storage::Storage(char const*) ../fst/storage/Storage.cc:184 #4 0x7ff53e7569be in eos::fst::Storage::Create(char const*) ../fst/storage/Storage.cc:56 #5 0x7ff53e5e2f63 in eos::fst::XrdFstOfs::Configure(XrdSysError&, XrdOucEnv*) ../fst/XrdFstOfs.cc:843 #6 0x7ff53e5dc668 in XrdSfsGetFileSystem2 ../fst/XrdFstOfs.cc:128 #7 0x7ff54889633a in XrdXrootdloadFileSystem(XrdSysError*, XrdSfsFileSystem*, char*, int, char const*, XrdOucEnv*) /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdLoadLib.cc:71 ==8066==ABORTING
apeters1971
pushed a commit
that referenced
this pull request
Aug 19, 2022
…eadPoolExecutor object and the local storage of the threads in this pool will lead to a segv due to a bug in the folly library. Therefore, we keep the FmdConverter object around until the FST daemon is shutdown. Referenced folly bug: facebook/folly#1252 The ASAN trace of such a crash: AddressSanitizer:DEADLYSIGNAL ================================================================= ==8066==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000038 (pc 0x7ff5370a3e11 bp 0x7ff521df0bf0 sp 0x7ff521df0bb0 T60) ==8066==The signal is caused by a READ memory access. ==8066==Hint: address points to the zero page. #0 0x7ff5370a3e10 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::get() const /root/rpmbuild/BUILD/eos-folly /folly-2019.11.11.00/folly/ThreadLocal.h:159 #1 0x7ff53709e9d9 in folly::ThreadLocal<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::get() const /root/rpmbuild/BUILD/eos-folly/fo lly-2019.11.11.00/folly/ThreadLocal.h:68 #2 0x7ff53709e9d9 in folly::ThreadLocal<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::Wrapper, void, void>::operator*() const /root/rpmbuild/BUILD/eos-fo lly/folly-2019.11.11.00/folly/ThreadLocal.h:77 #3 0x7ff53709c0fe in folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::getWrapper() /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/folly/SingletonThreadL ocal.h:148 #4 0x7ff53709c112 in folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::RequestContext> >, void>::LocalLifetime::~LocalLifetime() /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/fol ly/SingletonThreadLocal.h:120 #5 0x7ff547b80b18 in run ../../../../libstdc++-v3/libsupc++/atexit_thread.cc:64 #6 0x7ff54803aca1 in __nptl_deallocate_tsd /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:155 #7 0x7ff54803aeb2 in start_thread /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:314 #8 0x7ff54733c8dc in __clone (/lib64/libc.so.6+0xfe8dc) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV /root/rpmbuild/BUILD/eos-folly/folly-2019.11.11.00/folly/ThreadLocal.h:159 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<std::shared_ptr<folly::RequestContext>, folly::detail::DefaultTag, folly::detail::DefaultMake<std::shared_ptr<folly::Reque stContext> >, void>::Wrapper, void, void>::get() const Thread T60 created by T40 here: #0 0x7ff548ea7471 in pthread_create (/usr/lib64/libasan.so.5+0x3a471) #1 0x7ff539339c34 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/usr/lib64/libEosCommon.so.4+0x8f2c34) #2 0x7ff53e5b1e50 in void AssistedThread::reset<void (eos::fst::ScanDir::*)(ThreadAssistant&) noexcept, eos::fst::ScanDir*>(void (eos::fst::ScanDir::*&&)(ThreadAssistant&) noexcept, eos::fst::ScanDir*&&) (/lib64/libXrdEosFst-4.so+0x41be50) #3 0x7ff53e587b13 in eos::fst::ScanDir::SetConfig(std::string const&, long long) ../fst/ScanDir.cc:136 #4 0x7ff53e72dba2 in eos::fst::FileSystem::ConfigScanner(eos::fst::Load*, std::string const&, long long) ../fst/storage/FileSystem.cc:165 #5 0x7ff53e710721 in eos::fst::Storage::ProcessFsConfigChange(eos::fst::FileSystem*, std::string const&, std::string const&, std::string const&) ../fst/storage/Communicator.cc:321 #6 0x7ff53e711b67 in eos::fst::Storage::ProcessFsConfigChange(std::string const&, std::string const&) ../fst/storage/Communicator.cc:400 #7 0x7ff53e713e13 in eos::fst::Storage::Communicator(ThreadAssistant&) ../fst/storage/Communicator.cc:513 #8 0x7ff53e76f00f in void std::__invoke_impl<void, void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >(std::__invoke_memfun_deref, void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&, std::reference_wrap per<ThreadAssistant>&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/invoke.h:73 #9 0x7ff53e76a4cd in std::__invoke_result<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >::type std::__invoke<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> >(void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&, std::reference_wrapper<ThreadAssistant>&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/invoke.h:95 #10 0x7ff53e7786c4 in decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > >::_M_invoke<0ul, 1ul, 2ul>(std ::_Index_tuple<0ul, 1ul, 2ul>) /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:244 #11 0x7ff53e778663 in std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > >::operator()() /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:253 #12 0x7ff53e778647 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*, std::reference_wrapper<ThreadAssistant> > > >::_M_run() /opt/rh/devtoolset-8/root/usr/include/c++/8/thread:196 #13 0x7ff539339bce in execute_native_thread_routine (/usr/lib64/libEosCommon.so.4+0x8f2bce) Thread T40 created by T0 here: #0 0x7ff548ea7471 in pthread_create (/usr/lib64/libasan.so.5+0x3a471) #1 0x7ff539339c34 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/usr/lib64/libEosCommon.so.4+0x8f2c34) #2 0x7ff53e766d3c in void AssistedThread::reset<void (eos::fst::Storage::*)(ThreadAssistant&), eos::fst::Storage*>(void (eos::fst::Storage::*&&)(ThreadAssistant&), eos::fst::Storage*&&) ../common/AssistedThread.hh:217 #3 0x7ff53e758042 in eos::fst::Storage::Storage(char const*) ../fst/storage/Storage.cc:184 #4 0x7ff53e7569be in eos::fst::Storage::Create(char const*) ../fst/storage/Storage.cc:56 #5 0x7ff53e5e2f63 in eos::fst::XrdFstOfs::Configure(XrdSysError&, XrdOucEnv*) ../fst/XrdFstOfs.cc:843 #6 0x7ff53e5dc668 in XrdSfsGetFileSystem2 ../fst/XrdFstOfs.cc:128 #7 0x7ff54889633a in XrdXrootdloadFileSystem(XrdSysError*, XrdSfsFileSystem*, char*, int, char const*, XrdOucEnv*) /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdLoadLib.cc:71 ==8066==ABORTING
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
eos-fuse-sysv package's name was changed 2 month ago.