Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Double deletion at Aeron destructor #717

Closed
denizevrenci opened this issue Oct 12, 2019 · 5 comments
Closed

Double deletion at Aeron destructor #717

denizevrenci opened this issue Oct 12, 2019 · 5 comments
Labels
bug

Comments

@denizevrenci
Copy link
Contributor

denizevrenci commented Oct 12, 2019 •
edited
Loading

I am getting a double deletion error with the code below:

#include <Aeron.h>

int main()
{
  aeron::Context ctx;
  aeron::Aeron proxy(ctx);

  proxy.addSubscription("aeron:udp?endpoint=localhost:50000", 0);
  const auto pub_reg_id = proxy.addPublication("aeron:udp?endpoint=localhost:50000", 0);

  auto pub = proxy.findPublication(pub_reg_id);
  while (!pub)
    pub = proxy.findPublication(pub_reg_id);
}

Address sanitizer output is as follows:

==14==ERROR: AddressSanitizer: heap-use-after-free on address 0x61100000ff40 at pc 0x556f490994c3 bp 0x7f55658fce00 sp 0x7f55658fcdf0
READ of size 8 at 0x61100000ff40 thread T1
    #0 0x556f490994c2 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/9/bits/shared_ptr_base.h:730
    #1 0x556f490dd519 in std::__shared_ptr<aeron::Subscription, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/9/bits/shared_ptr_base.h:1169
    #2 0x556f490dd5a0 in std::shared_ptr<aeron::Subscription>::~shared_ptr() /usr/include/c++/9/bits/shared_ptr.h:103
    #3 0x556f490e18bd in aeron::ClientConductor::SubscriptionStateDefn::~SubscriptionStateDefn() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.h:325
    #4 0x556f4913779d in std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>::~pair() /usr/include/c++/9/bits/stl_pair.h:208
    #5 0x556f49137803 in void __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >::destroy<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >(std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>*) /usr/include/c++/9/ext/new_allocator.h:153
    #6 0x556f4912b3e6 in void std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::destroy<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >(std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >&, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>*) /usr/include/c++/9/bits/alloc_traits.h:497
    #7 0x556f4911cddf in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable_policy.h:2091
    #8 0x556f49125588 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable.h:1892
    #9 0x556f49115c63 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::erase(std::__detail::_Node_const_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/hashtable.h:1867
    #10 0x556f49106437 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::erase(std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/hashtable.h:768
    #11 0x556f490f3e3c in std::unordered_map<long, aeron::ClientConductor::SubscriptionStateDefn, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> > >::erase(std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/unordered_map.h:798
    #12 0x556f490abc4f in aeron::ClientConductor::releaseSubscription(long, std::shared_ptr<aeron::Image>*, unsigned long) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.cpp:293
    #13 0x556f4915a1a6 in aeron::Subscription::~Subscription() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/Subscription.cpp:42
    #14 0x556f4914d58b in void __gnu_cxx::new_allocator<aeron::Subscription>::destroy<aeron::Subscription>(aeron::Subscription*) /usr/include/c++/9/ext/new_allocator.h:153
    #15 0x556f4914d33a in void std::allocator_traits<std::allocator<aeron::Subscription> >::destroy<aeron::Subscription>(std::allocator<aeron::Subscription>&, aeron::Subscription*) /usr/include/c++/9/bits/alloc_traits.h:497
    #16 0x556f4914af77 in std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/9/bits/shared_ptr_base.h:557
    #17 0x556f4909b60f in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:155
    #18 0x556f49099554 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/9/bits/shared_ptr_base.h:730
    #19 0x556f490dd519 in std::__shared_ptr<aeron::Subscription, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/9/bits/shared_ptr_base.h:1169
    #20 0x556f490dd5a0 in std::shared_ptr<aeron::Subscription>::~shared_ptr() /usr/include/c++/9/bits/shared_ptr.h:103
    #21 0x556f490e18bd in aeron::ClientConductor::SubscriptionStateDefn::~SubscriptionStateDefn() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.h:325
    #22 0x556f4913779d in std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>::~pair() /usr/include/c++/9/bits/stl_pair.h:208
    #23 0x556f49137803 in void __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >::destroy<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >(std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>*) /usr/include/c++/9/ext/new_allocator.h:153
    #24 0x556f4912b3e6 in void std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::destroy<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >(std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >&, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>*) /usr/include/c++/9/bits/alloc_traits.h:497
    #25 0x556f4911cddf in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable_policy.h:2091
    #26 0x556f4910ef85 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::_M_deallocate_nodes(std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable_policy.h:2113
    #27 0x556f490fdeb4 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::clear() /usr/include/c++/9/bits/hashtable.h:2034
    #28 0x556f490f93a4 in std::unordered_map<long, aeron::ClientConductor::SubscriptionStateDefn, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> > >::clear() /usr/include/c++/9/bits/unordered_map.h:843
    #29 0x556f490c1cc6 in aeron::ClientConductor::closeAllResources(long long) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.cpp:940
    #30 0x556f4916faf4 in aeron::ClientConductor::onClose() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.h:136
    #31 0x556f49182cc8 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::run() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:140
    #32 0x556f4917f379 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}::operator()() const ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:104
    #33 0x556f49187cc4 in void std::__invoke_impl<void, aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>(std::__invoke_other, aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}&&) /usr/include/c++/9/bits/invoke.h:60
    #34 0x556f49187ba0 in std::__invoke_result<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>::type std::__invoke<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>(std::__invoke_result&&, (aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}&&)...) /usr/include/c++/9/bits/invoke.h:95
    #35 0x556f49187a96 in void std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/include/c++/9/thread:244
    #36 0x556f491879e0 in std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> >::operator()() /usr/include/c++/9/thread:251
    #37 0x556f4918799f in std::thread::_State_impl<std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> > >::_M_run() /usr/include/c++/9/thread:195
    #38 0x7f556a2d762f  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xd062f)
    #39 0x7f556a3f2181 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9181)
    #40 0x7f55697b7b1e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11db1e)

0x61100000ff40 is located 0 bytes inside of 240-byte region [0x61100000ff40,0x611000010030)
freed by thread T1 here:
    #0 0x7f556a5190ff in operator delete(void*) (/lib/x86_64-linux-gnu/libasan.so.5+0x10f0ff)
    #1 0x556f491476a9 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> >::deallocate(std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2>*, unsigned long) /usr/include/c++/9/ext/new_allocator.h:128
    #2 0x556f49144593 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> > >::deallocate(std::allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> >&, std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2>*, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:470
    #3 0x556f4913f000 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> > >::~__allocated_ptr() /usr/include/c++/9/bits/allocated_ptr.h:73
    #4 0x556f4914b211 in std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2>::_M_destroy() /usr/include/c++/9/bits/shared_ptr_base.h:565
    #5 0x556f49100f0d in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_weak_release() /usr/include/c++/9/bits/shared_ptr_base.h:194
    #6 0x556f490ecd6a in std::__weak_count<(__gnu_cxx::_Lock_policy)2>::~__weak_count() /usr/include/c++/9/bits/shared_ptr_base.h:823
    #7 0x556f490dd64d in std::__weak_ptr<aeron::Subscription, (__gnu_cxx::_Lock_policy)2>::~__weak_ptr() /usr/include/c++/9/bits/shared_ptr_base.h:1596
    #8 0x556f490dd736 in std::weak_ptr<aeron::Subscription>::~weak_ptr() /usr/include/c++/9/bits/shared_ptr.h:531
    #9 0x556f490e1845 in aeron::ClientConductor::SubscriptionStateDefn::~SubscriptionStateDefn() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.h:325
    #10 0x556f4913779d in std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>::~pair() /usr/include/c++/9/bits/stl_pair.h:208
    #11 0x556f49137803 in void __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >::destroy<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >(std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>*) /usr/include/c++/9/ext/new_allocator.h:153
    #12 0x556f4912b3e6 in void std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::destroy<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >(std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >&, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>*) /usr/include/c++/9/bits/alloc_traits.h:497
    #13 0x556f4911cddf in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable_policy.h:2091
    #14 0x556f49125588 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable.h:1892
    #15 0x556f49115c63 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::erase(std::__detail::_Node_const_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/hashtable.h:1867
    #16 0x556f49106437 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::erase(std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/hashtable.h:768
    #17 0x556f490f3e3c in std::unordered_map<long, aeron::ClientConductor::SubscriptionStateDefn, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> > >::erase(std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/unordered_map.h:798
    #18 0x556f490abc4f in aeron::ClientConductor::releaseSubscription(long, std::shared_ptr<aeron::Image>*, unsigned long) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.cpp:293
    #19 0x556f4915a1a6 in aeron::Subscription::~Subscription() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/Subscription.cpp:42
    #20 0x556f4914d58b in void __gnu_cxx::new_allocator<aeron::Subscription>::destroy<aeron::Subscription>(aeron::Subscription*) /usr/include/c++/9/ext/new_allocator.h:153
    #21 0x556f4914d33a in void std::allocator_traits<std::allocator<aeron::Subscription> >::destroy<aeron::Subscription>(std::allocator<aeron::Subscription>&, aeron::Subscription*) /usr/include/c++/9/bits/alloc_traits.h:497
    #22 0x556f4914af77 in std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/9/bits/shared_ptr_base.h:557
    #23 0x556f4909b60f in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:155
    #24 0x556f49099554 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/9/bits/shared_ptr_base.h:730
    #25 0x556f490dd519 in std::__shared_ptr<aeron::Subscription, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/9/bits/shared_ptr_base.h:1169
    #26 0x556f490dd5a0 in std::shared_ptr<aeron::Subscription>::~shared_ptr() /usr/include/c++/9/bits/shared_ptr.h:103
    #27 0x556f490e18bd in aeron::ClientConductor::SubscriptionStateDefn::~SubscriptionStateDefn() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.h:325
    #28 0x556f4913779d in std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>::~pair() /usr/include/c++/9/bits/stl_pair.h:208
    #29 0x556f49137803 in void __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >::destroy<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >(std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>*) /usr/include/c++/9/ext/new_allocator.h:153

previously allocated by thread T1 here:
    #0 0x7f556a51817f in operator new(unsigned long) (/lib/x86_64-linux-gnu/libasan.so.5+0x10e17f)
    #1 0x556f49147678 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /usr/include/c++/9/ext/new_allocator.h:114
    #2 0x556f4914443e in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:444
    #3 0x556f4913ee9b in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2> >&) /usr/include/c++/9/bits/allocated_ptr.h:97
    #4 0x556f49134f94 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<aeron::Subscription, std::allocator<aeron::Subscription>, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&>(aeron::Subscription*&, std::_Sp_alloc_shared_tag<std::allocator<aeron::Subscription> >, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&) /usr/include/c++/9/bits/shared_ptr_base.h:677
    #5 0x556f491295a5 in std::__shared_ptr<aeron::Subscription, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<aeron::Subscription>, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&>(std::_Sp_alloc_shared_tag<std::allocator<aeron::Subscription> >, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&) /usr/include/c++/9/bits/shared_ptr_base.h:1344
    #6 0x556f4911929a in std::shared_ptr<aeron::Subscription>::shared_ptr<std::allocator<aeron::Subscription>, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&>(std::_Sp_alloc_shared_tag<std::allocator<aeron::Subscription> >, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&) /usr/include/c++/9/bits/shared_ptr.h:359
    #7 0x556f49109a0a in std::shared_ptr<aeron::Subscription> std::allocate_shared<aeron::Subscription, std::allocator<aeron::Subscription>, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&>(std::allocator<aeron::Subscription> const&, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&) /usr/include/c++/9/bits/shared_ptr.h:702
    #8 0x556f490f73a9 in std::shared_ptr<aeron::Subscription> std::make_shared<aeron::Subscription, aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&>(aeron::ClientConductor&, long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int&, int&) /usr/include/c++/9/bits/shared_ptr.h:718
    #9 0x556f490b8868 in aeron::ClientConductor::onSubscriptionReady(long, int) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.cpp:627
    #10 0x556f4917b929 in aeron::DriverListenerAdapter<aeron::ClientConductor>::receiveMessages()::{lambda(int, aeron::concurrent::AtomicBuffer&, int, int)#1}::operator()(int, aeron::concurrent::AtomicBuffer&, int, int) const ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/DriverListenerAdapter.h:88
    #11 0x556f49184445 in std::_Function_handler<void (int, aeron::concurrent::AtomicBuffer&, int, int), aeron::DriverListenerAdapter<aeron::ClientConductor>::receiveMessages()::{lambda(int, aeron::concurrent::AtomicBuffer&, int, int)#1}>::_M_invoke(std::_Any_data const&, int&&, aeron::concurrent::AtomicBuffer&, std::_Any_data const&, std::_Any_data const&) /usr/include/c++/9/bits/std_function.h:300
    #12 0x556f49178204 in std::function<void (int, aeron::concurrent::AtomicBuffer&, int, int)>::operator()(int, aeron::concurrent::AtomicBuffer&, int, int) const /usr/include/c++/9/bits/std_function.h:690
    #13 0x556f49166e67 in aeron::concurrent::broadcast::CopyBroadcastReceiver::receive(std::function<void (int, aeron::concurrent::AtomicBuffer&, int, int)> const&) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/broadcast/CopyBroadcastReceiver.h:72
    #14 0x556f4917cff0 in aeron::DriverListenerAdapter<aeron::ClientConductor>::receiveMessages() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/DriverListenerAdapter.h:174
    #15 0x556f4916f643 in aeron::ClientConductor::doWork() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.h:125
    #16 0x556f49182b69 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::run() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:130
    #17 0x556f4917f379 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}::operator()() const ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:104
    #18 0x556f49187cc4 in void std::__invoke_impl<void, aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>(std::__invoke_other, aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}&&) /usr/include/c++/9/bits/invoke.h:60
    #19 0x556f49187ba0 in std::__invoke_result<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>::type std::__invoke<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>(std::__invoke_result&&, (aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}&&)...) /usr/include/c++/9/bits/invoke.h:95
    #20 0x556f49187a96 in void std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/include/c++/9/thread:244
    #21 0x556f491879e0 in std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> >::operator()() /usr/include/c++/9/thread:251
    #22 0x556f4918799f in std::thread::_State_impl<std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> > >::_M_run() /usr/include/c++/9/thread:195
    #23 0x7f556a2d762f  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xd062f)

Thread T1 created by T0 here:
    #0 0x7f556a444311 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a311)
    #1 0x7f556a2d78c4 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xd08c4)
    #2 0x556f4917f481 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:96
    #3 0x556f491600ab in aeron::Aeron::Aeron(aeron::Context&) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/Aeron.cpp:69
    #4 0x556f4908d58c in main ../../src/transport/test/operation/cpp/regression.cpp:10
    #5 0x7f55696c0b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

SUMMARY: AddressSanitizer: heap-use-after-free /usr/include/c++/9/bits/shared_ptr_base.h:730 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count()
Shadow bytes around the buggy address:
  0x0c227fff9f90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff9fa0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff9fb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff9fc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff9fd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c227fff9fe0: fa fa fa fa fa fa fa fa[fd]fd fd fd fd fd fd fd
  0x0c227fff9ff0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c227fffa000: fd fd fd fd fd fd fa fa fa fa fa fa fa fa fa fa
  0x0c227fffa010: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fffa020: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fffa030: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==14==ABORTING

Other information:

  • Using Java driver and version 1.22.1
  • This issue is only reproducible with UDP channels.
  • A publication (synchronized or exclusive) has to be added to reproduce.
  • Cannot reproduce if I obtain the registration ID of the subscription and get the subscription object.
@tmontgomery
Copy link
Contributor

I think the change to closing all resources in onClose feeb7c1 caused this. And potentially other double frees when the state object map is free'd while in a position of not transferring ownership to the application via a call to find. As a work around, please ensure find is used until we can untangle

tmontgomery added a commit that referenced this issue Oct 12, 2019
@tmontgomery
[C++]: add reset of subscription and counter caches to onClose for #717.
f9a9a64
@tmontgomery
Copy link
Contributor

@denizevrenci try the small change I added to onClose to see if the double free for this case goes away for you.

There is more that needs to be done, but this might be better.

@denizevrenci
Copy link
Contributor Author

Unfortunately, your fix has not solved the issue.

The stack trace from Address Sanitizer this time is as follows:

==15==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000000040 at pc 0x555eb2ab5ff0 bp 0x7fd59e7fd6f0 sp 0x7fd59e7fd6e0
READ of size 8 at 0x611000000040 thread T1
    #0 0x555eb2ab5fef in std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>::_M_next() const /usr/include/c++/9/bits/hashtable_policy.h:283
    #1 0x555eb2aa66f0 in std::__detail::_Node_iterator_base<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>::_M_incr() /usr/include/c++/9/bits/hashtable_policy.h:299
    #2 0x555eb2a92c2c in std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>::operator++() /usr/include/c++/9/bits/hashtable_policy.h:354
    #3 0x555eb2a62e2e in aeron::ClientConductor::closeAllResources(long long) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.cpp:918
    #4 0x555eb2b15b12 in aeron::ClientConductor::onClose() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.h:136
    #5 0x555eb2b28ce6 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::run() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:140
    #6 0x555eb2b25397 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}::operator()() const ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:104
    #7 0x555eb2b2dce2 in void std::__invoke_impl<void, aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>(std::__invoke_other, aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}&&) /usr/include/c++/9/bits/invoke.h:60
    #8 0x555eb2b2dbbe in std::__invoke_result<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>::type std::__invoke<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>(std::__invoke_result&&, (aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}&&)...) /usr/include/c++/9/bits/invoke.h:95
    #9 0x555eb2b2dab4 in void std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/include/c++/9/thread:244
    #10 0x555eb2b2d9fe in std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> >::operator()() /usr/include/c++/9/thread:251
    #11 0x555eb2b2d9bd in std::thread::_State_impl<std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> > >::_M_run() /usr/include/c++/9/thread:195
    #12 0x7fd5a31f562f  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xd062f)
    #13 0x7fd5a3310181 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9181)
    #14 0x7fd5a26d5b1e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11db1e)

0x611000000040 is located 0 bytes inside of 208-byte region [0x611000000040,0x611000000110)
freed by thread T1 here:
    #0 0x7fd5a34370ff in operator delete(void*) (/lib/x86_64-linux-gnu/libasan.so.5+0x10f0ff)
    #1 0x555eb2ae19c1 in __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >::deallocate(std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*, unsigned long) /usr/include/c++/9/ext/new_allocator.h:128
    #2 0x555eb2ad5e4a in std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::deallocate(std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >&, std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:470
    #3 0x555eb2ad03c7 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::_M_deallocate_node_ptr(std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable_policy.h:2102
    #4 0x555eb2ac1d26 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable_policy.h:2092
    #5 0x555eb2aca4a4 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>*) /usr/include/c++/9/bits/hashtable.h:1892
    #6 0x555eb2aba699 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::erase(std::__detail::_Node_const_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/hashtable.h:1867
    #7 0x555eb2aaad01 in std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::erase(std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/hashtable.h:768
    #8 0x555eb2a98508 in std::unordered_map<long, aeron::ClientConductor::SubscriptionStateDefn, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> > >::erase(std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>) /usr/include/c++/9/bits/unordered_map.h:798
    #9 0x555eb2a4cd55 in aeron::ClientConductor::releaseSubscription(long, std::shared_ptr<aeron::Image>*, unsigned long) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.cpp:293
    #10 0x555eb2b001c4 in aeron::Subscription::~Subscription() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/Subscription.cpp:42
    #11 0x555eb2af292d in void __gnu_cxx::new_allocator<aeron::Subscription>::destroy<aeron::Subscription>(aeron::Subscription*) /usr/include/c++/9/ext/new_allocator.h:153
    #12 0x555eb2af26dc in void std::allocator_traits<std::allocator<aeron::Subscription> >::destroy<aeron::Subscription>(std::allocator<aeron::Subscription>&, aeron::Subscription*) /usr/include/c++/9/bits/alloc_traits.h:497
    #13 0x555eb2af0319 in std::_Sp_counted_ptr_inplace<aeron::Subscription, std::allocator<aeron::Subscription>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/9/bits/shared_ptr_base.h:557
    #14 0x555eb2a3d48f in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:155
    #15 0x555eb2a3bac4 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/9/bits/shared_ptr_base.h:730
    #16 0x555eb2a812a3 in std::__shared_ptr<aeron::Subscription, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/9/bits/shared_ptr_base.h:1169
    #17 0x555eb2a8132a in std::shared_ptr<aeron::Subscription>::~shared_ptr() /usr/include/c++/9/bits/shared_ptr.h:103
    #18 0x555eb2a62e09 in aeron::ClientConductor::closeAllResources(long long) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.cpp:920
    #19 0x555eb2b15b12 in aeron::ClientConductor::onClose() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.h:136
    #20 0x555eb2b28ce6 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::run() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:140
    #21 0x555eb2b25397 in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}::operator()() const ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:104
    #22 0x555eb2b2dce2 in void std::__invoke_impl<void, aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>(std::__invoke_other, aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}&&) /usr/include/c++/9/bits/invoke.h:60
    #23 0x555eb2b2dbbe in std::__invoke_result<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>::type std::__invoke<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}>(std::__invoke_result&&, (aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}&&)...) /usr/include/c++/9/bits/invoke.h:95
    #24 0x555eb2b2dab4 in void std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/include/c++/9/thread:244
    #25 0x555eb2b2d9fe in std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> >::operator()() /usr/include/c++/9/thread:251
    #26 0x555eb2b2d9bd in std::thread::_State_impl<std::thread::_Invoker<std::tuple<aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start()::{lambda()#1}> > >::_M_run() /usr/include/c++/9/thread:195
    #27 0x7fd5a31f562f  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xd062f)

previously allocated by thread T0 here:
    #0 0x7fd5a343617f in operator new(unsigned long) (/lib/x86_64-linux-gnu/libasan.so.5+0x10e17f)
    #1 0x555eb2ae190e in __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >::allocate(unsigned long, void const*) /usr/include/c++/9/ext/new_allocator.h:114
    #2 0x555eb2ad5d81 in std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::allocate(std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> >&, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:444
    #3 0x555eb2ac8da7 in std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>* std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false> > >::_M_allocate_node<std::pair<long, aeron::ClientConductor::SubscriptionStateDefn> >(std::pair<long, aeron::ClientConductor::SubscriptionStateDefn>&&) /usr/include/c++/9/bits/hashtable_policy.h:2070
    #4 0x555eb2ab9a9f in std::pair<std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>, bool> std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_emplace<std::pair<long, aeron::ClientConductor::SubscriptionStateDefn> >(std::integral_constant<bool, true>, std::pair<long, aeron::ClientConductor::SubscriptionStateDefn>&&) /usr/include/c++/9/bits/hashtable.h:1644
    #5 0x555eb2aaa4ef in std::pair<std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>, bool> std::_Hashtable<long, std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::emplace<std::pair<long, aeron::ClientConductor::SubscriptionStateDefn> >(std::pair<long, aeron::ClientConductor::SubscriptionStateDefn>&&) /usr/include/c++/9/bits/hashtable.h:749
    #6 0x555eb2a97e97 in std::enable_if<std::is_constructible<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, std::pair<long, aeron::ClientConductor::SubscriptionStateDefn>&&>::value, std::pair<std::__detail::_Node_iterator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false, false>, bool> >::type std::unordered_map<long, aeron::ClientConductor::SubscriptionStateDefn, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn> > >::insert<std::pair<long, aeron::ClientConductor::SubscriptionStateDefn> >(std::pair<long, aeron::ClientConductor::SubscriptionStateDefn>&&) /usr/include/c++/9/bits/unordered_map.h:592
    #7 0x555eb2a4a630 in aeron::ClientConductor::addSubscription(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, std::function<void (aeron::Image&)> const&, std::function<void (aeron::Image&)> const&) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/ClientConductor.cpp:232
    #8 0x555eb2a3b36c in aeron::Aeron::addSubscription(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/Aeron.h:190
    #9 0x555eb2a37401 in main ../../src/transport/test/operation/cpp/regression.cpp:9
    #10 0x7fd5a25deb6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

Thread T1 created by T0 here:
    #0 0x7fd5a3362311 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a311)
    #1 0x7fd5a31f58c4 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xd08c4)
    #2 0x555eb2b2549f in aeron::concurrent::AgentRunner<aeron::ClientConductor, aeron::concurrent::SleepingIdleStrategy>::start() ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/concurrent/AgentRunner.h:96
    #3 0x555eb2b060c9 in aeron::Aeron::Aeron(aeron::Context&) ../common/thirdparty/aeron/source/aeron-client/src/main/cpp/Aeron.cpp:69
    #4 0x555eb2a3736a in main ../../src/transport/test/operation/cpp/regression.cpp:7
    #5 0x7fd5a25deb6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

SUMMARY: AddressSanitizer: heap-use-after-free /usr/include/c++/9/bits/hashtable_policy.h:283 in std::__detail::_Hash_node<std::pair<long const, aeron::ClientConductor::SubscriptionStateDefn>, false>::_M_next() const
Shadow bytes around the buggy address:
  0x0c227fff7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c227fff7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c227fff7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c227fff7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c227fff7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c227fff8000: fa fa fa fa fa fa fa fa[fd]fd fd fd fd fd fd fd
  0x0c227fff8010: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c227fff8020: fd fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff8030: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff8040: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff8050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==15==ABORTING

By the way, I am using GCC 9 to compile the source.

tmontgomery added a commit that referenced this issue Oct 13, 2019
@tmontgomery
[C++]: add temporary guard against closeAllResources calling back int…
7b0f503 
…o releaseSubscription. #717.
@tmontgomery
Copy link
Contributor

tmontgomery commented Oct 13, 2019 •
edited
Loading

Was not hopeful. The way that closeAllResources handles things, it would go into releaseSubscription. But I've added a guard, temporarily. Until the logic can be reworked to be cleaner and ownership straightened out.

@denizevrenci
Copy link
Contributor Author

denizevrenci commented Oct 13, 2019 •
edited
Loading

I can confirm that your second commit fixes the double deletion.

epickrram pushed a commit to epickrram/aeron that referenced this issue Oct 28, 2019
@tmontgomery @epickrram
[C++]: add reset of subscription and counter caches to onClose for re…
3545904 
…al-logic#717.
epickrram pushed a commit to epickrram/aeron that referenced this issue Oct 28, 2019
@tmontgomery @epickrram
[C++]: add temporary guard against closeAllResources calling back int…
8e4100e 
…o releaseSubscription. real-logic#717.
mikeb01 pushed a commit to mikeb01/Aeron that referenced this issue Oct 29, 2019
@tmontgomery @mikeb01
[C++]: add reset of subscription and counter caches to onClose for re…
3b87877 
…al-logic#717.
mikeb01 pushed a commit to mikeb01/Aeron that referenced this issue Oct 29, 2019
@tmontgomery @mikeb01
[C++]: add temporary guard against closeAllResources calling back int…
c8a4916 
…o releaseSubscription. real-logic#717.
@mjpt777 mjpt777 closed this as completed Nov 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mjpt777 @tmontgomery @denizevrenci