Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wip #1

Merged
merged 1 commit into from Aug 16, 2019
Merged

wip #1

merged 1 commit into from Aug 16, 2019

Conversation

tchaikov
Copy link

Signed-off-by: Kefu Chai kchai@redhat.com

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

Signed-off-by: Kefu Chai <kchai@redhat.com>
Copy link
Owner

@rzarzynski rzarzynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the commit, @tchaikov! I will alter the commit title manually and rebase.

@rzarzynski rzarzynski merged commit ea0b967 into rzarzynski:wip-denc-container_base Aug 16, 2019
@tchaikov tchaikov deleted the wip-24636 branch August 16, 2019 12:12
rzarzynski pushed a commit that referenced this pull request Jan 16, 2020
otherwise ODR is violated:

==449025==ERROR: AddressSanitizer: odr-violation (0x000000f03700):
  [1] size=8 'g_ceph_context' ../src/global/global_context.cc:24:14
  [2] size=8 'g_ceph_context' ../src/global/global_context.cc:24:14
These globals were registered at these points:
  [1]:
    #0 0x4779bd in __asan_register_globals (/var/ssd/ceph/clang-build/bin/ceph-conf+0x4779bd)
    #1 0x56e9cb in asan.module_ctor (/var/ssd/ceph/clang-build/bin/ceph-conf+0x56e9cb)

  [2]:
    #0 0x4779bd in __asan_register_globals (/var/ssd/ceph/clang-build/bin/ceph-conf+0x4779bd)
    #1 0x7fe5fed12aeb in asan.module_ctor (/var/ssd/ceph/clang-build/lib/libceph-common.so.2+0x2f34aeb)

==449025==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0

Signed-off-by: Kefu Chai <kchai@redhat.com>
rzarzynski added a commit that referenced this pull request Feb 12, 2020
Accordingly to cppreference.com [1]:

  "If multiple threads of execution access the same std::shared_ptr
  object without synchronization and any of those accesses uses
  a non-const member function of shared_ptr then a data race will
  occur (...)"

[1]: https://en.cppreference.com/w/cpp/memory/shared_ptr/atomic

One of the coredumps showed the `shared_ptr`-typed `OSD::osdmap`
with healthy looking content but damaged control block:

  ```
  [Current thread is 1 (Thread 0x7f7dcaf73700 (LWP 205295))]
  (gdb) bt
  #0  0x0000559cb81c3ea0 in ?? ()
  #1  0x0000559c97675b27 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  #2  std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  ceph#3  0x0000559c975ef8aa in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1167
  ceph#4  std::__shared_ptr<OSDMap const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1167
  ceph#5  std::shared_ptr<OSDMap const>::~shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr.h:103
  ceph#6  OSD::create_context (this=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9053
  ceph#7  0x0000559c97655571 in OSD::dequeue_peering_evt (this=0x559ca22ac000, sdata=0x559ca2ef2900, pg=0x559cb4aa3400, evt=std::shared_ptr<PGPeeringEvent> (use count 2, weak count 0) = {...}, handle=...)
      at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9665
  ceph#8  0x0000559c97886db6 in ceph::osd::scheduler::PGPeeringItem::run (this=<optimized out>, osd=<optimized out>, sdata=<optimized out>, pg=..., handle=...) at /usr/include/c++/8/ext/atomicity.h:96
  ceph#9  0x0000559c9764862f in ceph::osd::scheduler::OpSchedulerItem::run (handle=..., pg=..., sdata=<optimized out>, osd=<optimized out>, this=0x7f7dcaf703f0) at /usr/include/c++/8/bits/unique_ptr.h:342
  ceph#10 OSD::ShardedOpWQ::_process (this=<optimized out>, thread_index=<optimized out>, hb=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:10677
  ceph#11 0x0000559c97c76094 in ShardedThreadPool::shardedthreadpool_worker (this=0x559ca22aca28, thread_index=14) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/common/WorkQueue.cc:311
  ceph#12 0x0000559c97c78cf4 in ShardedThreadPool::WorkThreadSharded::entry (this=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/common/WorkQueue.h:706
  ceph#13 0x00007f7df17852de in start_thread () from /lib64/libpthread.so.0
  ceph#14 0x00007f7df052f133 in __libc_ifunc_impl_list () from /lib64/libc.so.6
  ceph#15 0x0000000000000000 in ?? ()
  (gdb) frame 7
  ceph#7  0x0000559c97655571 in OSD::dequeue_peering_evt (this=0x559ca22ac000, sdata=0x559ca2ef2900, pg=0x559cb4aa3400, evt=std::shared_ptr<PGPeeringEvent> (use count 2, weak count 0) = {...}, handle=...)
      at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9665
  9665      in /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc
  (gdb) print osdmap
  $24 = std::shared_ptr<const OSDMap> (expired, weak count 0) = {get() = 0x559cba028000}
  (gdb) print *osdmap
     # pretty sane OSDMap
  (gdb) print sizeof(osdmap)
  $26 = 16
  (gdb) x/2a &osdmap
  0x559ca22acef0:   0x559cba028000  0x559cba0ec900

  (gdb) frame 2
  #2  std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  148       /usr/include/c++/8/bits/shared_ptr_base.h: No such file or directory.
  (gdb) disassemble
  Dump of assembler code for function std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release():
  ...
     0x0000559c97675b1e <+62>:      mov    (%rdi),%rax
     0x0000559c97675b21 <+65>:      mov    %rdi,%rbx
     0x0000559c97675b24 <+68>:      callq  *0x10(%rax)
  => 0x0000559c97675b27 <+71>:      test   %rbp,%rbp
  ...
  End of assembler dump.
  (gdb) info registers rdi rbx rax
  rdi            0x559cba0ec900      94131624790272
  rbx            0x559cba0ec900      94131624790272
  rax            0x559cba0ec8a0      94131624790176
  (gdb) x/a 0x559cba0ec8a0 + 0x10
  0x559cba0ec8b0:   0x559cb81c3ea0
  (gdb) bt
  #0  0x0000559cb81c3ea0 in ?? ()
  ...
  (gdb) p $_siginfo._sifields._sigfault.si_addr
  $27 = (void *) 0x559cb81c3ea0
  ```

Helgrind seems to agree:
  ```
  ==00:00:02:54.519 510301== Possible data race during write of size 8 at 0xF123930 by thread ceph#90
  ==00:00:02:54.519 510301== Locks held: 2, at addresses 0xF122A58 0xF1239A8
  ==00:00:02:54.519 510301==    at 0x7218DD: operator= (shared_ptr_base.h:1078)
  ==00:00:02:54.519 510301==    by 0x7218DD: operator= (shared_ptr.h:103)
  ==00:00:02:54.519 510301==    by 0x7218DD: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8116)
  ==00:00:02:54.519 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:02:54.519 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:02:54.519 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:02:54.519 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:02:54.519 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:02:54.519 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:02:54.519 510301==
  ==00:00:02:54.519 510301== This conflicts with a previous read of size 8 by thread ceph#117
  ==00:00:02:54.519 510301== Locks held: 1, at address 0x2123E9A0
  ==00:00:02:54.519 510301==    at 0x6B5842: __shared_ptr (shared_ptr_base.h:1165)
  ==00:00:02:54.519 510301==    by 0x6B5842: shared_ptr (shared_ptr.h:129)
  ==00:00:02:54.519 510301==    by 0x6B5842: get_osdmap (OSD.h:1700)
  ==00:00:02:54.519 510301==    by 0x6B5842: OSD::create_context() (OSD.cc:9053)
  ==00:00:02:54.519 510301==    by 0x71B570: OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr<PGPeeringEvent>, ThreadPool::TPHandle&) (OSD.cc:9665)
  ==00:00:02:54.519 510301==    by 0x71B997: OSD::dequeue_delete(OSDShard*, PG*, unsigned int, ThreadPool::TPHandle&) (OSD.cc:9701)
  ==00:00:02:54.519 510301==    by 0x70E62E: run (OpSchedulerItem.h:148)
  ==00:00:02:54.519 510301==    by 0x70E62E: OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*) (OSD.cc:10677)
  ==00:00:02:54.519 510301==    by 0xD3C093: ShardedThreadPool::shardedthreadpool_worker(unsigned int) (WorkQueue.cc:311)
  ==00:00:02:54.519 510301==    by 0xD3ECF3: ShardedThreadPool::WorkThreadSharded::entry() (WorkQueue.h:706)
  ==00:00:02:54.519 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:02:54.519 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:02:54.519 510301==  Address 0xf123930 is 3,824 bytes inside a block of size 10,296 alloc'd
  ==00:00:02:54.519 510301==    at 0xA7DC0C3: operator new[](unsigned long) (vg_replace_malloc.c:433)
  ==00:00:02:54.519 510301==    by 0x66F766: main (ceph_osd.cc:688)
  ==00:00:02:54.519 510301==  Block was alloc'd by thread #1
  ```

Actually there is plenty of similar issues reported like:
  ```
  ==00:00:05:04.903 510301== Possible data race during read of size 8 at 0x1E3E0588 by thread ceph#119
  ==00:00:05:04.903 510301== Locks held: 1, at address 0x1EAD41D0
  ==00:00:05:04.903 510301==    at 0x753165: clear (hashtable.h:2051)
  ==00:00:05:04.903 510301==    by 0x753165: std::_Hashtable<entity_addr_t, std::pair<entity_addr_t const, utime_t>, mempool::pool_allocator<(mempool::pool_index_t)15, std::pair<entity_addr_t const, utime_t>
  >, std::__detail::_Select1st, std::equal_to<entity_addr_t>, std::hash<entity_addr_t>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__deta
  il::_Hashtable_traits<true, false, true> >::~_Hashtable() (hashtable.h:1369)
  ==00:00:05:04.903 510301==    by 0x75331C: ~unordered_map (unordered_map.h:102)
  ==00:00:05:04.903 510301==    by 0x75331C: OSDMap::~OSDMap() (OSDMap.h:350)
  ==00:00:05:04.903 510301==    by 0x753606: operator() (shared_cache.hpp:100)
  ==00:00:05:04.903 510301==    by 0x753606: std::_Sp_counted_deleter<OSDMap const*, SharedLRU<unsigned int, OSDMap const>::Cleanup, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr
  _base.h:471)
  ==00:00:05:04.903 510301==    by 0x73BB26: _M_release (shared_ptr_base.h:155)
  ==00:00:05:04.903 510301==    by 0x73BB26: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:148)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~__shared_count (shared_ptr_base.h:728)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~__shared_ptr (shared_ptr_base.h:1167)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~shared_ptr (shared_ptr.h:103)
  ==00:00:05:04.903 510301==    by 0x6B58A9: OSD::create_context() (OSD.cc:9053)
  ==00:00:05:04.903 510301==    by 0x71B570: OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr<PGPeeringEvent>, ThreadPool::TPHandle&) (OSD.cc:9665)
  ==00:00:05:04.903 510301==    by 0x71B997: OSD::dequeue_delete(OSDShard*, PG*, unsigned int, ThreadPool::TPHandle&) (OSD.cc:9701)
  ==00:00:05:04.903 510301==    by 0x70E62E: run (OpSchedulerItem.h:148)
  ==00:00:05:04.903 510301==    by 0x70E62E: OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*) (OSD.cc:10677)
  ==00:00:05:04.903 510301==    by 0xD3C093: ShardedThreadPool::shardedthreadpool_worker(unsigned int) (WorkQueue.cc:311)
  ==00:00:05:04.903 510301==    by 0xD3ECF3: ShardedThreadPool::WorkThreadSharded::entry() (WorkQueue.h:706)
  ==00:00:05:04.903 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:05:04.903 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:05:04.903 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:05:04.903 510301==
  ==00:00:05:04.903 510301== This conflicts with a previous write of size 8 by thread ceph#90
  ==00:00:05:04.903 510301== Locks held: 2, at addresses 0xF122A58 0xF1239A8
  ==00:00:05:04.903 510301==    at 0x7531E1: clear (hashtable.h:2054)
  ==00:00:05:04.903 510301==    by 0x7531E1: std::_Hashtable<entity_addr_t, std::pair<entity_addr_t const, utime_t>, mempool::pool_allocator<(mempool::pool_index_t)15, std::pair<entity_addr_t const, utime_t> >, std::__detail::_Select1st, std::equal_to<entity_addr_t>, std::hash<entity_addr_t>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::~_Hashtable() (hashtable.h:1369)
  ==00:00:05:04.903 510301==    by 0x75331C: ~unordered_map (unordered_map.h:102)
  ==00:00:05:04.903 510301==    by 0x75331C: OSDMap::~OSDMap() (OSDMap.h:350)
  ==00:00:05:04.903 510301==    by 0x753606: operator() (shared_cache.hpp:100)
  ==00:00:05:04.903 510301==    by 0x753606: std::_Sp_counted_deleter<OSDMap const*, SharedLRU<unsigned int, OSDMap const>::Cleanup, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:471)
  ==00:00:05:04.903 510301==    by 0x73BB26: _M_release (shared_ptr_base.h:155)
  ==00:00:05:04.903 510301==    by 0x73BB26: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:148)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr_base.h:747)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr_base.h:1078)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr.h:103)
  ==00:00:05:04.903 510301==    by 0x72191E: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8116)
  ==00:00:05:04.903 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:05:04.903 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:05:04.903 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:05:04.903 510301==  Address 0x1e3e0588 is 872 bytes inside a block of size 1,208 alloc'd
  ==00:00:05:04.903 510301==    at 0xA7DC0C3: operator new[](unsigned long) (vg_replace_malloc.c:433)
  ==00:00:05:04.903 510301==    by 0x6C7C0C: OSDService::try_get_map(unsigned int) (OSD.cc:1606)
  ==00:00:05:04.903 510301==    by 0x7213BD: get_map (OSD.h:699)
  ==00:00:05:04.903 510301==    by 0x7213BD: get_map (OSD.h:1732)
  ==00:00:05:04.903 510301==    by 0x7213BD: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8076)
  ==00:00:05:04.903 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:05:04.903 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:05:04.903 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:05:04.903 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:05:04.903 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:05:04.903 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Feb 12, 2020
Accordingly to cppreference.com [1]:

  "If multiple threads of execution access the same std::shared_ptr
  object without synchronization and any of those accesses uses
  a non-const member function of shared_ptr then a data race will
  occur (...)"

[1]: https://en.cppreference.com/w/cpp/memory/shared_ptr/atomic

One of the coredumps showed the `shared_ptr`-typed `OSD::osdmap`
with healthy looking content but damaged control block:

  ```
  [Current thread is 1 (Thread 0x7f7dcaf73700 (LWP 205295))]
  (gdb) bt
  #0  0x0000559cb81c3ea0 in ?? ()
  #1  0x0000559c97675b27 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  #2  std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  ceph#3  0x0000559c975ef8aa in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1167
  ceph#4  std::__shared_ptr<OSDMap const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1167
  ceph#5  std::shared_ptr<OSDMap const>::~shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr.h:103
  ceph#6  OSD::create_context (this=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9053
  ceph#7  0x0000559c97655571 in OSD::dequeue_peering_evt (this=0x559ca22ac000, sdata=0x559ca2ef2900, pg=0x559cb4aa3400, evt=std::shared_ptr<PGPeeringEvent> (use count 2, weak count 0) = {...}, handle=...)
      at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9665
  ceph#8  0x0000559c97886db6 in ceph::osd::scheduler::PGPeeringItem::run (this=<optimized out>, osd=<optimized out>, sdata=<optimized out>, pg=..., handle=...) at /usr/include/c++/8/ext/atomicity.h:96
  ceph#9  0x0000559c9764862f in ceph::osd::scheduler::OpSchedulerItem::run (handle=..., pg=..., sdata=<optimized out>, osd=<optimized out>, this=0x7f7dcaf703f0) at /usr/include/c++/8/bits/unique_ptr.h:342
  ceph#10 OSD::ShardedOpWQ::_process (this=<optimized out>, thread_index=<optimized out>, hb=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:10677
  ceph#11 0x0000559c97c76094 in ShardedThreadPool::shardedthreadpool_worker (this=0x559ca22aca28, thread_index=14) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/common/WorkQueue.cc:311
  ceph#12 0x0000559c97c78cf4 in ShardedThreadPool::WorkThreadSharded::entry (this=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/common/WorkQueue.h:706
  ceph#13 0x00007f7df17852de in start_thread () from /lib64/libpthread.so.0
  ceph#14 0x00007f7df052f133 in __libc_ifunc_impl_list () from /lib64/libc.so.6
  ceph#15 0x0000000000000000 in ?? ()
  (gdb) frame 7
  ceph#7  0x0000559c97655571 in OSD::dequeue_peering_evt (this=0x559ca22ac000, sdata=0x559ca2ef2900, pg=0x559cb4aa3400, evt=std::shared_ptr<PGPeeringEvent> (use count 2, weak count 0) = {...}, handle=...)
      at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9665
  9665      in /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc
  (gdb) print osdmap
  $24 = std::shared_ptr<const OSDMap> (expired, weak count 0) = {get() = 0x559cba028000}
  (gdb) print *osdmap
     # pretty sane OSDMap
  (gdb) print sizeof(osdmap)
  $26 = 16
  (gdb) x/2a &osdmap
  0x559ca22acef0:   0x559cba028000  0x559cba0ec900

  (gdb) frame 2
  #2  std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  148       /usr/include/c++/8/bits/shared_ptr_base.h: No such file or directory.
  (gdb) disassemble
  Dump of assembler code for function std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release():
  ...
     0x0000559c97675b1e <+62>:      mov    (%rdi),%rax
     0x0000559c97675b21 <+65>:      mov    %rdi,%rbx
     0x0000559c97675b24 <+68>:      callq  *0x10(%rax)
  => 0x0000559c97675b27 <+71>:      test   %rbp,%rbp
  ...
  End of assembler dump.
  (gdb) info registers rdi rbx rax
  rdi            0x559cba0ec900      94131624790272
  rbx            0x559cba0ec900      94131624790272
  rax            0x559cba0ec8a0      94131624790176
  (gdb) x/a 0x559cba0ec8a0 + 0x10
  0x559cba0ec8b0:   0x559cb81c3ea0
  (gdb) bt
  #0  0x0000559cb81c3ea0 in ?? ()
  ...
  (gdb) p $_siginfo._sifields._sigfault.si_addr
  $27 = (void *) 0x559cb81c3ea0
  ```

Helgrind seems to agree:
  ```
  ==00:00:02:54.519 510301== Possible data race during write of size 8 at 0xF123930 by thread ceph#90
  ==00:00:02:54.519 510301== Locks held: 2, at addresses 0xF122A58 0xF1239A8
  ==00:00:02:54.519 510301==    at 0x7218DD: operator= (shared_ptr_base.h:1078)
  ==00:00:02:54.519 510301==    by 0x7218DD: operator= (shared_ptr.h:103)
  ==00:00:02:54.519 510301==    by 0x7218DD: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8116)
  ==00:00:02:54.519 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:02:54.519 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:02:54.519 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:02:54.519 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:02:54.519 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:02:54.519 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:02:54.519 510301==
  ==00:00:02:54.519 510301== This conflicts with a previous read of size 8 by thread ceph#117
  ==00:00:02:54.519 510301== Locks held: 1, at address 0x2123E9A0
  ==00:00:02:54.519 510301==    at 0x6B5842: __shared_ptr (shared_ptr_base.h:1165)
  ==00:00:02:54.519 510301==    by 0x6B5842: shared_ptr (shared_ptr.h:129)
  ==00:00:02:54.519 510301==    by 0x6B5842: get_osdmap (OSD.h:1700)
  ==00:00:02:54.519 510301==    by 0x6B5842: OSD::create_context() (OSD.cc:9053)
  ==00:00:02:54.519 510301==    by 0x71B570: OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr<PGPeeringEvent>, ThreadPool::TPHandle&) (OSD.cc:9665)
  ==00:00:02:54.519 510301==    by 0x71B997: OSD::dequeue_delete(OSDShard*, PG*, unsigned int, ThreadPool::TPHandle&) (OSD.cc:9701)
  ==00:00:02:54.519 510301==    by 0x70E62E: run (OpSchedulerItem.h:148)
  ==00:00:02:54.519 510301==    by 0x70E62E: OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*) (OSD.cc:10677)
  ==00:00:02:54.519 510301==    by 0xD3C093: ShardedThreadPool::shardedthreadpool_worker(unsigned int) (WorkQueue.cc:311)
  ==00:00:02:54.519 510301==    by 0xD3ECF3: ShardedThreadPool::WorkThreadSharded::entry() (WorkQueue.h:706)
  ==00:00:02:54.519 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:02:54.519 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:02:54.519 510301==  Address 0xf123930 is 3,824 bytes inside a block of size 10,296 alloc'd
  ==00:00:02:54.519 510301==    at 0xA7DC0C3: operator new[](unsigned long) (vg_replace_malloc.c:433)
  ==00:00:02:54.519 510301==    by 0x66F766: main (ceph_osd.cc:688)
  ==00:00:02:54.519 510301==  Block was alloc'd by thread #1
  ```

Actually there is plenty of similar issues reported like:
  ```
  ==00:00:05:04.903 510301== Possible data race during read of size 8 at 0x1E3E0588 by thread ceph#119
  ==00:00:05:04.903 510301== Locks held: 1, at address 0x1EAD41D0
  ==00:00:05:04.903 510301==    at 0x753165: clear (hashtable.h:2051)
  ==00:00:05:04.903 510301==    by 0x753165: std::_Hashtable<entity_addr_t, std::pair<entity_addr_t const, utime_t>, mempool::pool_allocator<(mempool::pool_index_t)15, std::pair<entity_addr_t const, utime_t>
  >, std::__detail::_Select1st, std::equal_to<entity_addr_t>, std::hash<entity_addr_t>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__deta
  il::_Hashtable_traits<true, false, true> >::~_Hashtable() (hashtable.h:1369)
  ==00:00:05:04.903 510301==    by 0x75331C: ~unordered_map (unordered_map.h:102)
  ==00:00:05:04.903 510301==    by 0x75331C: OSDMap::~OSDMap() (OSDMap.h:350)
  ==00:00:05:04.903 510301==    by 0x753606: operator() (shared_cache.hpp:100)
  ==00:00:05:04.903 510301==    by 0x753606: std::_Sp_counted_deleter<OSDMap const*, SharedLRU<unsigned int, OSDMap const>::Cleanup, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr
  _base.h:471)
  ==00:00:05:04.903 510301==    by 0x73BB26: _M_release (shared_ptr_base.h:155)
  ==00:00:05:04.903 510301==    by 0x73BB26: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:148)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~__shared_count (shared_ptr_base.h:728)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~__shared_ptr (shared_ptr_base.h:1167)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~shared_ptr (shared_ptr.h:103)
  ==00:00:05:04.903 510301==    by 0x6B58A9: OSD::create_context() (OSD.cc:9053)
  ==00:00:05:04.903 510301==    by 0x71B570: OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr<PGPeeringEvent>, ThreadPool::TPHandle&) (OSD.cc:9665)
  ==00:00:05:04.903 510301==    by 0x71B997: OSD::dequeue_delete(OSDShard*, PG*, unsigned int, ThreadPool::TPHandle&) (OSD.cc:9701)
  ==00:00:05:04.903 510301==    by 0x70E62E: run (OpSchedulerItem.h:148)
  ==00:00:05:04.903 510301==    by 0x70E62E: OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*) (OSD.cc:10677)
  ==00:00:05:04.903 510301==    by 0xD3C093: ShardedThreadPool::shardedthreadpool_worker(unsigned int) (WorkQueue.cc:311)
  ==00:00:05:04.903 510301==    by 0xD3ECF3: ShardedThreadPool::WorkThreadSharded::entry() (WorkQueue.h:706)
  ==00:00:05:04.903 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:05:04.903 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:05:04.903 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:05:04.903 510301==
  ==00:00:05:04.903 510301== This conflicts with a previous write of size 8 by thread ceph#90
  ==00:00:05:04.903 510301== Locks held: 2, at addresses 0xF122A58 0xF1239A8
  ==00:00:05:04.903 510301==    at 0x7531E1: clear (hashtable.h:2054)
  ==00:00:05:04.903 510301==    by 0x7531E1: std::_Hashtable<entity_addr_t, std::pair<entity_addr_t const, utime_t>, mempool::pool_allocator<(mempool::pool_index_t)15, std::pair<entity_addr_t const, utime_t> >, std::__detail::_Select1st, std::equal_to<entity_addr_t>, std::hash<entity_addr_t>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::~_Hashtable() (hashtable.h:1369)
  ==00:00:05:04.903 510301==    by 0x75331C: ~unordered_map (unordered_map.h:102)
  ==00:00:05:04.903 510301==    by 0x75331C: OSDMap::~OSDMap() (OSDMap.h:350)
  ==00:00:05:04.903 510301==    by 0x753606: operator() (shared_cache.hpp:100)
  ==00:00:05:04.903 510301==    by 0x753606: std::_Sp_counted_deleter<OSDMap const*, SharedLRU<unsigned int, OSDMap const>::Cleanup, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:471)
  ==00:00:05:04.903 510301==    by 0x73BB26: _M_release (shared_ptr_base.h:155)
  ==00:00:05:04.903 510301==    by 0x73BB26: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:148)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr_base.h:747)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr_base.h:1078)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr.h:103)
  ==00:00:05:04.903 510301==    by 0x72191E: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8116)
  ==00:00:05:04.903 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:05:04.903 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:05:04.903 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:05:04.903 510301==  Address 0x1e3e0588 is 872 bytes inside a block of size 1,208 alloc'd
  ==00:00:05:04.903 510301==    at 0xA7DC0C3: operator new[](unsigned long) (vg_replace_malloc.c:433)
  ==00:00:05:04.903 510301==    by 0x6C7C0C: OSDService::try_get_map(unsigned int) (OSD.cc:1606)
  ==00:00:05:04.903 510301==    by 0x7213BD: get_map (OSD.h:699)
  ==00:00:05:04.903 510301==    by 0x7213BD: get_map (OSD.h:1732)
  ==00:00:05:04.903 510301==    by 0x7213BD: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8076)
  ==00:00:05:04.903 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:05:04.903 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:05:04.903 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:05:04.903 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:05:04.903 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:05:04.903 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Feb 12, 2020
Helgrind complains about:

  ```
  ==00:00:04:55.495 510301== Possible data race during read of size 8 at 0x1EF1B0B0 by thread ceph#96
  ==00:00:04:55.495 510301== Locks held: 3, at addresses 0xEE8D650 0xEE8D748 0x1FFEFFEE08
  ==00:00:04:55.495 510301==    at 0x107F5C5: ~unique_ptr (unique_ptr.h:273)
  ==00:00:04:55.495 510301==    by 0x107F5C5: ~AuthConnectionMeta (Auth.h:163)
  ==00:00:04:55.495 510301==    by 0x107F5C5: std::_Sp_counted_ptr<AuthConnectionMeta*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:377)
  ==00:00:04:55.495 510301==    by 0x10C19EE: _M_release (shared_ptr_base.h:155)
  ==00:00:04:55.495 510301==    by 0x10C19EE: _M_release (shared_ptr_base.h:148)
  ==00:00:04:55.495 510301==    by 0x10C19EE: ~__shared_count (shared_ptr_base.h:728)
  ==00:00:04:55.495 510301==    by 0x10C19EE: ~__shared_ptr (shared_ptr_base.h:1167)
  ==00:00:04:55.495 510301==    by 0x10C19EE: ~shared_ptr (shared_ptr.h:103)
  ==00:00:04:55.495 510301==    by 0x10C19EE: Protocol::~Protocol() (Protocol.cc:14)
  ==00:00:04:55.495 510301==    by 0x1081A5C: ProtocolV2::~ProtocolV2() (ProtocolV2.cc:100)
  ==00:00:04:55.495 510301==    by 0x105AA48: operator() (unique_ptr.h:81)
  ==00:00:04:55.495 510301==    by 0x105AA48: ~unique_ptr (unique_ptr.h:274)
  ==00:00:04:55.495 510301==    by 0x105AA48: AsyncConnection::~AsyncConnection() (AsyncConnection.cc:149)
  ==00:00:04:55.495 510301==    by 0x105ABAC: AsyncConnection::~AsyncConnection() (AsyncConnection.cc:154)
  ==00:00:04:55.495 510301==    by 0xD2BAA0: RefCountedObject::put() const (RefCountedObj.cc:27)
  ==00:00:04:55.495 510301==    by 0xEB1E58: intrusive_ptr_release (RefCountedObj.h:193)
  ==00:00:04:55.495 510301==    by 0xEB1E58: ~intrusive_ptr (intrusive_ptr.hpp:98)
  ==00:00:04:55.495 510301==    by 0xEB1E58: ~pair (stl_pair.h:208)
  ==00:00:04:55.495 510301==    by 0xEB1E58: destroy<std::pair<const entity_addrvec_t, boost::intrusive_ptr<AsyncConnection> > > (new_allocator.h:140)
  ==00:00:04:55.495 510301==    by 0xEB1E58: destroy<std::pair<const entity_addrvec_t, boost::intrusive_ptr<AsyncConnection> > > (alloc_traits.h:487)
  ==00:00:04:55.495 510301==    by 0xEB1E58: _M_deallocate_node (hashtable_policy.h:2100)
  ==00:00:04:55.495 510301==    by 0xEB1E58: _M_erase (hashtable.h:1909)
  ==00:00:04:55.495 510301==    by 0xEB1E58: std::_Hashtable<entity_addrvec_t, std::pair<entity_addrvec_t const, boost::intrusive_ptr<AsyncConnection> >, std::allocator<std::pair<entity_addrvec_t const, boost
  ::intrusive_ptr<AsyncConnection> > >, std::__detail::_Select1st, std::equal_to<entity_addrvec_t>, std::hash<entity_addrvec_t>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__
  detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::erase(std::__detail::_Node_const_iterator<std::pair<entity_addrvec_t const, boost::intrusive_ptr<AsyncConnection> >, fals
  e, true>) (hashtable.h:1884)
  ==00:00:04:55.495 510301==    by 0xEB24EA: erase (hashtable.h:762)
  ==00:00:04:55.495 510301==    by 0xEB24EA: erase (unordered_map.h:798)
  ==00:00:04:55.495 510301==    by 0xEB24EA: AsyncMessenger::_lookup_conn(entity_addrvec_t const&) (AsyncMessenger.h:312)
  ==00:00:04:55.495 510301==    by 0xEAA08A: AsyncMessenger::connect_to(int, entity_addrvec_t const&, bool, bool) (AsyncMessenger.cc:701)
  ==00:00:04:55.495 510301==    by 0xEF7477: connect_to_mon (Messenger.h:530)
  ==00:00:04:55.495 510301==    by 0xEF7477: MonClient::_add_conn(unsigned int, unsigned long) (MonClient.cc:716)
  ==00:00:04:55.495 510301==    by 0xEF7C30: MonClient::_add_conns(unsigned long) (MonClient.cc:779)
  ==00:00:04:55.495 510301==    by 0xEFDED6: MonClient::_reopen_session(int) (MonClient.cc:691)
  ==00:00:04:55.495 510301==    by 0xF025F2: MonClient::tick() (MonClient.cc:925)
  ==00:00:04:55.495 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:04:55.495 510301==    by 0xD35276: SafeTimer::timer_thread() (Timer.cc:96)
  ==00:00:04:55.495 510301==    by 0xD36850: SafeTimerThread::entry() (Timer.cc:30)
  ==00:00:04:55.495 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:04:55.495 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:04:55.495 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:04:55.495 510301==  Address 0x1ef1b0b0 is 128 bytes inside a block of size 136 alloc'd
  ==00:00:04:55.495 510301==    at 0xA7DC0C3: operator new[](unsigned long) (vg_replace_malloc.c:433)
  ==00:00:04:55.495 510301==    by 0x1082094: ProtocolV2::reset_recv_state()::{lambda()#2}::operator()() const [clone .isra.916] (ProtocolV2.cc:240)
  ==00:00:04:55.495 510301==    by 0x108244B: EventCenter::C_submit_event<ProtocolV2::reset_recv_state()::{lambda()#2}>::do_request(unsigned long) (Event.h:228)
  ==00:00:04:55.495 510301==    by 0xEB7465: EventCenter::process_events(unsigned int, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*) (Event.cc:433)
  ==00:00:04:55.495 510301==    by 0xEBC1BB: operator() (Stack.cc:53)
  ==00:00:04:55.495 510301==    by 0xEBC1BB: std::_Function_handler<void (), NetworkStack::add_thread(unsigned int)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (std_function.h:297)
  ==00:00:04:55.495 510301==    by 0xCF4AA32: ??? (in /usr/lib64/libstdc++.so.6.0.25)
  ==00:00:04:55.495 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:04:55.495 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:04:55.495 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:04:55.495 510301==  Block was alloc'd by thread ceph#3
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Feb 12, 2020
Helgrind complains about:

  ```
  ==00:00:04:55.495 510301== Possible data race during read of size 8 at 0x1EF1B0B0 by thread ceph#96
  ==00:00:04:55.495 510301== Locks held: 3, at addresses 0xEE8D650 0xEE8D748 0x1FFEFFEE08
  ==00:00:04:55.495 510301==    at 0x107F5C5: ~unique_ptr (unique_ptr.h:273)
  ==00:00:04:55.495 510301==    by 0x107F5C5: ~AuthConnectionMeta (Auth.h:163)
  ==00:00:04:55.495 510301==    by 0x107F5C5: std::_Sp_counted_ptr<AuthConnectionMeta*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:377)
  ==00:00:04:55.495 510301==    by 0x10C19EE: _M_release (shared_ptr_base.h:155)
  ==00:00:04:55.495 510301==    by 0x10C19EE: _M_release (shared_ptr_base.h:148)
  ==00:00:04:55.495 510301==    by 0x10C19EE: ~__shared_count (shared_ptr_base.h:728)
  ==00:00:04:55.495 510301==    by 0x10C19EE: ~__shared_ptr (shared_ptr_base.h:1167)
  ==00:00:04:55.495 510301==    by 0x10C19EE: ~shared_ptr (shared_ptr.h:103)
  ==00:00:04:55.495 510301==    by 0x10C19EE: Protocol::~Protocol() (Protocol.cc:14)
  ==00:00:04:55.495 510301==    by 0x1081A5C: ProtocolV2::~ProtocolV2() (ProtocolV2.cc:100)
  ==00:00:04:55.495 510301==    by 0x105AA48: operator() (unique_ptr.h:81)
  ==00:00:04:55.495 510301==    by 0x105AA48: ~unique_ptr (unique_ptr.h:274)
  ==00:00:04:55.495 510301==    by 0x105AA48: AsyncConnection::~AsyncConnection() (AsyncConnection.cc:149)
  ==00:00:04:55.495 510301==    by 0x105ABAC: AsyncConnection::~AsyncConnection() (AsyncConnection.cc:154)
  ==00:00:04:55.495 510301==    by 0xD2BAA0: RefCountedObject::put() const (RefCountedObj.cc:27)
  ==00:00:04:55.495 510301==    by 0xEB1E58: intrusive_ptr_release (RefCountedObj.h:193)
  ==00:00:04:55.495 510301==    by 0xEB1E58: ~intrusive_ptr (intrusive_ptr.hpp:98)
  ==00:00:04:55.495 510301==    by 0xEB1E58: ~pair (stl_pair.h:208)
  ==00:00:04:55.495 510301==    by 0xEB1E58: destroy<std::pair<const entity_addrvec_t, boost::intrusive_ptr<AsyncConnection> > > (new_allocator.h:140)
  ==00:00:04:55.495 510301==    by 0xEB1E58: destroy<std::pair<const entity_addrvec_t, boost::intrusive_ptr<AsyncConnection> > > (alloc_traits.h:487)
  ==00:00:04:55.495 510301==    by 0xEB1E58: _M_deallocate_node (hashtable_policy.h:2100)
  ==00:00:04:55.495 510301==    by 0xEB1E58: _M_erase (hashtable.h:1909)
  ==00:00:04:55.495 510301==    by 0xEB1E58: std::_Hashtable<entity_addrvec_t, std::pair<entity_addrvec_t const, boost::intrusive_ptr<AsyncConnection> >, std::allocator<std::pair<entity_addrvec_t const, boost
  ::intrusive_ptr<AsyncConnection> > >, std::__detail::_Select1st, std::equal_to<entity_addrvec_t>, std::hash<entity_addrvec_t>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__
  detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::erase(std::__detail::_Node_const_iterator<std::pair<entity_addrvec_t const, boost::intrusive_ptr<AsyncConnection> >, fals
  e, true>) (hashtable.h:1884)
  ==00:00:04:55.495 510301==    by 0xEB24EA: erase (hashtable.h:762)
  ==00:00:04:55.495 510301==    by 0xEB24EA: erase (unordered_map.h:798)
  ==00:00:04:55.495 510301==    by 0xEB24EA: AsyncMessenger::_lookup_conn(entity_addrvec_t const&) (AsyncMessenger.h:312)
  ==00:00:04:55.495 510301==    by 0xEAA08A: AsyncMessenger::connect_to(int, entity_addrvec_t const&, bool, bool) (AsyncMessenger.cc:701)
  ==00:00:04:55.495 510301==    by 0xEF7477: connect_to_mon (Messenger.h:530)
  ==00:00:04:55.495 510301==    by 0xEF7477: MonClient::_add_conn(unsigned int, unsigned long) (MonClient.cc:716)
  ==00:00:04:55.495 510301==    by 0xEF7C30: MonClient::_add_conns(unsigned long) (MonClient.cc:779)
  ==00:00:04:55.495 510301==    by 0xEFDED6: MonClient::_reopen_session(int) (MonClient.cc:691)
  ==00:00:04:55.495 510301==    by 0xF025F2: MonClient::tick() (MonClient.cc:925)
  ==00:00:04:55.495 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:04:55.495 510301==    by 0xD35276: SafeTimer::timer_thread() (Timer.cc:96)
  ==00:00:04:55.495 510301==    by 0xD36850: SafeTimerThread::entry() (Timer.cc:30)
  ==00:00:04:55.495 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:04:55.495 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:04:55.495 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:04:55.495 510301==  Address 0x1ef1b0b0 is 128 bytes inside a block of size 136 alloc'd
  ==00:00:04:55.495 510301==    at 0xA7DC0C3: operator new[](unsigned long) (vg_replace_malloc.c:433)
  ==00:00:04:55.495 510301==    by 0x1082094: ProtocolV2::reset_recv_state()::{lambda()#2}::operator()() const [clone .isra.916] (ProtocolV2.cc:240)
  ==00:00:04:55.495 510301==    by 0x108244B: EventCenter::C_submit_event<ProtocolV2::reset_recv_state()::{lambda()#2}>::do_request(unsigned long) (Event.h:228)
  ==00:00:04:55.495 510301==    by 0xEB7465: EventCenter::process_events(unsigned int, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*) (Event.cc:433)
  ==00:00:04:55.495 510301==    by 0xEBC1BB: operator() (Stack.cc:53)
  ==00:00:04:55.495 510301==    by 0xEBC1BB: std::_Function_handler<void (), NetworkStack::add_thread(unsigned int)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (std_function.h:297)
  ==00:00:04:55.495 510301==    by 0xCF4AA32: ??? (in /usr/lib64/libstdc++.so.6.0.25)
  ==00:00:04:55.495 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:04:55.495 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:04:55.495 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:04:55.495 510301==  Block was alloc'd by thread ceph#3
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Feb 14, 2020
Accordingly to cppreference.com [1]:

  "If multiple threads of execution access the same std::shared_ptr
  object without synchronization and any of those accesses uses
  a non-const member function of shared_ptr then a data race will
  occur (...)"

[1]: https://en.cppreference.com/w/cpp/memory/shared_ptr/atomic

One of the coredumps showed the `shared_ptr`-typed `OSD::osdmap`
with healthy looking content but damaged control block:

  ```
  [Current thread is 1 (Thread 0x7f7dcaf73700 (LWP 205295))]
  (gdb) bt
  #0  0x0000559cb81c3ea0 in ?? ()
  #1  0x0000559c97675b27 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  #2  std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  ceph#3  0x0000559c975ef8aa in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1167
  ceph#4  std::__shared_ptr<OSDMap const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1167
  ceph#5  std::shared_ptr<OSDMap const>::~shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr.h:103
  ceph#6  OSD::create_context (this=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9053
  ceph#7  0x0000559c97655571 in OSD::dequeue_peering_evt (this=0x559ca22ac000, sdata=0x559ca2ef2900, pg=0x559cb4aa3400, evt=std::shared_ptr<PGPeeringEvent> (use count 2, weak count 0) = {...}, handle=...)
      at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9665
  ceph#8  0x0000559c97886db6 in ceph::osd::scheduler::PGPeeringItem::run (this=<optimized out>, osd=<optimized out>, sdata=<optimized out>, pg=..., handle=...) at /usr/include/c++/8/ext/atomicity.h:96
  ceph#9  0x0000559c9764862f in ceph::osd::scheduler::OpSchedulerItem::run (handle=..., pg=..., sdata=<optimized out>, osd=<optimized out>, this=0x7f7dcaf703f0) at /usr/include/c++/8/bits/unique_ptr.h:342
  ceph#10 OSD::ShardedOpWQ::_process (this=<optimized out>, thread_index=<optimized out>, hb=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:10677
  ceph#11 0x0000559c97c76094 in ShardedThreadPool::shardedthreadpool_worker (this=0x559ca22aca28, thread_index=14) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/common/WorkQueue.cc:311
  ceph#12 0x0000559c97c78cf4 in ShardedThreadPool::WorkThreadSharded::entry (this=<optimized out>) at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/common/WorkQueue.h:706
  ceph#13 0x00007f7df17852de in start_thread () from /lib64/libpthread.so.0
  ceph#14 0x00007f7df052f133 in __libc_ifunc_impl_list () from /lib64/libc.so.6
  ceph#15 0x0000000000000000 in ?? ()
  (gdb) frame 7
  ceph#7  0x0000559c97655571 in OSD::dequeue_peering_evt (this=0x559ca22ac000, sdata=0x559ca2ef2900, pg=0x559cb4aa3400, evt=std::shared_ptr<PGPeeringEvent> (use count 2, weak count 0) = {...}, handle=...)
      at /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc:9665
  9665      in /usr/src/debug/ceph-15.0.0-10071.g5b5a3a3.el8.x86_64/src/osd/OSD.cc
  (gdb) print osdmap
  $24 = std::shared_ptr<const OSDMap> (expired, weak count 0) = {get() = 0x559cba028000}
  (gdb) print *osdmap
     # pretty sane OSDMap
  (gdb) print sizeof(osdmap)
  $26 = 16
  (gdb) x/2a &osdmap
  0x559ca22acef0:   0x559cba028000  0x559cba0ec900

  (gdb) frame 2
  #2  std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x559cba0ec900) at /usr/include/c++/8/bits/shared_ptr_base.h:148
  148       /usr/include/c++/8/bits/shared_ptr_base.h: No such file or directory.
  (gdb) disassemble
  Dump of assembler code for function std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release():
  ...
     0x0000559c97675b1e <+62>:      mov    (%rdi),%rax
     0x0000559c97675b21 <+65>:      mov    %rdi,%rbx
     0x0000559c97675b24 <+68>:      callq  *0x10(%rax)
  => 0x0000559c97675b27 <+71>:      test   %rbp,%rbp
  ...
  End of assembler dump.
  (gdb) info registers rdi rbx rax
  rdi            0x559cba0ec900      94131624790272
  rbx            0x559cba0ec900      94131624790272
  rax            0x559cba0ec8a0      94131624790176
  (gdb) x/a 0x559cba0ec8a0 + 0x10
  0x559cba0ec8b0:   0x559cb81c3ea0
  (gdb) bt
  #0  0x0000559cb81c3ea0 in ?? ()
  ...
  (gdb) p $_siginfo._sifields._sigfault.si_addr
  $27 = (void *) 0x559cb81c3ea0
  ```

Helgrind seems to agree:
  ```
  ==00:00:02:54.519 510301== Possible data race during write of size 8 at 0xF123930 by thread ceph#90
  ==00:00:02:54.519 510301== Locks held: 2, at addresses 0xF122A58 0xF1239A8
  ==00:00:02:54.519 510301==    at 0x7218DD: operator= (shared_ptr_base.h:1078)
  ==00:00:02:54.519 510301==    by 0x7218DD: operator= (shared_ptr.h:103)
  ==00:00:02:54.519 510301==    by 0x7218DD: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8116)
  ==00:00:02:54.519 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:02:54.519 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:02:54.519 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:02:54.519 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:02:54.519 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:02:54.519 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:02:54.519 510301==
  ==00:00:02:54.519 510301== This conflicts with a previous read of size 8 by thread ceph#117
  ==00:00:02:54.519 510301== Locks held: 1, at address 0x2123E9A0
  ==00:00:02:54.519 510301==    at 0x6B5842: __shared_ptr (shared_ptr_base.h:1165)
  ==00:00:02:54.519 510301==    by 0x6B5842: shared_ptr (shared_ptr.h:129)
  ==00:00:02:54.519 510301==    by 0x6B5842: get_osdmap (OSD.h:1700)
  ==00:00:02:54.519 510301==    by 0x6B5842: OSD::create_context() (OSD.cc:9053)
  ==00:00:02:54.519 510301==    by 0x71B570: OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr<PGPeeringEvent>, ThreadPool::TPHandle&) (OSD.cc:9665)
  ==00:00:02:54.519 510301==    by 0x71B997: OSD::dequeue_delete(OSDShard*, PG*, unsigned int, ThreadPool::TPHandle&) (OSD.cc:9701)
  ==00:00:02:54.519 510301==    by 0x70E62E: run (OpSchedulerItem.h:148)
  ==00:00:02:54.519 510301==    by 0x70E62E: OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*) (OSD.cc:10677)
  ==00:00:02:54.519 510301==    by 0xD3C093: ShardedThreadPool::shardedthreadpool_worker(unsigned int) (WorkQueue.cc:311)
  ==00:00:02:54.519 510301==    by 0xD3ECF3: ShardedThreadPool::WorkThreadSharded::entry() (WorkQueue.h:706)
  ==00:00:02:54.519 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:02:54.519 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:02:54.519 510301==  Address 0xf123930 is 3,824 bytes inside a block of size 10,296 alloc'd
  ==00:00:02:54.519 510301==    at 0xA7DC0C3: operator new[](unsigned long) (vg_replace_malloc.c:433)
  ==00:00:02:54.519 510301==    by 0x66F766: main (ceph_osd.cc:688)
  ==00:00:02:54.519 510301==  Block was alloc'd by thread #1
  ```

Actually there is plenty of similar issues reported like:
  ```
  ==00:00:05:04.903 510301== Possible data race during read of size 8 at 0x1E3E0588 by thread ceph#119
  ==00:00:05:04.903 510301== Locks held: 1, at address 0x1EAD41D0
  ==00:00:05:04.903 510301==    at 0x753165: clear (hashtable.h:2051)
  ==00:00:05:04.903 510301==    by 0x753165: std::_Hashtable<entity_addr_t, std::pair<entity_addr_t const, utime_t>, mempool::pool_allocator<(mempool::pool_index_t)15, std::pair<entity_addr_t const, utime_t>
  >, std::__detail::_Select1st, std::equal_to<entity_addr_t>, std::hash<entity_addr_t>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__deta
  il::_Hashtable_traits<true, false, true> >::~_Hashtable() (hashtable.h:1369)
  ==00:00:05:04.903 510301==    by 0x75331C: ~unordered_map (unordered_map.h:102)
  ==00:00:05:04.903 510301==    by 0x75331C: OSDMap::~OSDMap() (OSDMap.h:350)
  ==00:00:05:04.903 510301==    by 0x753606: operator() (shared_cache.hpp:100)
  ==00:00:05:04.903 510301==    by 0x753606: std::_Sp_counted_deleter<OSDMap const*, SharedLRU<unsigned int, OSDMap const>::Cleanup, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr
  _base.h:471)
  ==00:00:05:04.903 510301==    by 0x73BB26: _M_release (shared_ptr_base.h:155)
  ==00:00:05:04.903 510301==    by 0x73BB26: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:148)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~__shared_count (shared_ptr_base.h:728)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~__shared_ptr (shared_ptr_base.h:1167)
  ==00:00:05:04.903 510301==    by 0x6B58A9: ~shared_ptr (shared_ptr.h:103)
  ==00:00:05:04.903 510301==    by 0x6B58A9: OSD::create_context() (OSD.cc:9053)
  ==00:00:05:04.903 510301==    by 0x71B570: OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr<PGPeeringEvent>, ThreadPool::TPHandle&) (OSD.cc:9665)
  ==00:00:05:04.903 510301==    by 0x71B997: OSD::dequeue_delete(OSDShard*, PG*, unsigned int, ThreadPool::TPHandle&) (OSD.cc:9701)
  ==00:00:05:04.903 510301==    by 0x70E62E: run (OpSchedulerItem.h:148)
  ==00:00:05:04.903 510301==    by 0x70E62E: OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*) (OSD.cc:10677)
  ==00:00:05:04.903 510301==    by 0xD3C093: ShardedThreadPool::shardedthreadpool_worker(unsigned int) (WorkQueue.cc:311)
  ==00:00:05:04.903 510301==    by 0xD3ECF3: ShardedThreadPool::WorkThreadSharded::entry() (WorkQueue.h:706)
  ==00:00:05:04.903 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:05:04.903 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:05:04.903 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ==00:00:05:04.903 510301==
  ==00:00:05:04.903 510301== This conflicts with a previous write of size 8 by thread ceph#90
  ==00:00:05:04.903 510301== Locks held: 2, at addresses 0xF122A58 0xF1239A8
  ==00:00:05:04.903 510301==    at 0x7531E1: clear (hashtable.h:2054)
  ==00:00:05:04.903 510301==    by 0x7531E1: std::_Hashtable<entity_addr_t, std::pair<entity_addr_t const, utime_t>, mempool::pool_allocator<(mempool::pool_index_t)15, std::pair<entity_addr_t const, utime_t> >, std::__detail::_Select1st, std::equal_to<entity_addr_t>, std::hash<entity_addr_t>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::~_Hashtable() (hashtable.h:1369)
  ==00:00:05:04.903 510301==    by 0x75331C: ~unordered_map (unordered_map.h:102)
  ==00:00:05:04.903 510301==    by 0x75331C: OSDMap::~OSDMap() (OSDMap.h:350)
  ==00:00:05:04.903 510301==    by 0x753606: operator() (shared_cache.hpp:100)
  ==00:00:05:04.903 510301==    by 0x753606: std::_Sp_counted_deleter<OSDMap const*, SharedLRU<unsigned int, OSDMap const>::Cleanup, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:471)
  ==00:00:05:04.903 510301==    by 0x73BB26: _M_release (shared_ptr_base.h:155)
  ==00:00:05:04.903 510301==    by 0x73BB26: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:148)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr_base.h:747)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr_base.h:1078)
  ==00:00:05:04.903 510301==    by 0x72191E: operator= (shared_ptr.h:103)
  ==00:00:05:04.903 510301==    by 0x72191E: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8116)
  ==00:00:05:04.903 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:05:04.903 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:05:04.903 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:05:04.903 510301==  Address 0x1e3e0588 is 872 bytes inside a block of size 1,208 alloc'd
  ==00:00:05:04.903 510301==    at 0xA7DC0C3: operator new[](unsigned long) (vg_replace_malloc.c:433)
  ==00:00:05:04.903 510301==    by 0x6C7C0C: OSDService::try_get_map(unsigned int) (OSD.cc:1606)
  ==00:00:05:04.903 510301==    by 0x7213BD: get_map (OSD.h:699)
  ==00:00:05:04.903 510301==    by 0x7213BD: get_map (OSD.h:1732)
  ==00:00:05:04.903 510301==    by 0x7213BD: OSD::_committed_osd_maps(unsigned int, unsigned int, MOSDMap*) (OSD.cc:8076)
  ==00:00:05:04.903 510301==    by 0x7752CA: C_OnMapCommit::finish(int) (OSD.cc:7678)
  ==00:00:05:04.903 510301==    by 0x72A06C: Context::complete(int) (Context.h:77)
  ==00:00:05:04.903 510301==    by 0xD07F14: Finisher::finisher_thread_entry() (Finisher.cc:66)
  ==00:00:05:04.903 510301==    by 0xA7E1203: mythread_wrapper (hg_intercepts.c:389)
  ==00:00:05:04.903 510301==    by 0xC6182DD: start_thread (in /usr/lib64/libpthread-2.28.so)
  ==00:00:05:04.903 510301==    by 0xD8B34B2: clone (in /usr/lib64/libc-2.28.so)
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski pushed a commit that referenced this pull request Mar 27, 2020
* no need to discard_result(). as `output_stream::close()` returns an
  empty future<> already
* free the connected socket after the background task finishes, because:

we should not free the connected socket before the promise referencing it is fulfilled.

otherwise we have error messages from ASan, like

==287182==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000019aa0 at pc 0x55e2ae2de882 bp 0x7fff7e2bf080 sp 0x7fff7e2bf078
READ of size 8 at 0x611000019aa0 thread T0
    #0 0x55e2ae2de881 in seastar::reactor_backend_aio::await_events(int, __sigset_t const*) ../src/seastar/src/core/reactor_backend.cc:396
    #1 0x55e2ae2dfb59 in seastar::reactor_backend_aio::reap_kernel_completions() ../src/seastar/src/core/reactor_backend.cc:428
    #2 0x55e2adbea397 in seastar::reactor::reap_kernel_completions_pollfn::poll() (/var/ssd/ceph/build/bin/crimson-osd+0x155e9397)
    ceph#3 0x55e2adaec6d0 in seastar::reactor::poll_once() ../src/seastar/src/core/reactor.cc:2789
    ceph#4 0x55e2adae7cf7 in operator() ../src/seastar/src/core/reactor.cc:2687
    ceph#5 0x55e2adb7c595 in __invoke_impl<bool, seastar::reactor::run()::<lambda()>&> /usr/include/c++/10/bits/invoke.h:60
    ceph#6 0x55e2adb699b0 in __invoke_r<bool, seastar::reactor::run()::<lambda()>&> /usr/include/c++/10/bits/invoke.h:113
    ceph#7 0x55e2adb50222 in _M_invoke /usr/include/c++/10/bits/std_function.h:291
    ceph#8 0x55e2adc2ba00 in std::function<bool ()>::operator()() const /usr/include/c++/10/bits/std_function.h:622
    ceph#9 0x55e2adaea491 in seastar::reactor::run() ../src/seastar/src/core/reactor.cc:2713
    ceph#10 0x55e2ad98f1c7 in seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) ../src/seastar/src/core/app-template.cc:199
    ceph#11 0x55e2a9e57538 in main ../src/crimson/osd/main.cc:148
    ceph#12 0x7fae7f20de0a in __libc_start_main ../csu/libc-start.c:308
    ceph#13 0x55e2a9d431e9 in _start (/var/ssd/ceph/build/bin/crimson-osd+0x117421e9)

0x611000019aa0 is located 96 bytes inside of 240-byte region [0x611000019a40,0x611000019b30)
freed by thread T0 here:
    #0 0x7fae80a4e487 in operator delete(void*, unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.6+0xac487)
    #1 0x55e2ae302a0a in seastar::aio_pollable_fd_state::~aio_pollable_fd_state() ../src/seastar/src/core/reactor_backend.cc:458
    #2 0x55e2ae2e1059 in seastar::reactor_backend_aio::forget(seastar::pollable_fd_state&) ../src/seastar/src/core/reactor_backend.cc:524
    ceph#3 0x55e2adab9b9a in seastar::pollable_fd_state::forget() ../src/seastar/src/core/reactor.cc:1396
    ceph#4 0x55e2adab9d05 in seastar::intrusive_ptr_release(seastar::pollable_fd_state*) ../src/seastar/src/core/reactor.cc:1401
    ceph#5 0x55e2ace1b72b in boost::intrusive_ptr<seastar::pollable_fd_state>::~intrusive_ptr() /opt/ceph/include/boost/smart_ptr/intrusive_ptr.hpp:98
    ceph#6 0x55e2ace115a5 in seastar::pollable_fd::~pollable_fd() ../src/seastar/include/seastar/core/internal/pollable_fd.hh:109
    ceph#7 0x55e2ae0ed35c in seastar::net::posix_server_socket_impl::~posix_server_socket_impl() ../src/seastar/include/seastar/net/posix-stack.hh:161
    ceph#8 0x55e2ae0ed3cf in seastar::net::posix_server_socket_impl::~posix_server_socket_impl() ../src/seastar/include/seastar/net/posix-stack.hh:161
    ceph#9 0x55e2ae0ed943 in std::default_delete<seastar::net::api_v2::server_socket_impl>::operator()(seastar::net::api_v2::server_socket_impl*) const /usr/include/c++/10/bits/unique_ptr.h:81
    ceph#10 0x55e2ae0db357 in std::unique_ptr<seastar::net::api_v2::server_socket_impl, std::default_delete<seastar::net::api_v2::server_socket_impl> >::~unique_ptr()
	/usr/include/c++/10/bits/unique_ptr.h:357    ceph#11 0x55e2ae1438b7 in seastar::api_v2::server_socket::~server_socket() ../src/seastar/src/net/stack.cc:195
    ceph#12 0x55e2aa1c7656 in std::_Optional_payload_base<seastar::api_v2::server_socket>::_M_destroy() /usr/include/c++/10/optional:260
    ceph#13 0x55e2aa16c84b in std::_Optional_payload_base<seastar::api_v2::server_socket>::_M_reset() /usr/include/c++/10/optional:280
    ceph#14 0x55e2ac24b2b7 in std::_Optional_base_impl<seastar::api_v2::server_socket, std::_Optional_base<seastar::api_v2::server_socket, false, false> >::_M_reset() /usr/include/c++/10/optional:432
    ceph#15 0x55e2ac23f37b in std::optional<seastar::api_v2::server_socket>::reset() /usr/include/c++/10/optional:975
    ceph#16 0x55e2ac21a2e7 in crimson::admin::AdminSocket::stop() ../src/crimson/admin/admin_socket.cc:265
    ceph#17 0x55e2aa099825 in operator() ../src/crimson/osd/osd.cc:450
    ceph#18 0x55e2aa0d4e3e in apply ../src/seastar/include/seastar/core/apply.hh:36

Signed-off-by: Kefu Chai <kchai@redhat.com>
rzarzynski pushed a commit that referenced this pull request Apr 28, 2020
This fixes the the following selinux error when using ceph-iscsi's
rbd-target-api daemon (rbd-target-gw has the same issue). They are
a result of the a python library, rtslib, which the daemons use.

Additional Information:
Source Context                system_u:system_r:ceph_t:s0
Target Context                system_u:object_r:configfs_t:s0
Target Objects
/sys/kernel/config/target/iscsi/iqn.2003-01.com.re
                              dhat:ceph-iscsi/tpgt_1/attrib/authentication
[
                              file ]
Source                        rbd-target-api
Source Path                   /usr/libexec/platform-python3.6
Port                          <Unknown>
Host                          ans8
Source RPM Packages           platform-python-3.6.8-15.1.el8.x86_64
Target RPM Packages
Policy RPM                    selinux-policy-3.14.3-20.el8.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     ans8
Platform                      Linux ans8 4.18.0-147.el8.x86_64 #1 SMP
Thu Sep 26
                              15:52:44 UTC 2019 x86_64 x86_64
Alert Count                   1
First Seen                    2020-01-08 18:39:47 EST
Last Seen                     2020-01-08 18:39:47 EST
Local ID                      6f8c3415-7a50-4dc8-b3d2-2621e1d00ca3

Raw Audit Messages
type=AVC msg=audit(1578526787.577:68): avc:  denied  { ioctl } for
pid=995 comm="rbd-target-api"
path="/sys/kernel/config/target/iscsi/iqn.2003-01.com.redhat:ceph-iscsi/tpgt_1/attrib/authentication"
dev="configfs" ino=25703 ioctlcmd=0x5401
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:configfs_t:s0 tclass=file permissive=1

type=SYSCALL msg=audit(1578526787.577:68): arch=x86_64 syscall=ioctl
success=no exit=ENOTTY a0=34 a1=5401 a2=7ffd4f8f1f60 a3=3052cd2d95839b96
items=0 ppid=1 pid=995 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0
egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=rbd-target-api
exe=/usr/libexec/platform-python3.6 subj=system_u:system_r:ceph_t:s0
key=(null)

Hash: rbd-target-api,ceph_t,configfs_t,file,ioctl

Signed-off-by: Mike Christie <mchristi@redhat.com>
rzarzynski pushed a commit that referenced this pull request Apr 28, 2020
This fixes the selinux errors like this for /etc/target

-----------------------------------
Additional Information:
Source Context                system_u:system_r:ceph_t:s0
Target Context                system_u:object_r:targetd_etc_rw_t:s0
Target Objects                target [ dir ]
Source                        rbd-target-api
Source Path                   rbd-target-api
Port                          <Unknown>
Host                          ans8
Source RPM Packages
Target RPM Packages
Policy RPM                    selinux-policy-3.14.3-20.el8.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     ans8
Platform                      Linux ans8 4.18.0-147.el8.x86_64 #1 SMP
Thu Sep 26
                              15:52:44 UTC 2019 x86_64 x86_64
Alert Count                   1
First Seen                    2020-01-08 18:39:48 EST
Last Seen                     2020-01-08 18:39:48 EST
Local ID                      9a13ee18-eaf2-4f2a-872f-2809ee4928f6

Raw Audit Messages
type=AVC msg=audit(1578526788.148:69): avc:  denied  { search } for
pid=995 comm="rbd-target-api" name="target" dev="sda1" ino=52198
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:targetd_etc_rw_t:s0 tclass=dir permissive=1

Hash: rbd-target-api,ceph_t,targetd_etc_rw_t,dir,search

which are a result of the rtslib library the ceph-iscsi daemons use
accessing /etc/target to read/write a file which stores meta data the
target uses.

Signed-off-by: Mike Christie <mchristi@redhat.com>
rzarzynski pushed a commit that referenced this pull request Sep 11, 2020
Changes addressing comments in PR - commit to be
squashed prior to merge

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
rzarzynski pushed a commit that referenced this pull request May 7, 2021
Otherwise, if we assert, we'll hang here:

Thread 1 (Thread 0x7f74eba79580 (LWP 1688617)):
#0  0x00007f74eb2aa529 in futex_wait (private=<optimized out>, expected=132, futex_word=0x7ffd642b4b54) at ../sysdeps/unix/sysv/linux/futex-internal.h:61
#1  futex_wait_simple (private=<optimized out>, expected=132, futex_word=0x7ffd642b4b54) at ../sysdeps/nptl/futex-internal.h:135
#2  __pthread_cond_destroy (cond=0x7ffd642b4b30) at pthread_cond_destroy.c:54

ceph#3  0x0000563ff2e5a891 in LibRadosService_StatusFormat_Test::TestBody (this=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:78
ceph#4  0x0000563ff2e9dc3a in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (location=0x563ff2ea72e4 "the test body", method=<optimized out>, object=0x563ff422a6d0)
    at ./src/googletest/googletest/src/gtest.cc:2605
ceph#5  testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=object@entry=0x563ff422a6d0, method=<optimized out>, location=location@entry=0x563ff2ea72e4 "the test body")
    at ./src/googletest/googletest/src/gtest.cc:2641
ceph#6  0x0000563ff2e908c3 in testing::Test::Run (this=0x563ff422a6d0) at ./src/googletest/googletest/src/gtest.cc:2680
ceph#7  0x0000563ff2e90a25 in testing::TestInfo::Run (this=0x563ff41a3b70) at ./src/googletest/googletest/src/gtest.cc:2858
ceph#8  0x0000563ff2e90ec1 in testing::TestSuite::Run (this=0x563ff41b6230) at ./src/googletest/googletest/src/gtest.cc:3012
ceph#9  0x0000563ff2e92bdc in testing::internal::UnitTestImpl::RunAllTests (this=<optimized out>) at ./src/googletest/googletest/src/gtest.cc:5723
ceph#10 0x0000563ff2e9e14a in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (location=0x563ff2ea8728 "auxiliary test code (environments or event listeners)",
    method=<optimized out>, object=0x563ff41a2d10) at ./src/googletest/googletest/src/gtest.cc:2605
ceph#11 testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x563ff41a2d10, method=<optimized out>,
    location=location@entry=0x563ff2ea8728 "auxiliary test code (environments or event listeners)") at ./src/googletest/googletest/src/gtest.cc:2641
ceph#12 0x0000563ff2e90ae8 in testing::UnitTest::Run (this=0x563ff30c0660 <testing::UnitTest::GetInstance()::instance>) at ./src/googletest/googletest/src/gtest.cc:5306

Signed-off-by: Sage Weil <sage@newdream.net>
rzarzynski added a commit that referenced this pull request May 7, 2021
An attempt to `Connection::do_auth()` may finish in one of three states:
_success_, _failure_ and _cancelation_. Unfortunately, its callers were
missing the third treating cancelation like a failure. This was the root
cause of the following failure at Sepia:

```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-05-06_22:08:43-rados-master-distro-basic-smithi/6102605$ less ./remote/smithi204/log/ceph-osd.3.log.gz
...
WARN  2021-05-06 22:35:40,464 [shard 0] osd - ms_handle_reset
...
INFO  2021-05-06 22:35:40,465 [shard 0] monc - do_auth_single: connection closed
INFO  2021-05-06 22:35:40,465 [shard 0] ms - [osd.3(client) v2:172.21.15.204:6808/31418@57568 >> mon.? v2:172.21.15.204:3300/0] execute_connecting(): protocol aborted at CLOSING -- std::system_error (error crimson::net:6, protocol aborted)
...
ERROR 2021-05-06 22:35:40,465 [shard 0] osd - mon.osd.3 dispatch() ms_handle_reset caught exception: std::system_error (error crimson::net:3, negotiation failure)
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3909-g81233a18/rpm/el8/BUILD/ceph-17.0.0-3909-g81233a18/src/crimson/common/gated.h:36: crimson::common::Gated::dispatch(const char*, T&, Func&&) [with Func = crimson::mon::Client::ms_handle_reset(crimson::net::ConnectionRef, bool)::<lambda()>&; T = crimson::mon::Client]::<lambda(std::__exception_ptr::exception_ptr)>: Assertion `*eptr.__cxa_exception_type() == typeid(seastar::gate_closed_exception)' failed.
Aborting on shard 0.
Backtrace:
 0# 0x00005618C973932F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F7BB592EB20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007F7BB3F29B09 in /lib64/libc.so.6
 7# 0x00007F7BB3F37DE6 in /lib64/libc.so.6
 8# 0x00005618C9FF295C in ceph-osd
 9# 0x00005618C3907313 in ceph-osd
10# 0x00005618CCA2F84F in ceph-osd
11# 0x00005618CCA34D90 in ceph-osd
12# 0x00005618CCBEC9BB in ceph-osd
13# 0x00005618CC744E9A in ceph-osd
14# main in ceph-osd
15# __libc_start_main in /lib64/libc.so.6
16# _start in ceph-osd
daemon-helper: command crashed with signal 6
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 7, 2021
An attempt to `Connection::do_auth()` may finish in one of three states:
_success_, _failure_ and _cancellation_. Unfortunately, its callers were
missing the third treating cancellation like a failure. This was the root
cause of the following failure at Sepia:

```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-05-06_22:08:43-rados-master-distro-basic-smithi/6102605$ less ./remote/smithi204/log/ceph-osd.3.log.gz
...
WARN  2021-05-06 22:35:40,464 [shard 0] osd - ms_handle_reset
...
INFO  2021-05-06 22:35:40,465 [shard 0] monc - do_auth_single: connection closed
INFO  2021-05-06 22:35:40,465 [shard 0] ms - [osd.3(client) v2:172.21.15.204:6808/31418@57568 >> mon.? v2:172.21.15.204:3300/0] execute_connecting(): protocol aborted at CLOSING -- std::system_error (error crimson::net:6, protocol aborted)
...
ERROR 2021-05-06 22:35:40,465 [shard 0] osd - mon.osd.3 dispatch() ms_handle_reset caught exception: std::system_error (error crimson::net:3, negotiation failure)
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3909-g81233a18/rpm/el8/BUILD/ceph-17.0.0-3909-g81233a18/src/crimson/common/gated.h:36: crimson::common::Gated::dispatch(const char*, T&, Func&&) [with Func = crimson::mon::Client::ms_handle_reset(crimson::net::ConnectionRef, bool)::<lambda()>&; T = crimson::mon::Client]::<lambda(std::__exception_ptr::exception_ptr)>: Assertion `*eptr.__cxa_exception_type() == typeid(seastar::gate_closed_exception)' failed.
Aborting on shard 0.
Backtrace:
 0# 0x00005618C973932F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F7BB592EB20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007F7BB3F29B09 in /lib64/libc.so.6
 7# 0x00007F7BB3F37DE6 in /lib64/libc.so.6
 8# 0x00005618C9FF295C in ceph-osd
 9# 0x00005618C3907313 in ceph-osd
10# 0x00005618CCA2F84F in ceph-osd
11# 0x00005618CCA34D90 in ceph-osd
12# 0x00005618CCBEC9BB in ceph-osd
13# 0x00005618CC744E9A in ceph-osd
14# main in ceph-osd
15# __libc_start_main in /lib64/libc.so.6
16# _start in ceph-osd
daemon-helper: command crashed with signal 6
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 7, 2021
…dling.

In `crimson/osd/main.cc` we instruct Seastar to handle `SIGHUP`.

```
        // just ignore SIGHUP, we don't reread settings
        seastar::engine().handle_signal(SIGHUP, [] {})
```

This happens using the Seastar's signal handling infrastructure
which is incompliant with the alien world.

```
void
reactor::signals::handle_signal(int signo, noncopyable_function<void ()>&& handler) {
    // ...
    struct sigaction sa;
    sa.sa_sigaction = [](int sig, siginfo_t *info, void *p) {
        engine()._backend->signal_received(sig, info, p);
    };
    // ...
}
```

```
 extern __thread reactor* local_engine;
extern __thread size_t task_quota;

inline reactor& engine() {
    return *local_engine;
}
```

The low-level signal handler above assumes `local_engine._backend`
is not null which stays true only for threads from the S*'s world.
Unfortunately, as we don't block the `SIGHUP` for alien threads,
kernel is perfectly authorized to pick up one them to run the handler
leading to weirdly-looking segfaults like this one:

```
INFO  2021-04-23 07:06:57,807 [shard 0] bluestore - stat
DEBUG 2021-04-23 07:06:58,753 [shard 0] ms - [osd.1(client) v2:172.21.15.100:6802/30478@51064 >> mgr.4105 v2:172.21.15.109:6800/29891] --> ceph#7 === pg_stats(0 pgs seq 55834574872 v 0) v2 (87)
...
INFO  2021-04-23 07:06:58,813 [shard 0] bluestore - stat
DEBUG 2021-04-23 07:06:59,753 [shard 0] osd - AdminSocket::handle_client: incoming asok string: {"prefix": "get_command_descriptions"}
INFO  2021-04-23 07:06:59,753 [shard 0] osd - asok response length: 2947
INFO  2021-04-23 07:06:59,817 [shard 0] bluestore - stat
DEBUG 2021-04-23 07:06:59,865 [shard 0] osd - AdminSocket::handle_client: incoming asok string: {"prefix": "get_command_descriptions"}
INFO  2021-04-23 07:06:59,866 [shard 0] osd - asok response length: 2947
DEBUG 2021-04-23 07:07:00,020 [shard 0] osd - AdminSocket::handle_client: incoming asok string: {"prefix": "get_command_descriptions"}
INFO  2021-04-23 07:07:00,020 [shard 0] osd - asok response length: 2947
INFO  2021-04-23 07:07:00,820 [shard 0] bluestore - stat
...
Backtrace:
 0# 0x00005600CD0D6AAF in ceph-osd
 1# FatalSignal::signaled(int) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F5877C7EB20 in /lib64/libpthread.so.0
 4# 0x00005600CD830B81 in ceph-osd
 5# 0x00007F5877C7EB20 in /lib64/libpthread.so.0
 6# pthread_cond_timedwait in /lib64/libpthread.so.0
 7# crimson::os::ThreadPool::loop(std::chrono::duration<long, std::ratio<1l, 1000l> >, unsigned long) in ceph-osd
 8# 0x00007F5877999BA3 in /lib64/libstdc++.so.6
 9# 0x00007F5877C7414A in /lib64/libpthread.so.0
10# clone in /lib64/libc.so.6
daemon-helper: command crashed with signal 11
```

Ultimately, it turned out the thread came out from a syscall (`futex`)
and started crunching the `SIGHUP` handler's code in which a nullptr
dereference happened.

This patch blocks `SIGHUP` for all threads spawned by `AlienStore`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 7, 2021
An attempt to `Connection::do_auth()` may finish in one of three states:
_success_, _failure_ and _cancellation_. Unfortunately, its callers were
missing the third treating cancellation like a failure. This was the root
cause of the following failure at Sepia:

```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-05-06_22:08:43-rados-master-distro-basic-smithi/6102605$ less ./remote/smithi204/log/ceph-osd.3.log.gz
...
WARN  2021-05-06 22:35:40,464 [shard 0] osd - ms_handle_reset
...
INFO  2021-05-06 22:35:40,465 [shard 0] monc - do_auth_single: connection closed
INFO  2021-05-06 22:35:40,465 [shard 0] ms - [osd.3(client) v2:172.21.15.204:6808/31418@57568 >> mon.? v2:172.21.15.204:3300/0] execute_connecting(): protocol aborted at CLOSING -- std::system_error (error crimson::net:6, protocol aborted)
...
ERROR 2021-05-06 22:35:40,465 [shard 0] osd - mon.osd.3 dispatch() ms_handle_reset caught exception: std::system_error (error crimson::net:3, negotiation failure)
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3909-g81233a18/rpm/el8/BUILD/ceph-17.0.0-3909-g81233a18/src/crimson/common/gated.h:36: crimson::common::Gated::dispatch(const char*, T&, Func&&) [with Func = crimson::mon::Client::ms_handle_reset(crimson::net::ConnectionRef, bool)::<lambda()>&; T = crimson::mon::Client]::<lambda(std::__exception_ptr::exception_ptr)>: Assertion `*eptr.__cxa_exception_type() == typeid(seastar::gate_closed_exception)' failed.
Aborting on shard 0.
Backtrace:
 0# 0x00005618C973932F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F7BB592EB20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007F7BB3F29B09 in /lib64/libc.so.6
 7# 0x00007F7BB3F37DE6 in /lib64/libc.so.6
 8# 0x00005618C9FF295C in ceph-osd
 9# 0x00005618C3907313 in ceph-osd
10# 0x00005618CCA2F84F in ceph-osd
11# 0x00005618CCA34D90 in ceph-osd
12# 0x00005618CCBEC9BB in ceph-osd
13# 0x00005618CC744E9A in ceph-osd
14# main in ceph-osd
15# __libc_start_main in /lib64/libc.so.6
16# _start in ceph-osd
daemon-helper: command crashed with signal 6
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 17, 2021
The `send_message()` method is a high-level facility for
communicating with a monitor. If there is an active conn
available, it sends the message immediatelly; otherwise
the message is queued. This method assumes the queue is
already drained if the connection is available.

`active_con` is managed by `reopen_session()` where it's
first cleared and then reset after finding new alive mon.
This is followed by draining the `pending_messages` queue
which happens in `on_session_opened()` after the `MAuth`
exchange is finished.

Unfortunately, the path from the `active_con` clearing
to draining the queue is long and divided into multiple
continuations which results in lack of atomicity. When
e.g. `run_command()` interleaves the stages, following
crash happens:

```
INFO  2021-05-07 08:13:43,914 [shard 0] monc - do_auth_single: mon v2:172.21.15.82:6805/34166 => v2:172.21.15.82:3300/0 returns auth_reply(proto 2 0 (0) Success) v1: 0
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3910-g1b18e076/rpm/el8/BUILD/ceph-17.0.0-3910-g1b18e076/src/crimson/mon/MonClient.cc:1034: seastar::future<> crimson::mon::Client::send_message(MessageRef): Assertion `pending_messages.empty()' failed.
Aborting on shard 0.
Backtrace:
 0# 0x000055CDE6DB532F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007FC1BF20BB20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007FC1BD806B09 in /lib64/libc.so.6
 7# 0x00007FC1BD814DE6 in /lib64/libc.so.6
 8# crimson::mon::Client::send_message(boost::intrusive_ptr<Message>) in ceph-osd
 9# crimson::mon::Client::renew_subs() in ceph-osd
10# 0x000055CDE764FB0B in ceph-osd
11# 0x000055CDE10457F0 in ceph-osd
12# 0x000055CDEA0AB88F in ceph-osd
13# 0x000055CDEA0B0DD0 in ceph-osd
14# 0x000055CDEA2689FB in ceph-osd
15# 0x000055CDE9DC0EDA in ceph-osd
16# main in ceph-osd
17# __libc_start_main in /lib64/libc.so.6
18# _start in ceph-osd
```

The problem caused following failure at Sepia:
http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-07_07:41:02-rados-master-distro-basic-smithi/6104549

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 17, 2021
The `send_message()` method is a high-level facility for
communicating with a monitor. If there is an active conn
available, it sends the message immediately; otherwise
the message is queued. This method assumes the queue is
already drained if the connection is available.

`active_con` is managed by `reopen_session()` where it's
first cleared and then reset after finding new alive mon.
This is followed by draining the `pending_messages` queue
which happens in `on_session_opened()` after the `MAuth`
exchange is finished.

Unfortunately, the path from the `active_con` clearing
to draining the queue is long and divided into multiple
continuations which results in lack of atomicity. When
e.g. `run_command()` interleaves the stages, following
crash happens:

```
INFO  2021-05-07 08:13:43,914 [shard 0] monc - do_auth_single: mon v2:172.21.15.82:6805/34166 => v2:172.21.15.82:3300/0 returns auth_reply(proto 2 0 (0) Success) v1: 0
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3910-g1b18e076/rpm/el8/BUILD/ceph-17.0.0-3910-g1b18e076/src/crimson/mon/MonClient.cc:1034: seastar::future<> crimson::mon::Client::send_message(MessageRef): Assertion `pending_messages.empty()' failed.
Aborting on shard 0.
Backtrace:
 0# 0x000055CDE6DB532F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007FC1BF20BB20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007FC1BD806B09 in /lib64/libc.so.6
 7# 0x00007FC1BD814DE6 in /lib64/libc.so.6
 8# crimson::mon::Client::send_message(boost::intrusive_ptr<Message>) in ceph-osd
 9# crimson::mon::Client::renew_subs() in ceph-osd
10# 0x000055CDE764FB0B in ceph-osd
11# 0x000055CDE10457F0 in ceph-osd
12# 0x000055CDEA0AB88F in ceph-osd
13# 0x000055CDEA0B0DD0 in ceph-osd
14# 0x000055CDEA2689FB in ceph-osd
15# 0x000055CDE9DC0EDA in ceph-osd
16# main in ceph-osd
17# __libc_start_main in /lib64/libc.so.6
18# _start in ceph-osd
```

The problem caused following failure at Sepia:
http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-07_07:41:02-rados-master-distro-basic-smithi/6104549

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 24, 2021
`OpSequencer` assumes that ID of a previous client request
is always lower than ID of current one. This is reflected
by the assertion in `OpSequencer::start_op()`. It trigerred
the following failure [1] in Teuthology:

```
DEBUG 2021-05-07 08:01:41,227 [shard 0] osd - client_request(id=1, detail=osd_op(client.4171.0:1 2.2 2.7c339972 (undecoded) ondisk+retry+read+known_if_redirected e29) v8) same_interval_since: 31
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3910-g1b18e076/rpm/el8/BUILD/ceph-
17.0.0-3910-g1b18e076/src/crimson/osd/osd_operation_sequencer.h:38: seastar::futurize_t<Result> crimson::osd::OpSequencer::start_op(HandleT&, uint64_t, uint64_t, FuncT&&) [with HandleT = crimson::PipelineHa
ndle; FuncT = crimson::interruptible::interruptor<InterruptCond>::wrap_function(Func&&) [with Func = crimson::osd::ClientRequest::start()::<lambda()> mutable::<lambda(Ref<crimson::osd::PG>)> mutable::<lambd
a()> mutable::<lambda()>; InterruptCond = crimson::osd::IOInterruptCondition]::<lambda()>; Result = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>
>; seastar::futurize_t<Result> = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<> >; uint64_t = long unsigned int]: Assertion `prev_op < this_op' fai
led.
Aborting on shard 0.
Backtrace:
Segmentation fault.
Backtrace:
 0# 0x00005592B028932F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F57B72E7B20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007F57B58E2B09 in /lib64/libc.so.6
 7# 0x00007F57B58F0DE6 in /lib64/libc.so.6
 8# 0x00005592ABB8484D in ceph-osd
 9# 0x00005592ABB8ACB3 in ceph-osd
10# seastar::continuation<seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<boost::intrusive_ptr<crimson::osd::PG> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > >(seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >&&, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&, seastar::future_state<boost::intrusive_ptr<crimson::osd::PG> >&&)#1}, boost::intrusive_ptr<crimson::osd::PG> >::run_and_dispose() in ceph-osd
11# 0x00005592B357F88F in ceph-osd
12# 0x00005592B3584DD0 in ceph-osd
```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-07_07:41:02-rados-master-distro-basic-smithi/6104530

Crash analysis resulted in two observations:
1. during the request execution the acting set got
   changed, the request was interrupted and a try
   to re-execute it emerged;
2. the interrupted request was the very first client
   request the OSD has ever seen.

Code analysis showed a problem in how `ClientRequest`
establishes `prev_op_id`: although supposed to be performed
only once for a request, it can get executed twice but only
for the very first request `OpSequencer` saw.

```cpp
void ClientRequest::may_set_prev_op()
{
  // set prev_op_id if it's not set yet
  if (__builtin_expect(prev_op_id == 0, true)) {
    prev_op_id = sequencer.get_last_issued();
  }
}
```

Unfortunately, `0` isn't a distincted value that cannot
be returned by `get_last_issued()`:

```cpp
class OpSequencer {
  // ...

  uint64_t get_last_issued() const {
    return last_issued;
  }

  // ...

  // the id of last op which is issued
  uint64_t last_issued = 0;
```

As a result, `OpSequencer` returnted on the second call
a new value (actually `this_op`) violating the assertion.
The commit fixes the problem by switching from a designated
value to `std::optional`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 24, 2021
`OpSequencer` assumes that ID of a previous client request
is always lower than ID of current one. This is reflected
by the assertion in `OpSequencer::start_op()`. It triggered
the following failure [1] in Teuthology:

```
DEBUG 2021-05-07 08:01:41,227 [shard 0] osd - client_request(id=1, detail=osd_op(client.4171.0:1 2.2 2.7c339972 (undecoded) ondisk+retry+read+known_if_redirected e29) v8) same_interval_since: 31
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3910-g1b18e076/rpm/el8/BUILD/ceph-
17.0.0-3910-g1b18e076/src/crimson/osd/osd_operation_sequencer.h:38: seastar::futurize_t<Result> crimson::osd::OpSequencer::start_op(HandleT&, uint64_t, uint64_t, FuncT&&) [with HandleT = crimson::PipelineHa
ndle; FuncT = crimson::interruptible::interruptor<InterruptCond>::wrap_function(Func&&) [with Func = crimson::osd::ClientRequest::start()::<lambda()> mutable::<lambda(Ref<crimson::osd::PG>)> mutable::<lambd
a()> mutable::<lambda()>; InterruptCond = crimson::osd::IOInterruptCondition]::<lambda()>; Result = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>
>; seastar::futurize_t<Result> = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<> >; uint64_t = long unsigned int]: Assertion `prev_op < this_op' fai
led.
Aborting on shard 0.
Backtrace:
Segmentation fault.
Backtrace:
 0# 0x00005592B028932F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F57B72E7B20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007F57B58E2B09 in /lib64/libc.so.6
 7# 0x00007F57B58F0DE6 in /lib64/libc.so.6
 8# 0x00005592ABB8484D in ceph-osd
 9# 0x00005592ABB8ACB3 in ceph-osd
10# seastar::continuation<seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<boost::intrusive_ptr<crimson::osd::PG> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > >(seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >&&, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&, seastar::future_state<boost::intrusive_ptr<crimson::osd::PG> >&&)#1}, boost::intrusive_ptr<crimson::osd::PG> >::run_and_dispose() in ceph-osd
11# 0x00005592B357F88F in ceph-osd
12# 0x00005592B3584DD0 in ceph-osd
```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-07_07:41:02-rados-master-distro-basic-smithi/6104530

Crash analysis resulted in two observations:
1. during the request execution the acting set got
   changed, the request was interrupted and a try
   to re-execute it emerged;
2. the interrupted request was the very first client
   request the OSD has ever seen.

Code analysis showed a problem in how `ClientRequest`
establishes `prev_op_id`: although supposed to be performed
only once for a request, it can get executed twice but only
for the very first request `OpSequencer` saw.

```cpp
void ClientRequest::may_set_prev_op()
{
  // set prev_op_id if it's not set yet
  if (__builtin_expect(prev_op_id == 0, true)) {
    prev_op_id = sequencer.get_last_issued();
  }
}
```

Unfortunately, `0` isn't a distincted value that cannot
be returned by `get_last_issued()`:

```cpp
class OpSequencer {
  // ...

  uint64_t get_last_issued() const {
    return last_issued;
  }

  // ...

  // the id of last op which is issued
  uint64_t last_issued = 0;
```

As a result, `OpSequencer` returned on the second call
a new value (actually `this_op`) violating the assertion.
The commit fixes the problem by switching from a designated
value to `std::optional`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 24, 2021
`OpSequencer` assumes that ID of a previous client request
is always lower than ID of current one. This is reflected
by the assertion in `OpSequencer::start_op()`. It triggered
the following failure [1] in Teuthology:

```
DEBUG 2021-05-07 08:01:41,227 [shard 0] osd - client_request(id=1, detail=osd_op(client.4171.0:1 2.2 2.7c339972 (undecoded) ondisk+retry+read+known_if_redirected e29) v8) same_interval_since: 31
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3910-g1b18e076/rpm/el8/BUILD/ceph-
17.0.0-3910-g1b18e076/src/crimson/osd/osd_operation_sequencer.h:38: seastar::futurize_t<Result> crimson::osd::OpSequencer::start_op(HandleT&, uint64_t, uint64_t, FuncT&&) [with HandleT = crimson::PipelineHa
ndle; FuncT = crimson::interruptible::interruptor<InterruptCond>::wrap_function(Func&&) [with Func = crimson::osd::ClientRequest::start()::<lambda()> mutable::<lambda(Ref<crimson::osd::PG>)> mutable::<lambd
a()> mutable::<lambda()>; InterruptCond = crimson::osd::IOInterruptCondition]::<lambda()>; Result = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>
>; seastar::futurize_t<Result> = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<> >; uint64_t = long unsigned int]: Assertion `prev_op < this_op' fai
led.
Aborting on shard 0.
Backtrace:
Segmentation fault.
Backtrace:
 0# 0x00005592B028932F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F57B72E7B20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007F57B58E2B09 in /lib64/libc.so.6
 7# 0x00007F57B58F0DE6 in /lib64/libc.so.6
 8# 0x00005592ABB8484D in ceph-osd
 9# 0x00005592ABB8ACB3 in ceph-osd
10# seastar::continuation<seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<boost::intrusive_ptr<crimson::osd::PG> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > >(seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >&&, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&, seastar::future_state<boost::intrusive_ptr<crimson::osd::PG> >&&)#1}, boost::intrusive_ptr<crimson::osd::PG> >::run_and_dispose() in ceph-osd
11# 0x00005592B357F88F in ceph-osd
12# 0x00005592B3584DD0 in ceph-osd
```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-07_07:41:02-rados-master-distro-basic-smithi/6104530

Crash analysis resulted in two observations:
1. during the request execution the acting set got
   changed, the request was interrupted and a try
   to re-execute it emerged;
2. the interrupted request was the very first client
   request the OSD has ever seen.

Code analysis showed a problem in how `ClientRequest`
establishes `prev_op_id`: although supposed to be performed
only once for a request, it can get executed twice but only
for the very first request `OpSequencer` saw.

```cpp
void ClientRequest::may_set_prev_op()
{
  // set prev_op_id if it's not set yet
  if (__builtin_expect(prev_op_id == 0, true)) {
    prev_op_id = sequencer.get_last_issued();
  }
}
```

Unfortunately, `0` isn't a distincted value that cannot
be returned by `get_last_issued()`:

```cpp
class OpSequencer {
  // ...

  uint64_t get_last_issued() const {
    return last_issued;
  }

  // ...

  // the id of last op which is issued
  uint64_t last_issued = 0;
```

As a result, `OpSequencer` returned on the second call
a new value (actually `this_op`) violating the assertion.
The commit fixes the problem by switching from a designated
value to `std::optional`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 25, 2021
f7181ab has optimized the client
parallelism. To achieve that `PG::do_osd_ops()` were converted to
return basically future pair of futures. Unfortunately, the life-
time management of `OpsExecuter` was kept intact. In the result,
the object was valid only till fullfying the outer future while,
due to the `rollbacker` instances, it should be available till
`all_completed` becomes available.

This issue can explain the following problem has been observed
in a Teuthology job [1].

```
DEBUG 2021-05-20 08:03:22,617 [shard 0] osd - do_op_call: method returned ret=-17, outdata.length()=0 while num_read=1, num_write=0
DEBUG 2021-05-20 08:03:22,617 [shard 0] osd - rollback_obc_if_modified: object 19:e17d4708:test-rados-api-smithi095-38404-2::foo:head got erro
r generic:17, need_rollback=false
=================================================================
==33626==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0000b9320 at pc 0x560f486b8222 bp 0x7fffc467a1e0 sp 0x7fffc467a1d0
READ of size 4 at 0x60d0000b9320 thread T0
    #0 0x560f486b8221  (/usr/bin/ceph-osd+0x2c610221)
    #1 0x560f4880c6b1 in seastar::continuation<seastar::internal::promise_base_with_type<boost::intrusive_ptr<MOSDOpReply> >, seastar::noncopy
able_function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_
wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > >
 > ()>, seastar::future<void>::then_impl_nrvo<seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::osd::
IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson:
:errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInte
rruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::error
ated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > >(seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail
<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_
future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>&&)::{lambda(seastar::internal::promise_base_with_type<boos
t::intrusive_ptr<MOSDOpReply> >&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>&, seastar::future_state<seastar::internal::monostate>&&)#1}, void>::run_and_dispose() (/usr/bin/ceph-osd+0x2c7646b1)
    #2 0x560f5352c3ae  (/usr/bin/ceph-osd+0x374843ae)
    ceph#3 0x560f535318ef  (/usr/bin/ceph-osd+0x374898ef)
    ceph#4 0x560f536e395a  (/usr/bin/ceph-osd+0x3763b95a)
    ceph#5 0x560f532413d9  (/usr/bin/ceph-osd+0x371993d9)
    ceph#6 0x560f476af95a in main (/usr/bin/ceph-osd+0x2b60795a)
    ceph#7 0x7f7aa0af97b2 in __libc_start_main (/lib64/libc.so.6+0x237b2)
    ceph#8 0x560f477d2e8d in _start (/usr/bin/ceph-osd+0x2b72ae8d)

```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-20_07:28:16-rados-master-distro-basic-smithi/6124735/

The commit deals with the problem by repacking the outer future.
An alternative could be in switching from `std::unique_ptr` to
`seastar::shared_ptr` for managing `OpsExecuter`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 25, 2021
f7181ab has optimized the client
parallelism. To achieve that `PG::do_osd_ops()` were converted to
return basically future pair of futures. Unfortunately, the life-
time management of `OpsExecuter` was kept intact. In the result,
the object was valid only till fullfying the outer future while,
due to the `rollbacker` instances, it should be available till
`all_completed` becomes available.

This issue can explain the following problem has been observed
in a Teuthology job [1].

```
DEBUG 2021-05-20 08:03:22,617 [shard 0] osd - do_op_call: method returned ret=-17, outdata.length()=0 while num_read=1, num_write=0
DEBUG 2021-05-20 08:03:22,617 [shard 0] osd - rollback_obc_if_modified: object 19:e17d4708:test-rados-api-smithi095-38404-2::foo:head got erro
r generic:17, need_rollback=false
=================================================================
==33626==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0000b9320 at pc 0x560f486b8222 bp 0x7fffc467a1e0 sp 0x7fffc467a1d0
READ of size 4 at 0x60d0000b9320 thread T0
    #0 0x560f486b8221  (/usr/bin/ceph-osd+0x2c610221)
    #1 0x560f4880c6b1 in seastar::continuation<seastar::internal::promise_base_with_type<boost::intrusive_ptr<MOSDOpReply> >, seastar::noncopy
able_function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_
wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > >
 > ()>, seastar::future<void>::then_impl_nrvo<seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::osd::
IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson:
:errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInte
rruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::error
ated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > >(seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail
<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_
future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>&&)::{lambda(seastar::internal::promise_base_with_type<boos
t::intrusive_ptr<MOSDOpReply> >&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>&, seastar::future_state<seastar::internal::monostate>&&)#1}, void>::run_and_dispose() (/usr/bin/ceph-osd+0x2c7646b1)
    #2 0x560f5352c3ae  (/usr/bin/ceph-osd+0x374843ae)
    ceph#3 0x560f535318ef  (/usr/bin/ceph-osd+0x374898ef)
    ceph#4 0x560f536e395a  (/usr/bin/ceph-osd+0x3763b95a)
    ceph#5 0x560f532413d9  (/usr/bin/ceph-osd+0x371993d9)
    ceph#6 0x560f476af95a in main (/usr/bin/ceph-osd+0x2b60795a)
    ceph#7 0x7f7aa0af97b2 in __libc_start_main (/lib64/libc.so.6+0x237b2)
    ceph#8 0x560f477d2e8d in _start (/usr/bin/ceph-osd+0x2b72ae8d)

```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-20_07:28:16-rados-master-distro-basic-smithi/6124735/

The commit deals with the problem by repacking the outer future.
An alternative could be in switching from `std::unique_ptr` to
`seastar::shared_ptr` for managing `OpsExecuter`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 27, 2021
…ore.

```
INFO  2021-05-26 20:16:32,872 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] ProtocolV2::start_accept(): targ
et_addr=172.21.15.119:55220/0
DEBUG 2021-05-26 20:16:32,872 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] TRIGGER ACCEPTING, was NONE
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] SEND(26) banner: len_payload=16,
 supported=1, required=0, banner="ceph v2
"
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] RECV(10) banner: "ceph v2
"
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] GOT banner: payload_len=16
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] RECV(16) banner features: supported=1 required=0
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] WRITE HelloFrame: my_type=osd, peer_addr=172.21.15.119:55220/0
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] GOT HelloFrame: my_type=client peer_addr=v2:172.21.15.119:6803/31733
INFO  2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> client.? -@55220] UPDATE: peer_type=client, policy(lossy=true server=true standby=false resetcheck=false)
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> client.? -@55220] GOT AuthRequestFrame: method=2, preferred_modes={1, 2}, payload_len=174
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4622-gaa1dc559/rpm/el8/BUILD/ceph-17.0.0-4622-gaa1dc559/src/crimson/mon/MonClient.cc:399:10: runtime error: member access within null pointer of type 'struct Connection'
Segmentation fault on shard 0.
Backtrace:
 0# 0x000055E84CF44C1F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F2BC88C0B20 in /lib64/libpthread.so.0
 4# crimson::mon::Connection::get_conn() in ceph-osd
 5# crimson::mon::Client::handle_auth_request(seastar::shared_ptr<crimson::net::Connection>, seastar::lw_shared_ptr<AuthConnectionMeta>, bool, unsigned int, ceph::buffer::v15_2_0::list const&, ceph::buffer::v15_2_0::list*) in ceph-osd
 6# crimson::net::ProtocolV2::_handle_auth_request(ceph::buffer::v15_2_0::list&, bool) in ceph-osd
 7# 0x000055E84DF67669 in ceph-osd
 8# 0x000055E84DF68775 in ceph-osd
 9# 0x000055E846F47F60 in ceph-osd
10# 0x000055E85296770F in ceph-osd
11# 0x000055E85296CC50 in ceph-osd
12# 0x000055E852B1ECBB in ceph-osd
13# 0x000055E85267C73A in ceph-osd
14# main in ceph-osd
15# __libc_start_main in /lib64/libc.so.6
16# _start in ceph-osd
Fault at location: 0x98
```

When the `handle_auth_request()` happens, there is no guarantee
`active_con` is being available. This is reflected in the classical
implementation:

```cpp
int MonClient::handle_auth_request(
  Connection *con,
  // ...
  ceph::buffer::list *reply)
{
  // ...
  bool isvalid = ah->verify_authorizer(
    cct,
    *rotating_secrets,
    payload,
    auth_meta->get_connection_secret_length(),
    reply,
    &con->peer_name,
    &con->peer_global_id,
    &con->peer_caps_info,
    &auth_meta->session_key,
    &auth_meta->connection_secret,
    ac);
```

The patch transplate the same logic to crimson.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 27, 2021
Following crash occured at Sepia [1]:

```
INFO  2021-05-26 20:16:32,872 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] ProtocolV2::start_accept(): targ
et_addr=172.21.15.119:55220/0
DEBUG 2021-05-26 20:16:32,872 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] TRIGGER ACCEPTING, was NONE
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] SEND(26) banner: len_payload=16,
 supported=1, required=0, banner="ceph v2
"
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] RECV(10) banner: "ceph v2
"
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] GOT banner: payload_len=16
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] RECV(16) banner features: supported=1 required=0
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] WRITE HelloFrame: my_type=osd, peer_addr=172.21.15.119:55220/0
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> unknown.? -@55220] GOT HelloFrame: my_type=client peer_addr=v2:172.21.15.119:6803/31733
INFO  2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> client.? -@55220] UPDATE: peer_type=client, policy(lossy=true server=true standby=false resetcheck=false)
DEBUG 2021-05-26 20:16:32,873 [shard 0] ms - [osd.0(client) v2:172.21.15.119:6803/31733 >> client.? -@55220] GOT AuthRequestFrame: method=2, preferred_modes={1, 2}, payload_len=174
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4622-gaa1dc559/rpm/el8/BUILD/ceph-17.0.0-4622-gaa1dc559/src/crimson/mon/MonClient.cc:399:10: runtime error: member access within null pointer of type 'struct Connection'
Segmentation fault on shard 0.
Backtrace:
 0# 0x000055E84CF44C1F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F2BC88C0B20 in /lib64/libpthread.so.0
 4# crimson::mon::Connection::get_conn() in ceph-osd
 5# crimson::mon::Client::handle_auth_request(seastar::shared_ptr<crimson::net::Connection>, seastar::lw_shared_ptr<AuthConnectionMeta>, bool, unsigned int, ceph::buffer::v15_2_0::list const&, ceph::buffer::v15_2_0::list*) in ceph-osd
 6# crimson::net::ProtocolV2::_handle_auth_request(ceph::buffer::v15_2_0::list&, bool) in ceph-osd
 7# 0x000055E84DF67669 in ceph-osd
 8# 0x000055E84DF68775 in ceph-osd
 9# 0x000055E846F47F60 in ceph-osd
10# 0x000055E85296770F in ceph-osd
11# 0x000055E85296CC50 in ceph-osd
12# 0x000055E852B1ECBB in ceph-osd
13# 0x000055E85267C73A in ceph-osd
14# main in ceph-osd
15# __libc_start_main in /lib64/libc.so.6
16# _start in ceph-osd
Fault at location: 0x98
```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-26_12:20:26-rados-master-distro-basic-smithi/6136907

When the `handle_auth_request()` happens, there is no guarantee
`active_con` is being available. This is reflected in the classical
implementation:

```cpp
int MonClient::handle_auth_request(
  Connection *con,
  // ...
  ceph::buffer::list *reply)
{
  // ...
  bool isvalid = ah->verify_authorizer(
    cct,
    *rotating_secrets,
    payload,
    auth_meta->get_connection_secret_length(),
    reply,
    &con->peer_name,
    &con->peer_global_id,
    &con->peer_caps_info,
    &auth_meta->session_key,
    &auth_meta->connection_secret,
    ac);
```

The patch transplate the same logic to crimson.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 31, 2021
The `FuturizedStore` interface imposes the `get_attr()`
takes the `name` parameter as `std::string_view`, and
thus burdens implementations with extending the life-
time of the data the instance refers to.

Unfortunately, `AlienStore` is unaware that prolonging
the life of a `std::string_view` instance doesn't prolong
the data memory it points to. This problem has manifested
in the following use-after-free detected at Sepia:

```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-05-26_12:20:26-rados-master-distro-basic-smithi/6136929$ less ./remote/smithi194/log/ceph-osd.7.log.gz
...
DEBUG 2021-05-26 20:24:54,077 [shard 0] osd - do_osd_ops_execute: object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head - handling op
call
DEBUG 2021-05-26 20:24:54,077 [shard 0] osd - handling op call on object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - calling method lock.lock, num_read=0, num_write=0
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - handling op getxattr on object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - getxattr on obj=14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head for attr=_lock.TestLockPP1
DEBUG 2021-05-26 20:24:54,078 [shard 0] bluestore - get_attr
=================================================================
==34068==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030001851d0 at pc 0x7f824d6a5b27 bp 0x7f822b4201c0 sp 0x7f822b41f968
READ of size 17 at 0x6030001851d0 thread T28 (alien-store-tp)
...
    #0 0x7f824d6a5b26  (/lib64/libasan.so.5+0x40b26)
    #1 0x55e2cbb2e00b  (/usr/bin/ceph-osd+0x2b6dc00b)
    #2 0x55e2d31f086e  (/usr/bin/ceph-osd+0x32d9e86e)
    ceph#3 0x55e2d3467607 in crimson::os::ThreadPool::loop(std::chrono::duration<long, std::ratio<1l, 1000l> >, unsigned long) (/usr/bin/ceph-osd+0x33015607)
    ceph#4 0x55e2d346b14a  (/usr/bin/ceph-osd+0x3301914a)
    ceph#5 0x7f8249d32ba2  (/lib64/libstdc++.so.6+0xc2ba2)
    ceph#6 0x7f824a00d149 in start_thread (/lib64/libpthread.so.0+0x8149)
    ceph#7 0x7f82486edf22 in clone (/lib64/libc.so.6+0xfcf22)

0x6030001851d0 is located 0 bytes inside of 31-byte region [0x6030001851d0,0x6030001851ef)
freed by thread T0 here:
    #0 0x7f824d757688 in operator delete(void*) (/lib64/libasan.so.5+0xf2688)

previously allocated by thread T0 here:
    #0 0x7f824d7567b0 in operator new(unsigned long) (/lib64/libasan.so.5+0xf17b0)

Thread T28 (alien-store-tp) created by T0 here:
    #0 0x7f824d6b7ea3 in __interceptor_pthread_create (/lib64/libasan.so.5+0x52ea3)

SUMMARY: AddressSanitizer: heap-use-after-free (/lib64/libasan.so.5+0x40b26)
Shadow bytes around the buggy address:
  0x0c06800289e0: fd fd fd fa fa fa fd fd fd fa fa fa 00 00 00 fa
  0x0c06800289f0: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
  0x0c0680028a00: fd fa fa fa fd fd fd fa fa fa fd fd fd fa fa fa
  0x0c0680028a10: fd fd fd fa fa fa fd fd fd fa fa fa fd fd fd fa
  0x0c0680028a20: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
=>0x0c0680028a30: fd fd fa fa fd fd fd fd fa fa[fd]fd fd fd fa fa
  0x0c0680028a40: fd fd fd fd fa fa fd fd fd fd fa fa 00 00 00 07
  0x0c0680028a50: fa fa 00 00 00 fa fa fa 00 00 00 fa fa fa fd fd
  0x0c0680028a60: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c0680028a70: 00 00 00 00 fa fa fd fd fd fd fa fa fd fd fd fd
  0x0c0680028a80: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==34068==ABORTING
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request May 31, 2021
The `FuturizedStore` interface imposes the `get_attr()`
takes the `name` parameter as `std::string_view`, and
thus burdens implementations with extending the life-
time of the data the instance refers to.

Unfortunately, `AlienStore` is unaware that prolonging
the life of a `std::string_view` instance doesn't prolong
the data memory it points to. This problem has manifested
in the following use-after-free detected at Sepia:

```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-05-26_12:20:26-rados-master-distro-basic-smithi/6136929$ less ./remote/smithi194/log/ceph-osd.7.log.gz
...
DEBUG 2021-05-26 20:24:54,077 [shard 0] osd - do_osd_ops_execute: object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head - handling op
call
DEBUG 2021-05-26 20:24:54,077 [shard 0] osd - handling op call on object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - calling method lock.lock, num_read=0, num_write=0
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - handling op getxattr on object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - getxattr on obj=14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head for attr=_lock.TestLockPP1
DEBUG 2021-05-26 20:24:54,078 [shard 0] bluestore - get_attr
=================================================================
==34068==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030001851d0 at pc 0x7f824d6a5b27 bp 0x7f822b4201c0 sp 0x7f822b41f968
READ of size 17 at 0x6030001851d0 thread T28 (alien-store-tp)
...
    #0 0x7f824d6a5b26  (/lib64/libasan.so.5+0x40b26)
    #1 0x55e2cbb2e00b  (/usr/bin/ceph-osd+0x2b6dc00b)
    #2 0x55e2d31f086e  (/usr/bin/ceph-osd+0x32d9e86e)
    ceph#3 0x55e2d3467607 in crimson::os::ThreadPool::loop(std::chrono::duration<long, std::ratio<1l, 1000l> >, unsigned long) (/usr/bin/ceph-osd+0x33015607)
    ceph#4 0x55e2d346b14a  (/usr/bin/ceph-osd+0x3301914a)
    ceph#5 0x7f8249d32ba2  (/lib64/libstdc++.so.6+0xc2ba2)
    ceph#6 0x7f824a00d149 in start_thread (/lib64/libpthread.so.0+0x8149)
    ceph#7 0x7f82486edf22 in clone (/lib64/libc.so.6+0xfcf22)

0x6030001851d0 is located 0 bytes inside of 31-byte region [0x6030001851d0,0x6030001851ef)
freed by thread T0 here:
    #0 0x7f824d757688 in operator delete(void*) (/lib64/libasan.so.5+0xf2688)

previously allocated by thread T0 here:
    #0 0x7f824d7567b0 in operator new(unsigned long) (/lib64/libasan.so.5+0xf17b0)

Thread T28 (alien-store-tp) created by T0 here:
    #0 0x7f824d6b7ea3 in __interceptor_pthread_create (/lib64/libasan.so.5+0x52ea3)

SUMMARY: AddressSanitizer: heap-use-after-free (/lib64/libasan.so.5+0x40b26)
Shadow bytes around the buggy address:
  0x0c06800289e0: fd fd fd fa fa fa fd fd fd fa fa fa 00 00 00 fa
  0x0c06800289f0: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
  0x0c0680028a00: fd fa fa fa fd fd fd fa fa fa fd fd fd fa fa fa
  0x0c0680028a10: fd fd fd fa fa fa fd fd fd fa fa fa fd fd fd fa
  0x0c0680028a20: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
=>0x0c0680028a30: fd fd fa fa fd fd fd fd fa fa[fd]fd fd fd fa fa
  0x0c0680028a40: fd fd fd fd fa fa fd fd fd fd fa fa 00 00 00 07
  0x0c0680028a50: fa fa 00 00 00 fa fa fa 00 00 00 fa fa fa fd fd
  0x0c0680028a60: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c0680028a70: 00 00 00 00 fa fa fd fd fd fd fa fa fd fd fd fd
  0x0c0680028a80: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==34068==ABORTING
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Aug 10, 2022
This commit switches `ceph::osd::Transaction` to hold `Context`
instances with `std::unique_ptr` instead of raw pointers which
addesses the following memory leak observed in crimson:

```
INFO  2022-08-01 11:31:29,176 [shard 0] osd - crimson shutdown complete

=================================================================
==101541==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7fbaadd30307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307)
    #1 0x55922ad3d81b in non-virtual thunk to crimson::osd::PG::start_flush_on_transaction(ceph::os::Transaction&) (/usr/bin/ceph-osd+0x32fda81b)
    #2 0x60800015019f  (<unknown module>)

SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
daemon-helper: command failed with exit status 1
```

It's likely similar problem affects the classical OSD as well.

TODO: refactor usages of `register_on_*` and switch them
to `std::make_unique`. This will free us from *assuming*
we always get the ownership (a compiler will validate).

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Aug 10, 2022
This commit switches `ceph::osd::Transaction` to hold `Context`
instances with `std::unique_ptr` instead of raw pointers which
addesses the following memory leak observed in crimson:

```
INFO  2022-08-01 11:31:29,176 [shard 0] osd - crimson shutdown complete

=================================================================
==101541==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7fbaadd30307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307)
    #1 0x55922ad3d81b in non-virtual thunk to crimson::osd::PG::start_flush_on_transaction(ceph::os::Transaction&) (/usr/bin/ceph-osd+0x32fda81b)
    #2 0x60800015019f  (<unknown module>)

SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
daemon-helper: command failed with exit status 1
```

It's likely similar problem affects the classical OSD as well.

TODO: refactor usages of `register_on_*` and switch them
to `std::make_unique`. This will free us from *assuming*
we always get the ownership (a compiler will validate).

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Aug 10, 2022
This commit switches `ceph::osd::Transaction` to hold `Context`
instances with `std::unique_ptr` instead of raw pointers which
addesses the following memory leak observed in crimson:

```
INFO  2022-08-01 11:31:29,176 [shard 0] osd - crimson shutdown complete

=================================================================
==101541==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7fbaadd30307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307)
    #1 0x55922ad3d81b in non-virtual thunk to crimson::osd::PG::start_flush_on_transaction(ceph::os::Transaction&) (/usr/bin/ceph-osd+0x32fda81b)
    #2 0x60800015019f  (<unknown module>)

SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
daemon-helper: command failed with exit status 1
```

It's likely similar problem affects the classical OSD as well.

TODO: refactor usages of `register_on_*` and switch them
to `std::make_unique`. This will free us from *assuming*
we always get the ownership (a compiler will validate).

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Aug 23, 2022
`Transaction` aggregates multiple contexts (event handlers) of
three different genres:
  * on applied,
  * on commit,
  * on sync.

Although not expressed within language means, a transaction holds
ownership towards these contextes, and thus should free them when
it gets destructed. Usually, this isn't an issue as those objects
are taken from it by `collect_contexts()`, `get_on_*` or similar.

However, there are some `Transaction` instances that can get
destructed with contexts on board. Such a situation already
happened in crimson:

```
INFO  2022-08-01 11:31:29,176 [shard 0] osd - crimson shutdown complete

=================================================================
==101541==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7fbaadd30307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307)
    #1 0x55922ad3d81b in non-virtual thunk to crimson::osd::PG::start_flush_on_transaction(ceph::os::Transaction&) (/usr/bin/ceph-osd+0x32fda81b)
    #2 0x60800015019f  (<unknown module>)

SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
daemon-helper: command failed with exit status 1
```

I'm afraid the same can affect the classical world but we may be
missing it as the valgrind coverage is narrower than the one that
comes from ASan in crimson (every single run hunts for memleaks).

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski added a commit that referenced this pull request Aug 23, 2022
This commit switches `ceph::osd::Transaction` to hold `Context`
instances with `std::unique_ptr` instead of raw pointers which
addesses the following memory leak observed in crimson:

```
INFO  2022-08-01 11:31:29,176 [shard 0] osd - crimson shutdown complete

=================================================================
==101541==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7fbaadd30307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307)
    #1 0x55922ad3d81b in non-virtual thunk to crimson::osd::PG::start_flush_on_transaction(ceph::os::Transaction&) (/usr/bin/ceph-osd+0x32fda81b)
    #2 0x60800015019f  (<unknown module>)

SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
daemon-helper: command failed with exit status 1
```

It's likely similar problem affects the classical OSD as well.

TODO: refactor usages of `register_on_*` and switch them
to `std::make_unique`. This will free us from *assuming*
we always get the ownership (a compiler will validate).

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
rzarzynski pushed a commit that referenced this pull request Apr 27, 2023
Improvement #1:

CapTester.write_test_files() not only creates the test file but also
does the following for every mount object it receives in parameters -

* carefully produces the path for the test file as per parameters
  received
* generates the unique data for each test file on a CephFS mount
* creates a data structure -- list of lists -- that holds all this
  information along with mount object itself for each mount object so
  that tests can be conducted at a later point

Untangle this mess of code by splitting this method into 3 separate
methods -

1. To produce the path for test file (as per user's need).
2. To generate the data that will be written into the test file.
3. To actually create the test file on CephFS.

Improvement #2:

Remove the internal data structure used for testing -- self.test_set --
and use separate class attributes to store all the data required for
testing instead of a tuple. This serves two purpose -

One, it makes it easy to manipulate all this data from helper methods
and during debugging session, especially while using a PDB session.

And two, make it impossible to have multiple mounts/multiple "test sets"
within same CapTester instance for the sake of simplicity. Users can
instead create two instances of CapTester instances if needed.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
rzarzynski pushed a commit that referenced this pull request Aug 28, 2023
```
=================================================================
==80592==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f5c76eb6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
    #1 0x7f5c76a2fb81 in MallocExtension::Register(MallocExtension*) (/lib64/libtcmalloc.so.4+0x2fb81)

SUMMARY: AddressSanitizer: 8 byte(s) leaked in 1 allocation(s)
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
rzarzynski pushed a commit that referenced this pull request Dec 4, 2023
Sanitized backtrace:
```
DEBUG 2023-11-14 15:23:50,871 [shard 0] osd - snaptrim_event(id=10610, detail=SnapTrimEvent(pgid=16.1a snapid=a needs_pause=0)): interrupted crimson::common::actingset_changed (acting set changed)

    #0 0x5653c613c071 in seastar::shared_mutex::unlock() (/usr/bin/ceph-osd+0x1ed27071)
    #1 0x5653c8670acf in auto seastar::futurize_invoke<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&>(crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&) (/usr/bin/ceph-osd+0x2125bacf)
    #2 0x5653c8670e22 in _ZN7seastar20noncopyable_functionIFNS_6futureIvEEvEE17direct_vtable_forIZNS2_4thenIZN7crimson23OrderedConcurrentPhaseTINS7_3osd13SnapTrimEvent9WaitSubopEE11ExitBarrierINSC_13BlockingEvent7TriggerISA_EEE4exitEvEUlvE_S2_EET0_OT_EUlDpOT_E_E4callEPKS4_ (/usr/bin/ceph-osd+0x2125be22)

freed by thread T1 here:
    #0 0x7f10628b73cf in operator delete(void*, unsigned long) (/lib64/libasan.so.6+0xb73cf)
    #1 0x5653c8794bff in crimson::osd::SnapTrimEvent::~SnapTrimEvent() (/usr/bin/ceph-osd+0x2137fbff)

previously allocated by thread T1 here:
    #0 0x7f10628b6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)

SUMMARY: AddressSanitizer: heap-use-after-free (/usr/bin/ceph-osd+0x1ed27071) in seastar::shared_mutex::unlock()
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
rzarzynski pushed a commit that referenced this pull request Dec 4, 2023
```
    // SnapTrimEvent is a background operation,
    // it's lifetime is not guarnteed since the caller
    // returned future is being ignored. We should capture
    // a self reference thourhgout the entire execution
    // progress (not only on finally() continuations).
    // See: PG::on_active_actmap()
```

Sanitized backtrace:
```
DEBUG 2023-11-16 08:42:48,441 [shard 0] osd - snaptrim_event(id=21122, detail=SnapTrimEvent(pgid=3.1 snapid=3cb needs_pause=1)): interrupted crimson::common::actingset_changed (acting set changed

kernel callstack:
    #0 0x55e310e0ace7 in seastar::shared_mutex::unlock() (/usr/bin/ceph-osd+0x1edd0ce7)
    #1 0x55e313325d9c in auto seastar::futurize_invoke<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&>(crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&) (/usr/bin/ceph-osd+0x212ebd9c)
    #2 0x55e3133260ef in _ZN7seastar20noncopyable_functionIFNS_6futureIvEEvEE17direct_vtable_forIZNS2_4thenIZN7crimson23OrderedConcurrentPhaseTINS7_3osd13SnapTrimEvent9WaitSubopEE11ExitBarrierINSC_13BlockingEvent7TriggerISA_EEE4exitEvEUlvE_S2_EET0_OT_EUlDpOT_E_E4callEPKS4_ (/usr/bin/ceph-osd+0x212ec0ef)
0x61500013365c is located 92 bytes inside of 472-byte region [0x615000133600,0x6150001337d8)
freed by thread T2 here:
    #0 0x7fb345ab73cf in operator delete(void*, unsigned long) (/lib64/libasan.so.6+0xb73cf)
    #1 0x55e313474863 in crimson::osd::SnapTrimEvent::~SnapTrimEvent() (/usr/bin/ceph-osd+0x2143a863)

previously allocated by thread T2 here:
    #0 0x7fb345ab6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
    #1 0x55e31183ac18 in auto crimson::OperationRegistryI::create_operation<crimson::osd::SnapTrimEvent, crimson::osd::PG*, SnapMapper&, snapid_t const&, bool const&>(crimson::osd::PG*&&, SnapMapper&, snapid_t const&, bool const&) (/usr/bin/ceph-osd+0x1f800c18)
SUMMARY: AddressSanitizer: heap-use-after-free (/usr/bin/ceph-osd+0x1edd0ce7) in seastar::shared_mutex::unlock()
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
rzarzynski pushed a commit that referenced this pull request Jan 11, 2024
```
=================================================================
==80592==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f5c76eb6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
    #1 0x7f5c76a2fb81 in MallocExtension::Register(MallocExtension*) (/lib64/libtcmalloc.so.4+0x2fb81)

SUMMARY: AddressSanitizer: 8 byte(s) leaked in 1 allocation(s)
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit bc19097)
rzarzynski pushed a commit that referenced this pull request Mar 27, 2024
before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance, this not only annoys ASan, and defeats
the purpose of 14d878c.
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x5564d173537d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_ipaddr+0x19b37d) (BuildId: 45c0c7f28b253c04fcb7bb1a43aed52a5526d734)
    #1 0x7fe7f2ccd189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7fe7f2ccc563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    ceph#3 0x7fe7f2ccc563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    ceph#4 0x7fe7f2ccc2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    ceph#5 0x7fe7f2cc6192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    ceph#6 0x7fe7f2cb91e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    ceph#7 0x7fe7f1f8f96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    ceph#8 0x7fe7f1f8e93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    ceph#9 0x5564d1752eb9 in pick_address_find_ip_in_subnet_list_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_ipaddr.cc:706:47
    ceph#10 0x5564d18694d6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#11 0x5564d1820fc2 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#12 0x5564d17d19dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#13 0x5564d17d3a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#14 0x5564d17d504b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#15 0x5564d17f24d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#16 0x5564d1871d06 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#17 0x5564d1827932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#18 0x5564d17f1862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#19 0x5564d1775d80 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#20 0x5564d1775d11 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
```

so, in this change, we do not increase the refcount when
creating cct.

the same applies to `test/common/test_fault_injector.cc`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
rzarzynski pushed a commit that referenced this pull request Mar 27, 2024
before this change, in test_util.cc, we increment the refcount of
when constructing it. but at that moment, nobody really owns it.
also, `CephContext` 's refcount is set to 1 in its constructor.
so, we should not do this. otherwise, the created `CephContext`
is leaked as LeakSanitizer rightly points out:
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x5632320d27ed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_util+0x1917ed) (BuildId: ff1df1455bd07b651ad580584a17ea204afeb36e)
    #1 0x7ff9d535b189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7ff9d535a563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    ceph#3 0x7ff9d535a563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    ceph#4 0x7ff9d535a2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    ceph#5 0x7ff9d5354192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    ceph#6 0x7ff9d53471e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    ceph#7 0x7ff9d461d96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    ceph#8 0x7ff9d461c93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    ceph#9 0x5632320d52e0 in util_collect_sys_info_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/common/test_util.cc:34:27
    ceph#10 0x563232205c16 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#11 0x5632321c2742 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#12 0x5632321736dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
```
in this change, instead of using a raw pointer, let's
use `boost::intrusive_ptr<CephContext>` to manage the lifecyle
of `CephContext`, this also address the leakage reported by
LeakSanitizer.

the same applies to common/test_context.cc

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
rzarzynski pushed a commit that referenced this pull request Mar 27, 2024
before this change, we allocate memory chunks with specified
size using `new []`, but we never free them. when testing with
LeakSanitizer enabled, it rightly points identifies the leakage:

```
Direct leak of 8754 byte(s) in 184 object(s) allocated from:
    #0 0x55c0b2470f0d in operator new[](unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_memory+0x196f0d) (BuildId: d3267dd8819427b804c4729e0467dbe7601fb321)
    #1 0x55c0b247456c in MemoryIsZeroSmallTest_MemoryIsZeroTestSmall_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/common/test_memory.cc:33:18
    #2 0x55c0b2598ee6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#3 0x55c0b2553b92 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#4 0x55c0b25049dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#5 0x55c0b2506a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#6 0x55c0b250804b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#7 0x55c0b25254d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#8 0x55c0b25a16f6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#9 0x55c0b255a502 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#10 0x55c0b2524862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#11 0x55c0b24ab4c0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#12 0x55c0b24ab451 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    ceph#13 0x7f45e065ad8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we free the allocate memory.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
rzarzynski pushed a commit that referenced this pull request Mar 27, 2024
…sive_ptr<CephContext>

before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance. and LeakSanitizer points this out:

```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac359c7c8 in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/bin/unittest_rgw_iam_policy+0x211c7c8) (BuildId: 060fadb10da261b52fd5757c7b1e9812d34542f1)
    #1 0xffff96f764e4 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0xffff96f757cc in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    ceph#3 0xffff96f757cc in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:2396:39
    ceph#4 0xffff96f75500 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:2494:18
    ceph#5 0xffff96f6ec4c in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:1039:9
    ceph#6 0xffff96f63528 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/log/Log.cc:53:5
    ceph#7 0xffff96045300 in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/common/ceph_context.cc:729:16
    ceph#8 0xffff960446ec in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/common/ceph_context.cc:697:5
    ceph#9 0xaaaac3629238 in IPPolicyTest::IPPolicyTest() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/rgw/test_rgw_iam_policy.cc:864:15
    ceph#10 0xaaaac3628da0 in IPPolicyTest_MaskedIPOperations_Test::IPPolicyTest_MaskedIPOperations_Test() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/rgw/test_rgw_iam_policy.cc:869:1
    ceph#11 0xaaaac3628d3c in testing::internal::TestFactoryImpl<IPPolicyTest_MaskedIPOperations_Test>::CreateTest() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
```

so, in this change, we do not increase the refcount when creating cct.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
rzarzynski pushed a commit that referenced this pull request Mar 27, 2024
before this change, we create a new cct instance with `new`, but
we never free this instance after done with it. and LeakSanitizer
points this out:

```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x561afe148fed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_config_map+0x1c2fed) (BuildId: 3ce9eeed38cee335628fa74fdd08cd215b15019e)
    #1 0x7f37dc9ac189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7f37dc9ab563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    ceph#3 0x7f37dc9ab563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    ceph#4 0x7f37dc9ab2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    ceph#5 0x7f37dc9a5192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    ceph#6 0x7f37dc9981e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    ceph#7 0x7f37dbc6e96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    ceph#8 0x7f37dbc6d93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    ceph#9 0x561afe14e983 in ConfigMap_add_option_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/test_config_map.cc:58:18
    ceph#10 0x561afe2689b6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#11 0x561afe221262 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#12 0x561afe1d1f7c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#13 0x561afe1d3fb2 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#14 0x561afe1d55eb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#15 0x561afe1f2a78 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#16 0x561afe2711e6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#17 0x561afe227bd2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#18 0x561afe1f1e02 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#19 0x561afe176ec0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#20 0x561afe176e51 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    ceph#21 0x7f37d9397d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so in this change, we manage the `CephContext` pointer with a smart
pointer. because the size of CephContext could be large, we don't create
it on stack.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
rzarzynski pushed a commit that referenced this pull request Mar 27, 2024
before this change, we create a new CrushWrapper instance with `new`, but
we never free this instance after done with it. and LeakSanitizer
points this out:

```
Direct leak of 544 byte(s) in 1 object(s) allocated from:
    #0 0x561afe148fed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_config_map+0x1c2fed) (BuildId: 3ce9eeed38cee335628fa74fdd08cd215b15019e)
    #1 0x561afe151cbd in ConfigMap_result_sections_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/test_config_map.cc:93:16
    #2 0x561afe2689b6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#3 0x561afe221262 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#4 0x561afe1d1f7c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#5 0x561afe1d3fb2 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#6 0x561afe1d55eb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#7 0x561afe1f2a78 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#8 0x561afe2711e6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#9 0x561afe227bd2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#10 0x561afe1f1e02 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#11 0x561afe176ec0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#12 0x561afe176e51 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    ceph#13 0x7f37d9397d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so in this change, we manage the `CrushWrapper` pointer with a smart
pointer. because the size of `CrushWrapper` is relatively large, we
don't create it on stack.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
ljflores pushed a commit that referenced this pull request Apr 8, 2024
LeakSanitizer reports
```
==688591==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 45 byte(s) in 1 object(s) allocated from:
    #0 0x55f8dd9969dd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_fastbmap_allocator+0x1f89dd) (BuildId: cac39eac8ef1e8774f9dd48e6e3f677fdd864776)
    #1 0x55f8dd99c730 in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x55f8dd99c690 in std::allocator<char>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    ceph#3 0x55f8dd99c690 in std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    ceph#4 0x55f8dd99c393 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:153:14
    ceph#5 0x55f8dda96a6c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:307:21
    ceph#6 0x55f8dda96852 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:395:8
    ceph#7 0x7f4a751ab6f0 in
    MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId:
    eeef3d1257388a806e122398dbce3157ee568ef4)
```

this is a global object allocated by the allocator, so we can suppress this report.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
ljflores pushed a commit that referenced this pull request Apr 8, 2024
before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance. and LeakSanitizer points this out:
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x558d341d837d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_ipaddr+0x19b37d) (BuildId: 1b7e7e5abfc2b58ce2334712e4c00b2441c25870)
    #1 0x7fd74c957559 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7fd74c956933 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    ceph#3 0x7fd74c956933 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    ceph#4 0x7fd74c956690 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    ceph#5 0x7fd74c950562 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circ
ular_buffer/base.hpp:1039:9
    ceph#6 0x7fd74c9435b4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    ceph#7 0x7fd74bc1891d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    ceph#8 0x7fd74bc178eb in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    ceph#9 0x558d341f97e9 in pick_address_filtering_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_ipaddr.cc:774:47
    ceph#10 0x558d3430c4f6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#11 0x558d342c3fc2 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#12 0x558d342749dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#13 0x558d34276a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#14 0x558d3427804b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#15 0x558d342954d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#16 0x558d34314d26 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#17 0x558d342ca932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#18 0x558d34294862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#19 0x558d34218d80 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#20 0x558d34218d11 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    ceph#21 0x7fd749331d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so, in this change, we do not increase the refcount when creating cct.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
ljflores pushed a commit that referenced this pull request Apr 8, 2024
in BlueFS.test_shared_alloc, we keep the return value of
`fs.get_perf_counters()`, and deference it after umounting the fs,
but the `PerfCounters*` pointer returned from `fs.get_perf_counters()`
is destroyed in `BlueFS::_shutdown_logger()` which is in turn called
by `BlueFS::umount()`. so ASan points this out:
```
==1662613==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110000b2d80 at pc 0x7f0eefc30644 bp 0x7ffcdbab6430 sp 0x7ffcdbab6428
READ of size 8 at 0x6110000b2d80 thread T0
    #0 0x7f0eefc30643 in ceph::common::PerfCounters::get(int) const /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:246:8
    #1 0x557595ddfc15 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1182:3
    #2 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#3 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#4 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#5 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#6 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#7 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#8 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#9 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#10 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#11 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#12 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    ceph#13 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#14 0x7f0eed083e3f in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#15 0x557595cd46a4 in _start (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x2856a4) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)

0x6110000b2d80 is located 0 bytes inside of 208-byte region [0x6110000b2d80,0x6110000b2e50)
freed by thread T0 here:
    #0 0x557595d92b1d in operator delete(void*) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x343b1d) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
    #1 0x557595f31c43 in BlueFS::_shutdown_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:462:3
    #2 0x557595f54ab5 in BlueFS::umount(bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:1076:3
    ceph#3 0x557595ddfbd7 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1180:6
    ceph#4 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#5 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#6 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#7 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#8 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#9 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#10 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#11 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#12 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#13 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#14 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    ceph#15 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

previously allocated by thread T0 here:
    #0 0x557595d922bd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x3432bd) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
    #1 0x7f0eefc33180 in ceph::common::PerfCountersBuilder::PerfCountersBuilder(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:537:21
    #2 0x557595f30ac9 in BlueFS::_init_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:221:23
    ceph#3 0x557595f42bc6 in BlueFS::mount() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:977:3
    ceph#4 0x557595ddd339 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1139:3
    ceph#5 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#6 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#7 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#8 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#9 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#10 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#11 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#12 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#13 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#14 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#15 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    ceph#16 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, instead of keeping `logger` across the `umount()` and
`mount()` calls, we get another instance of `logger`, query it for
the perf counter that we are interested, and compare the value
to see if it is unchanged.

this should address the ASan warning above.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
ljflores pushed a commit that referenced this pull request Apr 8, 2024
before this change, we allocate an instance of `RocksDBStore` with
`new`, but we never free it. and LeanSanitizer points this out:

```
Direct leak of 952 byte(s) in 1 object(s) allocated from:
    #0 0x55f31440bc2d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_rocksdb_option+0xaebc2d) (BuildId: 81b849dbc41cbc6b05d5e603d9ba8a002dab2d24)
    #1 0x55f3144132fd in RocksDBOption_simple_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/TestRocksdbOptionParse.cc:17:22
    #2 0x55f3144ecf26 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#3 0x55f3144a4312 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#4 0x55f314453ccc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#5 0x55f314455d02 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#6 0x55f31445733b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#7 0x55f3144747c8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#8 0x55f3144f5576 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#9 0x55f3144ab1a2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#10 0x55f314473b52 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#11 0x55f31440f690 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#12 0x55f31440e4c3 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/unit.cc:45:10
    ceph#13 0x7f0d32551d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we manage the life cycle of `RocksDBStore` using
a smart pointer. this should address the leak.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
ljflores pushed a commit that referenced this pull request Apr 8, 2024
before this change, we allocate memory chunks using malloc(), but
we never free them. and LeakSanitizer points this out

```
Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x5588bfe532de in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_on_exit+0xa52de) (BuildId: 7c7a92bf5719592938c5307214bcd9b080bd847f)
    #1 0x5588bfe911d7 in func_scope() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:33:22
    #2 0x5588bfe90804 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:64:3
    ceph#3 0x7f23081c1d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x5588bfe532de in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_on_exit+0xa52de) (BuildId: 7c7a92bf5719592938c5307214bcd9b080bd847f)
    #1 0x5588bfe91160 in func_scope() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:29:22
    #2 0x5588bfe90804 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:64:3
    ceph#3 0x7f23081c1d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, instead of allocating the variables using `malloc()`,
we keep them in static variables, so that they can be accessed by
`OnExitManager` even if it is a static variable.
with this change, the memory leak reports for this source file go away.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
ljflores pushed a commit that referenced this pull request Apr 8, 2024
we allocate a hitset without freeing it in this test, and LeakSanitizer
points this out
```
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x557a0841a9dd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_hitset+0x1ae9dd) (BuildId: ad9be2b52b3d6fb1a567b262c3becaab6373e88d)
    #1 0x557a0843b98e in ExplicitHashHitSetTest::ExplicitHashHitSetTest() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/hitset.cc:128:46
    #2 0x557a0843b918 in ExplicitHashHitSetTest_Construct_Test::ExplicitHashHitSetTest_Construct_Test() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/hitset.cc:133:1
    ceph#3 0x557a0843b8cb in testing::internal::TestFactoryImpl<ExplicitHashHitSetTest_Construct_Test>::CreateTest() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
    ceph#4 0x557a08532406 in testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#5 0x557a084eb892 in testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#6 0x557a0849dd55 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2848:22
    ceph#7 0x557a0849f3bb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#8 0x557a084bc848 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#9 0x557a0853a6d6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#10 0x557a084f1222 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#11 0x557a084bbbd2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#12 0x557a08441c10 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#13 0x557a08441ba1 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    ceph#14 0x7faa493d6d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we just free it in the dtor. this should address
the warning from the sanitizer.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
rzarzynski pushed a commit that referenced this pull request Apr 16, 2024
When sanitizer is enabled, unittest_mds_quiesce_agent fails as following

```
[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from QuiesceAgentTest
[ RUN      ] QuiesceAgentTest.ThreadManagement
[       OK ] QuiesceAgentTest.ThreadManagement (3 ms)
[ RUN      ] QuiesceAgentTest.DbUpdates
[       OK ] QuiesceAgentTest.DbUpdates (1 ms)
[ RUN      ] QuiesceAgentTest.QuiesceProtocol
[       OK ] QuiesceAgentTest.QuiesceProtocol (3 ms)
[ RUN      ] QuiesceAgentTest.DuplicateQuiesceRequest
[       OK ] QuiesceAgentTest.DuplicateQuiesceRequest (2 ms)
[ RUN      ] QuiesceAgentTest.TimeoutBeforeComplete
[       OK ] QuiesceAgentTest.TimeoutBeforeComplete (2 ms)
[----------] 5 tests from QuiesceAgentTest (11 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test suite ran. (11 ms total)
[  PASSED  ] 5 tests.

=================================================================
==3975692==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0xaaaadd81c7c8 in operator new(unsigned long) (/root/ceph/build/bin/unittest_mds_quiesce_agent+0x1fc7c8) (BuildId: 7d45344ba1e43661d9de484f0a5d129377c4d4ae)
    #1 0xaaaadd8878d8 in QuiesceAgent::agent_thread_main() /root/ceph/src/mds/QuiesceAgent.cc:136:68
    #2 0xaaaadd86de38 in QuiesceAgent::AgentThread::entry() /root/ceph/src/mds/QuiesceAgent.h:244:24
    ceph#3 0xffff83d6b554 in Thread::entry_wrapper() /root/ceph/src/common/Thread.cc:87:10
    ceph#4 0xffff83d6b314 in Thread::_entry_func(void*) /root/ceph/src/common/Thread.cc:74:29
    ceph#5 0xffff8154d5c4 in start_thread nptl/./nptl/pthread_create.c:442:8
    ceph#6 0xffff815b5ed8  misc/../sysdeps/unix/sysv/linux/aarch64/clone.S:79

Indirect leak of 120 byte(s) in 1 object(s) allocated from:
    #0 0xaaaadd81c7c8 in operator new(unsigned long) (/root/ceph/build/bin/unittest_mds_quiesce_agent+0x1fc7c8) (BuildId: 7d45344ba1e43661d9de484f0a5d129377c4d4ae)
    #1 0xaaaadd8af4f4 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0xaaaadd8af3d8 in std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    ceph#3 0xaaaadd8af3d8 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    ceph#4 0xaaaadd8aef00 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/allocated_ptr.h:98:21
    ceph#5 0xaaaadd8aec14 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(QuiesceAgent::TrackedRoot*&, std::_Sp_alloc_shared_tag<std::allocator<QuiesceAgent::TrackedRoot> >, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:648:19
    ceph#6 0xaaaadd8ae988 in std::__shared_ptr<QuiesceAgent::TrackedRoot, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<QuiesceAgent::TrackedRoot>, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(std::_Sp_alloc_shared_tag<std::allocator<QuiesceAgent::TrackedRoot> >, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1342:14
    ceph#7 0xaaaadd8ae70c in std::shared_ptr<QuiesceAgent::TrackedRoot>::shared_ptr<std::allocator<QuiesceAgent::TrackedRoot>, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(std::_Sp_alloc_shared_tag<std::allocator<QuiesceAgent::TrackedRoot> >, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:409:4
    ceph#8 0xaaaadd8ae484 in std::shared_ptr<QuiesceAgent::TrackedRoot> std::allocate_shared<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(std::allocator<QuiesceAgent::TrackedRoot> const&, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:862:14
    ceph#9 0xaaaadd88ff0c in std::shared_ptr<QuiesceAgent::TrackedRoot> std::make_shared<QuiesceAgent::TrackedRoot, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:878:14
    ceph#10 0xaaaadd884a6c in QuiesceAgent::db_update(QuiesceMap&) /root/ceph/src/mds/QuiesceAgent.cc:60:26
    ceph#11 0xaaaadd84a840 in QuiesceAgentTest::update(QuiesceDbVersion, std::initializer_list<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, QuiesceMap::RootInfo> >) /root/ceph/src/test/mds/TestQuiesceAgent.cc:156:18
    ceph#12 0xaaaadd84985c in QuiesceAgentTest::update(unsigned long, std::initializer_list<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, QuiesceMap::RootInfo> >) /root/ceph/src/test/mds/TestQuiesceAgent.cc:165:14
    ceph#13 0xaaaadd8288a8 in QuiesceAgentTest_DbUpdates_Test::TestBody() /root/ceph/src/test/mds/TestQuiesceAgent.cc:213:16
    ceph#14 0xaaaadd977230 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#15 0xaaaadd924590 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#16 0xaaaadd8d4a40 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#17 0xaaaadd8d6984 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#18 0xaaaadd8d7f84 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#19 0xaaaadd8f3d48 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#20 0xaaaadd981130 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#21 0xaaaadd92bb64 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#22 0xaaaadd8f31c0 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#23 0xaaaadd820710 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#24 0xaaaadd81ed3c in main /root/ceph/src/test/unit.cc:45:10
    ceph#25 0xffff814f73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#26 0xffff814f74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#27 0xaaaadd76e6ac in _start (/root/ceph/build/bin/unittest_mds_quiesce_agent+0x14e6ac) (BuildId: 7d45344ba1e43661d9de484f0a5d129377c4d4ae)

SUMMARY: AddressSanitizer: 184 byte(s) leaked in 2 allocation(s).
```

quiesce_requests Context should be freed.

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
rzarzynski pushed a commit that referenced this pull request Apr 16, 2024
When sanitizer is enabled, unittest__rgw_crypto shows

```
=================================================================
==136464==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 75023 byte(s) in 22 object(s) allocated from:
    #0 0xaaaabf7fb86c in operator new[](unsigned long) (/root/ceph/build/bin/unittest_rgw_crypto+0x48b86c) (BuildId: 8023dc30820215da92d6d4883620bedd8ac1190d)
    #1 0xaaaabf81db48 in TestRGWCrypto_verify_Encrypt_Decrypt_Test::TestBody() /root/ceph/src/test/rgw/test_rgw_crypto.cc:780:24
    #2 0xaaaabf9018ac in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#3 0xaaaabf8b08a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#4 0xaaaabf861f88 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#5 0xaaaabf863ecc in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#6 0xaaaabf8654cc in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#7 0xaaaabf881290 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#8 0xaaaabf90b7ac in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#9 0xaaaabf8b7ac0 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#10 0xaaaabf880708 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#11 0xaaaabf823d70 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#12 0xaaaabf81f390 in main /root/ceph/src/test/rgw/test_rgw_crypto.cc:822:10
    ceph#13 0xffff878673f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#14 0xffff878674c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#15 0xaaaabf74d62c in _start (/root/ceph/build/bin/unittest_rgw_crypto+0x3dd62c) (BuildId: 8023dc30820215da92d6d4883620bedd8ac1190d)

SUMMARY: AddressSanitizer: 75023 byte(s) leaked in 22 allocation(s).
```

test_in should be freed to address the warning.

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
rzarzynski pushed a commit that referenced this pull request Apr 16, 2024
When sanitizer is enabled, unittest_bluestore_types fails as following
```
[ RUN      ] sb_info_space_efficient_map_t.basic
=================================================================
==143714==ERROR: AddressSanitizer: heap-buffer-overflow on address 0xffff99f8b7f4 at pc 0xaaaab50bde18 bp 0xffffebefcdb0 sp 0xffffebefcda8
READ of size 8 at 0xffff99f8b7f4 thread T0
    #0 0xaaaab50bde14 in sb_info_t::get_sbid() const /root/ceph/src/os/bluestore/bluestore_types.h:1337:30
    #1 0xaaaab50a5908 in sb_info_space_efficient_map_t::find(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1385:10
    #2 0xaaaab50bd638 in sb_info_space_efficient_map_t::_add(long) /root/ceph/src/os/bluestore/bluestore_types.h:1424:15
    ceph#3 0xaaaab50a52bc in sb_info_space_efficient_map_t::add_maybe_stray(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1358:12
    ceph#4 0xaaaab4fec03c in sb_info_space_efficient_map_t_basic_Test::TestBody() /root/ceph/src/test/objectstore/test_bluestore_types.cc:113:11
    ceph#5 0xaaaab51e9a40 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#6 0xaaaab5197040 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#7 0xaaaab51488a4 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#8 0xaaaab514a7e8 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#9 0xaaaab514bde8 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#10 0xaaaab5167bac in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#11 0xaaaab51f3940 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#12 0xaaaab519e5d8 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#13 0xaaaab5167024 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#14 0xaaaab50b4d6c in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#15 0xaaaab50a1080 in main /root/ceph/src/test/objectstore/test_bluestore_types.cc:2847:10
    ceph#16 0xffff9d6c73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#17 0xffff9d6c74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#18 0xaaaab4f3812c in _start (/root/ceph/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)

0xffff99f8b7f4 is located 0 bytes to the right of 20-byte region [0xffff99f8b7e0,0xffff99f8b7f4)
allocated by thread T0 here:
    #0 0xaaaab4fe636c in operator new[](unsigned long) (/root/ceph/build/bin/unittest_bluestore_types+0xef636c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)
    #1 0xaaaab50c0d2c in mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t>::allocate(unsigned long, void*) /root/ceph/src/include/mempool.h:375:33
    #2 0xaaaab50c0c0c in std::allocator_traits<mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::allocate(mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t>&, unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:318:20
    ceph#3 0xaaaab50c044c in std::_Vector_base<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:346:20
    ceph#4 0xaaaab50bf954 in void std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::_M_realloc_insert<long&>(__gnu_cxx::__normal_iterator<sb_info_t*, std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> > >, long&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:440:33
    ceph#5 0xaaaab50be0d8 in sb_info_t& std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::emplace_back<long&>(long&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:121:4
    ceph#6 0xaaaab50bd760 in sb_info_space_efficient_map_t::_add(long) /root/ceph/src/os/bluestore/bluestore_types.h:1429:24
    ceph#7 0xaaaab50a5e78 in sb_info_space_efficient_map_t::add_or_adopt(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1361:15
    ceph#8 0xaaaab4feb07c in sb_info_space_efficient_map_t_basic_Test::TestBody() /root/ceph/src/test/objectstore/test_bluestore_types.cc:103:11
    ceph#9 0xaaaab51e9a40 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#10 0xaaaab5197040 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#11 0xaaaab51488a4 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#12 0xaaaab514a7e8 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#13 0xaaaab514bde8 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#14 0xaaaab5167bac in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#15 0xaaaab51f3940 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#16 0xaaaab519e5d8 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#17 0xaaaab5167024 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#18 0xaaaab50b4d6c in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#19 0xaaaab50a1080 in main /root/ceph/src/test/objectstore/test_bluestore_types.cc:2847:10
    ceph#20 0xffff9d6c73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#21 0xffff9d6c74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#22 0xaaaab4f3812c in _start (/root/ceph/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)

SUMMARY: AddressSanitizer: heap-buffer-overflow /root/ceph/src/os/bluestore/bluestore_types.h:1337:30 in sb_info_t::get_sbid() const
Shadow bytes around the buggy address:
  0x200ff33f16a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f16b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f16c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f16d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f16e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x200ff33f16f0: fa fa fa fa fa fa fa fa fa fa fa fa 00 00[04]fa
  0x200ff33f1700: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f1710: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f1720: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f1730: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f1740: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==143714==ABORTING
```

'it' might be invalid, so before using 'it', need to figure validity out

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
rzarzynski pushed a commit that referenced this pull request Apr 16, 2024
When sanitizer is enabled, unittest_osdscrub shows

```
=================================================================
==1633952==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 28 byte(s) in 1 object(s) allocated from:
    #0 0xaaaab4e108e0 in malloc (/root/ceph/build/bin/unittest_osdscrub+0x1ed08e0) (BuildId: b3cfa2137be96d75535beecf0f2500cec10c7550)
    #1 0xffffa8cac2f8 in __res_context_send resolv/./resolv/res_send.c:334:9
    #2 0xffffa8ca9c54 in __res_context_query resolv/./resolv/res_query.c:216:6
    ceph#3 0xffffa8caa4a8 in __res_context_querydomain resolv/./resolv/res_query.c:625:9
    ceph#4 0xffffa8caa4a8 in __res_context_search resolv/./resolv/res_query.c:381:9
    ceph#5 0xffffa8caaa20 in context_search_common resolv/./resolv/res_query.c:550:16
    ceph#6 0xffffa8caaa20 in res_nsearch resolv/./resolv/res_query.c:563:10
    ceph#7 0xffffabbf1f64 in ceph::ResolvHWrapper::res_nsearch(__res_state*, char const*, int, int, unsigned char*, int) /root/ceph/src/common/dns_resolve.cc:37:10
    ceph#8 0xffffabbf6574 in ceph::DNSResolver::resolve_srv_hosts(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ceph::DNSResolver::SRV_Protocol, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::DNSResolver::Record, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::DNSResolver::Record> > >*) /root/ceph/src/common/dns_resolve.cc:295:19
    ceph#9 0xffffac8edaf0 in MonMap::init_with_dns_srv(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, std::ostream&) /root/ceph/src/mon/MonMap.cc:935:36
    ceph#10 0xffffac8eeec8 in MonMap::build_initial(ceph::common::CephContext*, bool, std::ostream&) /root/ceph/src/mon/MonMap.cc:1014:20
    ceph#11 0xffffac85beb0 in MonClient::build_initial_monmap() /root/ceph/src/mon/MonClient.cc:93:18
    ceph#12 0xaaaab4e50d98 in TestOSDScrub_scrub_time_permit_Test::TestBody() /root/ceph/src/test/osd/TestOSDScrub.cc:73:6
    ceph#13 0xaaaab4f655b0 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#14 0xaaaab4f16264 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#15 0xaaaab4ec6ca8 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#16 0xaaaab4ec8bec in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#17 0xaaaab4eca1ec in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#18 0xaaaab4ee5fb0 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#19 0xaaaab4f6f4c4 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#20 0xaaaab4f1d4bc in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#21 0xaaaab4ee5428 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#22 0xaaaab4e4b790 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#23 0xaaaab4e49dbc in main /root/ceph/src/test/unit.cc:45:10
    ceph#24 0xffffa8bc73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#25 0xffffa8bc74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#26 0xaaaab4d9972c in _start (/root/ceph/build/bin/unittest_osdscrub+0x1e5972c) (BuildId: b3cfa2137be96d75535beecf0f2500cec10c7550)

-----------------------------------------------------
Suppressions used:
  count      bytes template
      1         45 ^MallocExtension::Initialize
-----------------------------------------------------

SUMMARY: AddressSanitizer: 28 byte(s) leaked in 1 allocation(s).
```

1. 'res_ninit/res_nquery' memory should be freed.

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
rzarzynski pushed a commit that referenced this pull request Apr 23, 2024
…ntrusive_ptr get

When sanitizer is on, unittest_rgw_iam_policy shows

```
=================================================================
==2957634==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 9936 byte(s) in 1 object(s) allocated from:
    #0 0xaaaae4496248 in operator new(unsigned long) (/root/ceph-19.0.0/build/bin/unittest_bluestore_types+0xef6248) (BuildId: f63b15ff165fbd81074275a31e37535a1938809e)
    #1 0xaaaae46b8624 in BlueStore::OnodeCacheShard::create(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::common::PerfCounters*) /root/ceph-19.0.0/src/os/bluestore/BlueStore.cc:1187:7
    #2 0xaaaae44fa8a0 in Blob_put_ref_Test::TestBody() /root/ceph-19.0.0/src/test/objectstore/test_bluestore_types.cc:883:38
    ceph#3 0xaaaae4699aa8 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#4 0xaaaae46470a8 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#5 0xaaaae45f890c in testing::Test::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#6 0xaaaae45fa850 in testing::TestInfo::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#7 0xaaaae45fbe50 in testing::TestSuite::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#8 0xaaaae4617c14 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#9 0xaaaae46a39a8 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#10 0xaaaae464e640 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#11 0xaaaae461708c in testing::UnitTest::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#12 0xaaaae4564dd4 in RUN_ALL_TESTS() /root/ceph-19.0.0/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#13 0xaaaae4551080 in main /root/ceph-19.0.0/src/test/objectstore/test_bluestore_types.cc:2847:10
    ceph#14 0xffff998b73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#15 0xffff998b74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#16 0xaaaae43e812c in _start (/root/ceph-19.0.0/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: f63b15ff165fbd81074275a31e37535a1938809e)

Direct leak of 9936 byte(s) in 1 object(s) allocated from:
    #0 0xaaaae4496248 in operator new(unsigned long) (/root/ceph-19.0.0/build/bin/unittest_bluestore_types+0xef6248) (BuildId: f63b15ff165fbd81074275a31e37535a1938809e)
    #1 0xaaaae46b8624 in BlueStore::OnodeCacheShard::create(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::common::PerfCounters*) /root/ceph-19.0.0/src/os/bluestore/BlueStore.cc:1187:7
    #2 0xaaaae458ad68 in ExtentMapFixture::ExtentMapFixture() /root/ceph-19.0.0/src/test/objectstore/test_bluestore_types.cc:1307:10
    ceph#3 0xaaaae4595414 in ExtentMapFixture_pollock_Test::ExtentMapFixture_pollock_Test() /root/ceph-19.0.0/src/test/objectstore/test_bluestore_types.cc:1572:1
    ceph#4 0xaaaae4595304 in testing::internal::TestFactoryImpl<ExtentMapFixture_pollock_Test>::CreateTest() /root/ceph-19.0.0/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
    ceph#5 0xaaaae469a054 in testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#6 0xaaaae4647fcc in testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#7 0xaaaae45fa82c in testing::TestInfo::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2848:22
    ceph#8 0xaaaae45fbe50 in testing::TestSuite::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#9 0xaaaae4617c14 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#10 0xaaaae46a39a8 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#11 0xaaaae464e640 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#12 0xaaaae461708c in testing::UnitTest::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#13 0xaaaae4564dd4 in RUN_ALL_TESTS() /root/ceph-19.0.0/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#14 0xaaaae4551080 in main /root/ceph-19.0.0/src/test/objectstore/test_bluestore_types.cc:2847:10
    ceph#15 0xffff998b73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#16 0xffff998b74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#17 0xaaaae43e812c in _start (/root/ceph-19.0.0/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: f63b15ff165fbd81074275a31e37535a1938809e)

Direct leak of 9936 byte(s) in 1 object(s) allocated from:
    #0 0xaaaae4496248 in operator new(unsigned long) (/root/ceph-19.0.0/build/bin/unittest_bluestore_types+0xef6248) (BuildId: f63b15ff165fbd81074275a31e37535a1938809e)
    #1 0xaaaae46b8624 in BlueStore::OnodeCacheShard::create(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::common::PerfCounters*) /root/ceph-19.0.0/src/os/bluestore/BlueStore.cc:1187:7
    #2 0xaaaae45102ac in ExtentMap_has_any_lextents_Test::TestBody() /root/ceph-19.0.0/src/test/objectstore/test_bluestore_types.cc:1154:36
    ceph#3 0xaaaae4699aa8 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#4 0xaaaae46470a8 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#5 0xaaaae45f890c in testing::Test::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2680:5
    ceph#6 0xaaaae45fa850 in testing::TestInfo::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2858:11
    ceph#7 0xaaaae45fbe50 in testing::TestSuite::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#8 0xaaaae4617c14 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#9 0xaaaae46a39a8 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#10 0xaaaae464e640 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#11 0xaaaae461708c in testing::UnitTest::Run() /root/ceph-19.0.0/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#12 0xaaaae4564dd4 in RUN_ALL_TESTS() /root/ceph-19.0.0/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#13 0xaaaae4551080 in main /root/ceph-19.0.0/src/test/objectstore/test_bluestore_types.cc:2847:10
    ceph#14 0xffff998b73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#15 0xffff998b74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#16 0xaaaae43e812c in _start (/root/ceph-19.0.0/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: f63b15ff165fbd81074275a31e37535a1938809e)

...
...
...
Indirect leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0xaaaacf905688 in operator new(unsigned long) (/root/ceph/build/bin/unittest_rgw_iam_policy+0x25f5688) (BuildId: c8dd019430a8d0162485760a78979347f29d225f)
    #1 0xffff7f0b7ca8 in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /root/ceph/src/common/ceph_context.cc:781:24
    #2 0xffff7f0b514c in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /root/ceph/src/common/ceph_context.cc:697:5
    ceph#3 0xaaaacf997dcc in ManagedPolicyTest::ManagedPolicyTest() /root/ceph/src/test/rgw/test_rgw_iam_policy.cc:788:33
    ceph#4 0xaaaacf998294 in ManagedPolicyTest_IAMReadOnlyAccess_Test::ManagedPolicyTest_IAMReadOnlyAccess_Test() /root/ceph/src/test/rgw/test_rgw_iam_policy.cc:802:1
    ceph#5 0xaaaacf998230 in testing::internal::TestFactoryImpl<ManagedPolicyTest_IAMReadOnlyAccess_Test>::CreateTest() /root/ceph/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
    ceph#6 0xaaaacfb1a2d4 in testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#7 0xaaaacfaced5c in testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#8 0xaaaacfa83274 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2848:22
    ceph#9 0xaaaacfa84898 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#10 0xaaaacfaa065c in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#11 0xaaaacfb23b9c in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#12 0xaaaacfad4968 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#13 0xaaaacfa9fad4 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#14 0xaaaacfa23588 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#15 0xaaaacfa23504 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    ceph#16 0xffff7c5a73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#17 0xffff7c5a74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#18 0xaaaacf85756c in _start (/root/ceph/build/bin/unittest_rgw_iam_policy+0x254756c) (BuildId: c8dd019430a8d0162485760a78979347f29d225f)

Indirect leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0xaaaacf905688 in operator new(unsigned long) (/root/ceph/build/bin/unittest_rgw_iam_policy+0x25f5688) (BuildId: c8dd019430a8d0162485760a78979347f29d225f)
    #1 0xffff7f0b7ca8 in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /root/ceph/src/common/ceph_context.cc:781:24
    #2 0xffff7f0b514c in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /root/ceph/src/common/ceph_context.cc:697:5
    ceph#3 0xaaaacf997dcc in ManagedPolicyTest::ManagedPolicyTest() /root/ceph/src/test/rgw/test_rgw_iam_policy.cc:788:33
    ceph#4 0xaaaacf997d08 in ManagedPolicyTest_IAMFullAccess_Test::ManagedPolicyTest_IAMFullAccess_Test() /root/ceph/src/test/rgw/test_rgw_iam_policy.cc:791:1
    ceph#5 0xaaaacf997ca4 in testing::internal::TestFactoryImpl<ManagedPolicyTest_IAMFullAccess_Test>::CreateTest() /root/ceph/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
    ceph#6 0xaaaacfb1a2d4 in testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#7 0xaaaacfaced5c in testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#8 0xaaaacfa83274 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2848:22
    ceph#9 0xaaaacfa84898 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    ceph#10 0xaaaacfaa065c in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    ceph#11 0xaaaacfb23b9c in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    ceph#12 0xaaaacfad4968 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    ceph#13 0xaaaacfa9fad4 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    ceph#14 0xaaaacfa23588 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    ceph#15 0xaaaacfa23504 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    ceph#16 0xffff7c5a73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ceph#17 0xffff7c5a74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    ceph#18 0xaaaacf85756c in _start (/root/ceph/build/bin/unittest_rgw_iam_policy+0x254756c) (BuildId: c8dd019430a8d0162485760a78979347f29d225f)

SUMMARY: AddressSanitizer: 67625436 byte(s) leaked in 21960 allocation(s).
```

To avoid memory not free, use reset for not increasing RC and keep code uniformity as above codes.

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
Matan-B pushed a commit that referenced this pull request Apr 30, 2024
When sanitizer is enabled, result shows as following

```
=================================================================
==207167==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac50ff7dc in PITest_past_intervals_ec_Test::TestBody() /root/ceph/src/test/osd/types.cc:1707:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac510ac3c in PITest_past_intervals_ec_no_subsets_Test::TestBody() /root/ceph/src/test/osd/types.cc:1799:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac50fce54 in PITest_past_intervals_rep_Test::TestBody() /root/ceph/src/test/osd/types.cc:1683:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac5107e3c in PITest_past_intervals_rep_no_subsets_Test::TestBody() /root/ceph/src/test/osd/types.cc:1776:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac5107e3c in PITest_past_intervals_rep_no_subsets_Test::TestBody() /root/ceph/src/test/osd/types.cc:1776:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac51054d0 in PITest_past_intervals_ec_down_Test::TestBody() /root/ceph/src/test/osd/types.cc:1753:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac51054d0 in PITest_past_intervals_ec_down_Test::TestBody() /root/ceph/src/test/osd/types.cc:1753:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac510248c in PITest_past_intervals_rep_down_Test::TestBody() /root/ceph/src/test/osd/types.cc:1729:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac51109b0 in PITest_past_intervals_rep_lost_Test::TestBody() /root/ceph/src/test/osd/types.cc:1846:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac51109b0 in PITest_past_intervals_rep_lost_Test::TestBody() /root/ceph/src/test/osd/types.cc:1846:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac510dc88 in PITest_past_intervals_ec_no_subsets2_Test::TestBody() /root/ceph/src/test/osd/types.cc:1822:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac510dc88 in PITest_past_intervals_ec_no_subsets2_Test::TestBody() /root/ceph/src/test/osd/types.cc:1822:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac510ac3c in PITest_past_intervals_ec_no_subsets_Test::TestBody() /root/ceph/src/test/osd/types.cc:1799:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac50fce54 in PITest_past_intervals_rep_Test::TestBody() /root/ceph/src/test/osd/types.cc:1683:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac5113760 in PITest_past_intervals_ec_lost_Test::TestBody() /root/ceph/src/test/osd/types.cc:1870:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac5113760 in PITest_past_intervals_ec_lost_Test::TestBody() /root/ceph/src/test/osd/types.cc:1870:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5149264 in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1673:7
    #2 0xaaaac50ff7dc in PITest_past_intervals_ec_Test::TestBody() /root/ceph/src/test/osd/types.cc:1707:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac50a2c48 in operator new(unsigned long) (/root/ceph/build/bin/unittest_osd_types+0x242c48) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)
    #1 0xaaaac5148fcc in PITest::run(bool, std::__cxx11::list<PastIntervals::pg_interval_t, std::allocator<PastIntervals::pg_interval_t> >, unsigned int, unsigned int, std::vector<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> >, std::allocator<std::pair<int, std::pair<PastIntervals::osd_state_t, unsigned int> > > >, std::vector<int, std::allocator<int> >, std::vector<int, std::allocator<int> >, std::set<pg_shard_t, std::less<pg_shard_t>, std::allocator<pg_shard_t> >, std::set<int, std::less<int>, std::allocator<int> >, std::map<int, unsigned int, std::less<int>, std::allocator<std::pair<int const, unsigned int> > >, bool) /root/ceph/src/test/osd/types.cc:1664:7
    #2 0xaaaac510248c in PITest_past_intervals_rep_down_Test::TestBody() /root/ceph/src/test/osd/types.cc:1729:3
    #3 0xaaaac531ee18 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #4 0xaaaac52d27bc in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #5 0xaaaac5285848 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #6 0xaaaac528778c in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #7 0xaaaac5288d8c in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0xaaaac52a4b50 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0xaaaac5326548 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaac52d9500 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaac52a3fc8 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0xaaaac5226f84 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0xaaaac5226f00 in main /root/ceph/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0xffff930273f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #15 0xffff930274c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #16 0xaaaac4ff4b2c in _start (/root/ceph/build/bin/unittest_osd_types+0x194b2c) (BuildId: 54d356bd0f2daa8bb228f480da3ed4fd9ac8f632)

SUMMARY: AddressSanitizer: 288 byte(s) leaked in 18 allocation(s).
```

PriorSet::pcontdec should be freed

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
Matan-B pushed a commit that referenced this pull request May 1, 2024
ASan warns
```
==445793==ERROR: AddressSanitizer: alloc-dealloc-mismatch (malloc vs operator delete) on 0x602000039b10
    #0 0x5604a544112d in operator delete(void*) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_erasure_code_shec_all+0x1e012d) (BuildId: 8cfc74d22471b6905f9b23304aed2af945265a13)
    #1 0x7fc14752f588 in ErasureCodeShecTableCache::~ErasureCodeShecTableCache() /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/shec/ErasureCodeShecTableCache.cc:61:19
    #2 0x5604a544ccbe in ParameterTest_parameter_all_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/TestErasureCodeShec_all.cc:263:1
...
0x602000039b10 is located 0 bytes inside of 4-byte region [0x602000039b10,0x602000039b14)
allocated by thread T0 here:
    #0 0x5604a5405afe in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_erasure_code_shec_all+0x1a4afe) (BuildId: 8cfc74d22471b6905f9b23304aed2af945265a13)
    #1 0x7fc1474c9617 in reed_sol_vandermonde_coding_matrix /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/jerasure/jerasure/src/reed_sol.c:86:10
    #2 0x7fc147528634 in ErasureCodeShec::shec_reedsolomon_coding_matrix(int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/shec/ErasureCodeShec.cc:514:12
    ceph#3 0x7fc147526cd8 in ErasureCodeShecReedSolomonVandermonde::prepare() /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/shec/ErasureCodeShec.cc:390:14
    ceph#4 0x7fc1475187aa in ErasureCodeShec::init(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&, std::ostream*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/shec/ErasureCodeShec.cc:57:3
```

where we use `delete` to free the encoder matrix allocated using
`malloc()`. as jerasure is a library implemented in C language,
unless we want to reimplment it in C++, we should use `free()` to
free the memory chunk allocated by
`reed_sol_vandermonde_coding_matrix()`. also, please note,
jerasure does not provide a function to free the memory allocated
by this function, we have to explore its implementation, and use
`malloc()` directly. this should silence the ASan warning.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Matan-B pushed a commit that referenced this pull request May 1, 2024
in TestErasureCodeLrc.cc, we create a `CrushWrapper` without
destroying and freeing it. and ASan points this out:
```
Indirect leak of 72000 byte(s) in 1000 object(s) allocated from:
    #0 0x559e00acc2bd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_erasure_code_lrc+0x1ed2bd) (BuildId: 1cd93d0f231006242239eb0d81a8d677e36aeba2)
    #1 0x559e00b2bb76 in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x559e00b2bb00 in std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    ceph#3 0x559e00b2bb00 in std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > > >::allocate(std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    ceph#4 0x559e00b2b9d1 in std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_get_node() /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:561:16
    ceph#5 0x559e00b2aae4 in std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >* std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_create_node<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:611:23
    ceph#6 0x559e00b29520 in std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:2431:19
    ceph#7 0x559e00b255c2 in std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_map.h:501:15
    ceph#8 0x559e00b188da in CrushWrapper::set_item_name(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/crush/CrushWrapper.h:466:7
    ceph#9 0x7feb54a4dcc2 in CrushWrapper::insert_item(ceph::common::CephContext*, int, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/crush/CrushWrapper.cc:1119:5
    ceph#10 0x559e00ada922 in ErasureCodeTest_create_rule_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/TestErasureCodeLrc.cc:127:5
    ceph#11 0x559e00c0e0a6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
```

so, in this change, let's manage the lifecycle of the `CrushWrapper`
instance with a smart pointer, so that it is destroyed and free'd
properly, and this should silence the ASan warning.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants