Skip to content

[Bug] ShardedLRUCache may coredump when destructor was called #10991

@liaoxin01

Description

@liaoxin01

Search before asking

  • I had searched in the issues and found no similar issues.

Version

master

What's Wrong?

==360464==ERROR: AddressSanitizer: heap-use-after-free on address 0x612000521148 at pc 0x55cc6f2ae7e7 bp 0x7f184c5f5640 sp 0x7f184c5f5630
READ of size 8 at 0x612000521148 thread T4 (calculate_metri)
#0 0x55cc6f2ae7e6 in doris::LRUCache::get_capacity() const /mnt/data1/yixiu/selectdb-core/be/src/olap/lru_cache.h:326
#1 0x55cc6f2a90f9 in doris::ShardedLRUCache::update_cache_metrics() const /mnt/data1/yixiu/selectdb-core/be/src/olap/lru_cache.cpp:534
#2 0x55cc6f2b110d in void std::_invoke_impl<void, void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&>(std::_invoke_memfun_deref, void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&) /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:74
#3 0x55cc6f2b0f7a in std::_invoke_result<void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&>::type std::_invoke<void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&>(void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&) /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:96
#4 0x55cc6f2b0eb1 in void std::Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>::_call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/data1/ldb_toolchain/include/c++/11/functional:420
#5 0x55cc6f2b0d62 in void std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>::operator()<, void>() /mnt/data1/ldb_toolchain/include/c++/11/functional:503
#6 0x55cc6f2b0a7f in void std::_invoke_impl<void, std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&>(std::_invoke_other, std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&) /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:61
#7 0x55cc6f2b04db in std::enable_if<is_invocable_r_v<void, std::Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&>, void>::type std::_invoke_r<void, std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&>(std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&) /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:111
#8 0x55cc6f2b0057 in std::_Function_handler<void (), std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const> >::_M_invoke(std::_Any_data const&) /mnt/data1/ldb_toolchain/include/c++/11/bits/std_function.h:291
#9 0x55cc7017fa75 in std::function<void ()>::operator()() const /mnt/data1/ldb_toolchain/include/c++/11/bits/std_function.h:560
#10 0x55cc708293ba in doris::MetricEntity::trigger_hook_unlocked(bool) const /mnt/data1/yixiu/selectdb-core/be/src/util/metrics.cpp:262
#11 0x55cc7082a0be in doris::MetricRegistry::trigger_all_hooks(bool) const /mnt/data1/yixiu/selectdb-core/be/src/util/metrics.cpp:311
#12 0x55cc6e893cf2 in doris::Daemon::calculate_metrics_thread() /mnt/data1/yixiu/selectdb-core/be/src/common/daemon.cpp:122
#13 0x55cc6e895d89 in operator() /mnt/data1/yixiu/selectdb-core/be/src/common/daemon.cpp:294
#14 0x55cc6e89809f in __invoke_impl<void, doris::Daemon::start()::<lambda()>&> /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:61
#15 0x55cc6e897e43 in __invoke_r<void, doris::Daemon::start()::<lambda()>&> /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:111
#16 0x55cc6e897ad9 in _M_invoke /mnt/data1/ldb_toolchain/include/c++/11/bits/std_function.h:291
#17 0x55cc7017fa75 in std::function<void ()>::operator()() const /mnt/data1/ldb_toolchain/include/c++/11/bits/std_function.h:560
#18 0x55cc7098f2f1 in doris::Thread::supervise_thread(void*) /mnt/data1/yixiu/selectdb-core/be/src/util/thread.cpp:408
#19 0x7f18529f1608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608)
#20 0x7f1852b2b132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)

What You Expected?

fix

How to Reproduce?

ShardedLRUCache::~ShardedLRUCache() {
if (_shards) {
for (int s = 0; s < _num_shards; s++) {
delete _shards[s];
}
delete[] _shards;
}
_entity->deregister_hook(_name);
DorisMetrics::instance()->metric_registry()->deregister_entity(_entity);
}

delete _shards has a competition with MetricRegistry::trigger_all_hooks.
when called in the following order, it will coredump.
delete _shards -> MetricRegistry::trigger_all_hooks->deregister

Anything Else?

All ShardedLRUCache are global in our code now, so has a small probability of coredump.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions