2023.03.22 15:30:55.022734 [ 60 ] {} <Error> business.asset_log_local::49e1a074a7d9fb28e20290014823a76e_0_1199_785 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 241. DB::Exception: Memory limit (total) exceeded: would use 183.54 GiB (attempt to allocate chunk of 4994983 bytes), maximum: 172.80 GiB. OvercommitTracker decision: Memory overcommit isn't used. Waiting time or overcommit denominator are set to zero.: (while reading column data): While executing MergeTreeSequentialSource. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):
0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0xe0c67d5 in /usr/bin/clickhouse
1. ? @ 0xe0d20d1 in /usr/bin/clickhouse
2. MemoryTracker::allocImpl(long, bool, MemoryTracker*) @ 0xe0d1ab7 in /usr/bin/clickhouse
3. MemoryTracker::allocImpl(long, bool, MemoryTracker*) @ 0xe0d13d2 in /usr/bin/clickhouse
4. MemoryTracker::allocImpl(long, bool, MemoryTracker*) @ 0xe0d13d2 in /usr/bin/clickhouse
5. CurrentMemoryTracker::realloc(long, long) @ 0xe096e5f in /usr/bin/clickhouse
6. Allocator<false, false>::realloc(void*, unsigned long, unsigned long, unsigned long) @ 0x82c5187 in /usr/bin/clickhouse
7. ? @ 0xe127980 in /usr/bin/clickhouse
8. DB::CompressedReadBufferFromFile::nextImpl() @ 0x127e4f11 in /usr/bin/clickhouse
9. ? @ 0x1298de66 in /usr/bin/clickhouse
10. DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::__1::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&, std::__1::unordered_map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>*) const @ 0x1295484b in /usr/bin/clickhouse
11. DB::MergeTreeReaderCompact::readData(DB::NameAndTypePair const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, unsigned long, unsigned long, unsigned long, std::__1::optional<unsigned long>) @ 0x143bf79f in /usr/bin/clickhouse
12. DB::MergeTreeReaderCompact::readRows(unsigned long, unsigned long, bool, unsigned long, std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x143bf0f3 in /usr/bin/clickhouse
13. DB::MergeTreeSequentialSource::generate() @ 0x143d1a51 in /usr/bin/clickhouse
14. DB::ISource::tryGenerate() @ 0x14747875 in /usr/bin/clickhouse
15. DB::ISource::work() @ 0x147473e6 in /usr/bin/clickhouse
16. DB::ExecutionThreadContext::executeTask() @ 0x14760d0c in /usr/bin/clickhouse
17. DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) @ 0x14755f1b in /usr/bin/clickhouse
18. DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*) @ 0x14755308 in /usr/bin/clickhouse
19. DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x14765dd7 in /usr/bin/clickhouse
20. DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x14765f93 in /usr/bin/clickhouse
21. DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x1422b26f in /usr/bin/clickhouse
22. DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x1422b1cb in /usr/bin/clickhouse
23. DB::MergeTask::execute() @ 0x142303ba in /usr/bin/clickhouse
24. DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x144be86c in /usr/bin/clickhouse
25. DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x144bd6b4 in /usr/bin/clickhouse
26. DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::__1::shared_ptr<DB::TaskRuntimeData>) @ 0x8925a8c in /usr/bin/clickhouse
27. DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x892572a in /usr/bin/clickhouse
28. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__1::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0xe198dca in /usr/bin/clickhouse
29. void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) @ 0xe19b697 in /usr/bin/clickhouse
30. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xe194f8a in /usr/bin/clickhouse
31. ? @ 0xe19a641 in /usr/bin/clickhouse
(version 23.2.3.17 (official build))
2023.03.22 15:30:55.022838 [ 1 ] {} <Information> Application: Closed connections. But 157 remain. Tip: To increase wait time add to config: <shutdown_wait_unfinished>60</shutdown_wait_unfinished>
2023.03.22 15:30:55.022903 [ 1 ] {} <Information> Application: Will shutdown forcefully.
Describe the unexpected behaviour
As described in title, memory consumption increases insanely out of acceptable and causing servers to restart.
How to reproduce
ClickHouse Server 23.2.3.17 (Official Build)
Expected behavior
memory consumption of inverted index be reasonabe.
Error message and/or stacktrace
this same error flushs my screen before server exits.