Try fix 02151_hash_table_sizes_stats.sh test

[nickitat]

Changelog category (leave one):

• Not for changelog (changelog entry is not required)

[tavplubix]

Our memory trackers work with sanitizers as well. So why does OOM happen?

[nickitat]

I have no idea, let's at least reduce the amount of noise

[tavplubix]

Let's try to debug it. OOMs like this usually happen when we allocate a huge chunk of memory without checking that it doesn't exceed limitations. In other words, when we call CurrentMemoryTracker::allocNoThrow instead of CurrentMemoryTracker::alloc for huge allocations. In other words, when we use new or malloc for huge allocations without additional checks (malloc and new call allocNoThrow because they cannot throw). I quickly looked through Aggregator and found two suspicious places:

ClickHouse/src/Interpreters/Aggregator.cpp

Line 1175 in 73e98de

std::unique_ptr<AggregateDataPtr[]> places(new AggregateDataPtr[row_end]);

ClickHouse/src/Interpreters/Aggregator.cpp

Line 2809 in 73e98de

std::unique_ptr<AggregateDataPtr[]> places(new AggregateDataPtr[row_end]);

Where row_end is actually a block size and can be quite big. I tried increasing max_block_size and got an interesting result:
https://pastila.nl/?00b0c940/f011d9e7124b0c30b20cdb4be9872be6

At first, it logs

[ 982431 ] ... <Debug> MemoryTracker: Current memory usage (for query): 16.00 GiB.

and after that it throw an exception in another thread:

Memory limit (for user) exceeded: would use 16.00 GiB (attempt to allocate chunk of 17179953272 bytes), maximum: 953.67 MiB

The maximum was about 1GiB, but looks like Aggregator had allocated 16GiB.

Can it be the reason? Do we have other (less obvious) places in Aggregator where huge allocations may happen without proper checks?

[tavplubix]

We can merge it to suppress the issue temporarily, but the root cause must be investigated. We definitely have some bugs with memory accounting in Aggregator.