AggregateFunctionSumMap fails with ASAN build with RelWithDebInfo on v23.3-lts

[bharatnc]

Describe the unexpected behaviour

We are observing that that when building ClickHouse from source on our internal CI infrastructure from v23.3 branch - it's failing ASAN build with CMAKE_BUILD_TYPE=RelWithDebInfo (passes with CMAKE_BUILD_TYPE=Debug). I'm not exactly sure what is different with our environment that it doesn't fail on the upstream 23.3 branch of the official ClickHouse repo itself :)

Reverting this particular change https://github.com/ClickHouse/ClickHouse/pull/48556- which has also been ported to the latest v23.3.x-lts versions seems to resolve this issue and makes the ASAN build pass again.

Timeline: We didn't observe this on v23.3.1.2823-lts (but only after that).

Error from CI is:

19:57:52     [6148/6160] Building CXX object programs/copier/CMakeFiles/clickhouse-copier-lib.dir/ClusterCopier.cpp.o
19:58:00     [6149/6160] Building CXX object src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionUniqCombined.cpp.o
19:58:51     [6150/6160] Building CXX object src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionMax.cpp.o
20:02:41     [6151/6160] Building CXX object src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o
20:02:41     FAILED: src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o
20:02:41     prlimit --as=10000000000 --data=5000000000 --cpu=1000 /usr/bin/ccache /usr/lib/ccache/clang++-15 --target=x86_64-linux-gnu --sysroot=/build/clickhouse/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc -DANNOYLIB_MULTITHREADED_BUILD -DBOOST_ASIO_HAS_STD_INVOKE_RESULT=1 -DBOOST_ASIO_STANDALONE=1 -DENABLE_ANNOY -DENABLE_MULTITARGET_CODE=1 -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUNALIGNED_OK -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -I/build/clickhouse/obj-x86_64-linux-gnu/includes/configs -I/build/clickhouse/base/glibc-compatibility/memcpy -I/build/clickhouse/src -I/build/clickhouse/obj-x86_64-linux-gnu/src -I/build/clickhouse/obj-x86_64-linux-gnu/src/Core/include -I/build/clickhouse/base/base/.. -I/build/clickhouse/obj-x86_64-linux-gnu/base/base/.. -I/build/clickhouse/contrib/cctz/include -I/build/clickhouse/base/pcg-random/. -I/build/clickhouse/contrib/miniselect/include -I/build/clickhouse/contrib/zstd/lib -isystem /build/clickhouse/contrib/llvm-project/libcxx/include -isystem /build/clickhouse/contrib/llvm-project/libcxxabi/include -isystem /build/clickhouse/contrib/libunwind/include -isystem /build/clickhouse/contrib/abseil-cpp -isystem /build/clickhouse/contrib/croaring/cpp -isystem /build/clickhouse/contrib/croaring/include -isystem /build/clickhouse/contrib/cityhash102/include -isystem /build/clickhouse/contrib/boost -isystem /build/clickhouse/base/poco/Net/include -isystem /build/clickhouse/base/poco/Foundation/include -isystem /build/clickhouse/base/poco/NetSSL_OpenSSL/include -isystem /build/clickhouse/base/poco/Crypto/include -isystem /build/clickhouse/contrib/boringssl/include -isystem /build/clickhouse/base/poco/Util/include -isystem /build/clickhouse/base/poco/JSON/include -isystem /build/clickhouse/base/poco/XML/include -isystem /build/clickhouse/contrib/replxx/include -isystem /build/clickhouse/contrib/fmtlib-cmake/../fmtlib/include -isystem /build/clickhouse/contrib/magic_enum/include -isystem /build/clickhouse/contrib/double-conversion -isystem /build/clickhouse/contrib/dragonbox/include -isystem /build/clickhouse/contrib/re2 -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/re2-cmake -isystem /build/clickhouse/contrib/zlib-ng -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/zlib-ng-cmake -isystem /build/clickhouse/contrib/pdqsort -isystem /build/clickhouse/contrib/xz/src/liblzma/api -isystem /build/clickhouse/contrib/snappy -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/snappy-cmake -isystem /build/clickhouse/contrib/msgpack-c/include -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/liburing/src/include-compat -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/liburing/src/include -isystem /build/clickhouse/contrib/liburing/src/include -isystem /build/clickhouse/contrib/fast_float/include -isystem /build/clickhouse/contrib/NuRaft/include -isystem /build/clickhouse/contrib/nuraft-cmake/, -isystem /build/clickhouse/base/poco/MongoDB/include -isystem /build/clickhouse/contrib/s2geometry/src -isystem /build/clickhouse/contrib/rocksdb/include -isystem /build/clickhouse/contrib/ulid-c/include -isystem /build/clickhouse/contrib/consistent-hashing -isystem /build/clickhouse/contrib/annoy/src -isystem /build/clickhouse/contrib/datasketches-cpp/common/include -isystem /build/clickhouse/contrib/datasketches-cpp/theta/include --gcc-toolchain=/build/clickhouse/cmake/linux/../../contrib/sysroot/linux-x86_64 -std=c++20  -g -fno-omit-frame-pointer -DSANITIZER -fsanitize=address -fno-sanitize-address-use-after-scope -fdiagnostics-color=always -Xclang -fuse-ctor-homing -fsized-deallocation  -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -ffile-prefix-map=/build/clickhouse=. -falign-functions=32 -mbranches-within-32B-boundaries -fdiagnostics-absolute-paths -fstrict-vtable-pointers -fexperimental-new-pass-manager -Wall -Wextra -Weverything -Wpedantic -Wno-zero-length-array -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c++20-compat -Wno-sign-conversion -Wno-implicit-int-conversion -Wno-implicit-int-float-conversion -Wno-ctad-maybe-unsupported -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-padded -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -Wno-thread-safety-negative -O2 -g -DNDEBUG -O3 -g -gdwarf-4  -fno-pie   -D OS_LINUX -Werror -nostdinc++ -std=c++2b -MD -MT src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o -MF src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o.d -o src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o -c /build/clickhouse/src/AggregateFunctions/AggregateFunctionSumMap.cpp
20:02:41     LLVM ERROR: out of memory
20:02:41     Allocation failed
20:02:41     PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
20:02:41     Stack dump:
20:02:41     0.  Program arguments: /usr/bin/clang++-15 --target=x86_64-linux-gnu --sysroot=../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc --gcc-toolchain=/build/clickhouse/cmake/linux/../../contrib/sysroot/linux-x86_64 -std=c++20 -g -fno-omit-frame-pointer -fsanitize=address -fno-sanitize-address-use-after-scope -Xclang -fuse-ctor-homing -fsized-deallocation -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -ffile-prefix-map=/build/clickhouse=. -falign-functions=32 -mbranches-within-32B-boundaries -fdiagnostics-absolute-paths -fstrict-vtable-pointers -fexperimental-new-pass-manager -Wall -Wextra -Weverything -Wpedantic -Wno-zero-length-array -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c++20-compat -Wno-sign-conversion -Wno-implicit-int-conversion -Wno-implicit-int-float-conversion -Wno-ctad-maybe-unsupported -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-padded -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -Wno-thread-safety-negative -O2 -g -O3 -g -gdwarf-4 -fno-pie -std=c++2b -fdiagnostics-color=always -Werror -DANNOYLIB_MULTITHREADED_BUILD -DBOOST_ASIO_HAS_STD_INVOKE_RESULT=1 -DBOOST_ASIO_STANDALONE=1 -DENABLE_ANNOY -DENABLE_MULTITARGET_CODE=1 -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUNALIGNED_OK -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -Iincludes/configs -I../base/glibc-compatibility/memcpy -I../src -Isrc -Isrc/Core/include -I../base -Ibase -I../contrib/cctz/include -I../base/pcg-random -I../contrib/miniselect/include -I../contrib/zstd/lib -isystem ../contrib/llvm-project/libcxx/include -isystem ../contrib/llvm-project/libcxxabi/include -isystem ../contrib/libunwind/include -isystem ../contrib/abseil-cpp -isystem ../contrib/croaring/cpp -isystem ../contrib/croaring/include -isystem ../contrib/cityhash102/include -isystem ../contrib/boost -isystem ../base/poco/Net/include -isystem ../base/poco/Foundation/include -isystem ../base/poco/NetSSL_OpenSSL/include -isystem ../base/poco/Crypto/include -isystem ../contrib/boringssl/include -isystem ../base/poco/Util/include -isystem ../base/poco/JSON/include -isystem ../base/poco/XML/include -isystem ../contrib/replxx/include -isystem ../contrib/fmtlib/include -isystem ../contrib/magic_enum/include -isystem ../contrib/double-conversion -isystem ../contrib/dragonbox/include -isystem ../contrib/re2 -isystem contrib/re2-cmake -isystem ../contrib/zlib-ng -isystem contrib/zlib-ng-cmake -isystem ../contrib/pdqsort -isystem ../contrib/xz/src/liblzma/api -isystem ../contrib/snappy -isystem contrib/snappy-cmake -isystem ../contrib/msgpack-c/include -isystem contrib/liburing/src/include-compat -isystem contrib/liburing/src/include -isystem ../contrib/liburing/src/include -isystem ../contrib/fast_float/include -isystem ../contrib/NuRaft/include -isystem ../contrib/nuraft-cmake/, -isystem ../base/poco/MongoDB/include -isystem ../contrib/s2geometry/src -isystem ../contrib/rocksdb/include -isystem ../contrib/ulid-c/include -isystem ../contrib/consistent-hashing -isystem ../contrib/annoy/src -isystem ../contrib/datasketches-cpp/common/include -isystem ../contrib/datasketches-cpp/theta/include -DSANITIZER -DNDEBUG -D OS_LINUX -nostdinc++ -c -MD -MT src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o -MF src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o.d -fcolor-diagnostics -o src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o ../src/AggregateFunctions/AggregateFunctionSumMap.cpp
20:02:42     ++ upload_ccache
20:02:42     1.  <eof> parser at end of file
20:02:42     ++ ccache --show-stats
20:02:42     2.  Code generation

How to reproduce

I reproduced on dev locally with:

$:/build/clickhouse$ sudo ln -s /usr/bin/llvm-symbolizer-15 /usr/bin/llvm-symbolizer
$:/build/clickhouse$ which llvm-symbolizer
/usr/bin/llvm-symbolizer
$:/build/clickhouse$ prlimit --as=10000000000 --data=5000000000 --cpu=1000 /usr/bin/ccache /usr/lib/ccache/clang++-15 --target=x86_64-linux-gnu --sysroot=/build/clickhouse/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc -DANNOYLIB_MULTITHREADED_build -DBOOST_ASIO_HAS_STD_INVOKE_RESULT=1 -DBOOST_ASIO_STANDALONE=1 -DENABLE_ANNOY -DENABLE_MULTITARGET_CODE=1 -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUNALIGNED_OK -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -I/build/clickhouse/obj-x86_64-linux-gnu/includes/configs -I/build/clickhouse/base/glibc-compatibility/memcpy -I/build/clickhouse/src -I/build/clickhouse/obj-x86_64-linux-gnu/src -I/build/clickhouse/obj-x86_64-linux-gnu/src/Core/include -I/build/clickhouse/base/base/.. -I/build/clickhouse/obj-x86_64-linux-gnu/base/base/.. -I/build/clickhouse/contrib/cctz/include -I/build/clickhouse/base/pcg-random/. -I/build/clickhouse/contrib/miniselect/include -I/build/clickhouse/contrib/zstd/lib -isystem /build/clickhouse/contrib/llvm-project/libcxx/include -isystem /build/clickhouse/contrib/llvm-project/libcxxabi/include -isystem /build/clickhouse/contrib/libunwind/include -isystem /build/clickhouse/contrib/abseil-cpp -isystem /build/clickhouse/contrib/croaring/cpp -isystem /build/clickhouse/contrib/croaring/include -isystem /build/clickhouse/contrib/cityhash102/include -isystem /build/clickhouse/contrib/boost -isystem /build/clickhouse/base/poco/Net/include -isystem /build/clickhouse/base/poco/Foundation/include -isystem /build/clickhouse/base/poco/NetSSL_OpenSSL/include -isystem /build/clickhouse/base/poco/Crypto/include -isystem /build/clickhouse/contrib/boringssl/include -isystem /build/clickhouse/base/poco/Util/include -isystem /build/clickhouse/base/poco/JSON/include -isystem /build/clickhouse/base/poco/XML/include -isystem /build/clickhouse/contrib/replxx/include -isystem /build/clickhouse/contrib/fmtlib-cmake/../fmtlib/include -isystem /build/clickhouse/contrib/magic_enum/include -isystem /build/clickhouse/contrib/double-conversion -isystem /build/clickhouse/contrib/dragonbox/include -isystem /build/clickhouse/contrib/re2 -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/re2-cmake -isystem /build/clickhouse/contrib/zlib-ng -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/zlib-ng-cmake -isystem /build/clickhouse/contrib/pdqsort -isystem /build/clickhouse/contrib/xz/src/liblzma/api -isystem /build/clickhouse/contrib/snappy -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/snappy-cmake -isystem /build/clickhouse/contrib/msgpack-c/include -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/liburing/src/include-compat -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/liburing/src/include -isystem /build/clickhouse/contrib/liburing/src/include -isystem /build/clickhouse/contrib/fast_float/include -isystem /build/clickhouse/contrib/NuRaft/include -isystem /build/clickhouse/contrib/nuraft-cmake/, -isystem /build/clickhouse/base/poco/MongoDB/include -isystem /build/clickhouse/contrib/s2geometry/src -isystem /build/clickhouse/contrib/rocksdb/include -isystem /build/clickhouse/contrib/ulid-c/include -isystem /build/clickhouse/contrib/consistent-hashing -isystem /build/clickhouse/contrib/annoy/src -isystem /build/clickhouse/contrib/datasketches-cpp/common/include -isystem /build/clickhouse/contrib/datasketches-cpp/theta/include --gcc-toolchain=/build/clickhouse/cmake/linux/../../contrib/sysroot/linux-x86_64 -std=c++20  -g -fno-omit-frame-pointer -DSANITIZER -fsanitize=address -fno-sanitize-address-use-after-scope -fdiagnostics-color=always -Xclang -fuse-ctor-homing -fsized-deallocation  -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -ffile-prefix-map=/build/clickhouse=. -falign-functions=32 -mbranches-within-32B-boundaries -fdiagnostics-absolute-paths -fstrict-vtable-pointers -fexperimental-new-pass-manager -Wall -Wextra -Weverything -Wpedantic -Wno-zero-length-array -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c++20-compat -Wno-sign-conversion -Wno-implicit-int-conversion -Wno-implicit-int-float-conversion -Wno-ctad-maybe-unsupported -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-padded -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -Wno-thread-safety-negative -O2 -g -DNDEBUG -O3 -g -gdwarf-4  -fno-pie   -D OS_LINUX -Werror -nostdinc++ -std=c++2b -MD -MT src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o -MF src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o.d -o src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o -c /build/clickhouse/src/AggregateFunctions/AggregateFunctionSumMap.cpp
LLVM ERROR: out of memory
Allocation failed
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /usr/bin/clang++-15 --target=x86_64-linux-gnu --sysroot=/build/clickhouse/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc --gcc-toolchain=/build/clickhouse/cmake/linux/../../contrib/sysroot/linux-x86_64 -std=c++20 -g -fno-omit-frame-pointer -fsanitize=address -fno-sanitize-address-use-after-scope -Xclang -fuse-ctor-homing -fsized-deallocation -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -ffile-prefix-map=/build/clickhouse=. -falign-functions=32 -mbranches-within-32B-boundaries -fdiagnostics-absolute-paths -fstrict-vtable-pointers -fexperimental-new-pass-manager -Wall -Wextra -Weverything -Wpedantic -Wno-zero-length-array -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c++20-compat -Wno-sign-conversion -Wno-implicit-int-conversion -Wno-implicit-int-float-conversion -Wno-ctad-maybe-unsupported -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-padded -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -Wno-thread-safety-negative -O2 -g -O3 -g -gdwarf-4 -fno-pie -std=c++2b -fdiagnostics-color=always -Werror -DANNOYLIB_MULTITHREADED_build -DBOOST_ASIO_HAS_STD_INVOKE_RESULT=1 -DBOOST_ASIO_STANDALONE=1 -DENABLE_ANNOY -DENABLE_MULTITARGET_CODE=1 -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUNALIGNED_OK -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -I/build/clickhouse/obj-x86_64-linux-gnu/includes/configs -I/build/clickhouse/base/glibc-compatibility/memcpy -I/build/clickhouse/src -I/build/clickhouse/obj-x86_64-linux-gnu/src -I/build/clickhouse/obj-x86_64-linux-gnu/src/Core/include -I/build/clickhouse/base/base/.. -I/build/clickhouse/obj-x86_64-linux-gnu/base/base/.. -I/build/clickhouse/contrib/cctz/include -I/build/clickhouse/base/pcg-random/. -I/build/clickhouse/contrib/miniselect/include -I/build/clickhouse/contrib/zstd/lib -isystem /build/clickhouse/contrib/llvm-project/libcxx/include -isystem /build/clickhouse/contrib/llvm-project/libcxxabi/include -isystem /build/clickhouse/contrib/libunwind/include -isystem /build/clickhouse/contrib/abseil-cpp -isystem /build/clickhouse/contrib/croaring/cpp -isystem /build/clickhouse/contrib/croaring/include -isystem /build/clickhouse/contrib/cityhash102/include -isystem /build/clickhouse/contrib/boost -isystem /build/clickhouse/base/poco/Net/include -isystem /build/clickhouse/base/poco/Foundation/include -isystem /build/clickhouse/base/poco/NetSSL_OpenSSL/include -isystem /build/clickhouse/base/poco/Crypto/include -isystem /build/clickhouse/contrib/boringssl/include -isystem /build/clickhouse/base/poco/Util/include -isystem /build/clickhouse/base/poco/JSON/include -isystem /build/clickhouse/base/poco/XML/include -isystem /build/clickhouse/contrib/replxx/include -isystem /build/clickhouse/contrib/fmtlib-cmake/../fmtlib/include -isystem /build/clickhouse/contrib/magic_enum/include -isystem /build/clickhouse/contrib/double-conversion -isystem /build/clickhouse/contrib/dragonbox/include -isystem /build/clickhouse/contrib/re2 -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/re2-cmake -isystem /build/clickhouse/contrib/zlib-ng -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/zlib-ng-cmake -isystem /build/clickhouse/contrib/pdqsort -isystem /build/clickhouse/contrib/xz/src/liblzma/api -isystem /build/clickhouse/contrib/snappy -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/snappy-cmake -isystem /build/clickhouse/contrib/msgpack-c/include -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/liburing/src/include-compat -isystem /build/clickhouse/obj-x86_64-linux-gnu/contrib/liburing/src/include -isystem /build/clickhouse/contrib/liburing/src/include -isystem /build/clickhouse/contrib/fast_float/include -isystem /build/clickhouse/contrib/NuRaft/include -isystem /build/clickhouse/contrib/nuraft-cmake/, -isystem /build/clickhouse/base/poco/MongoDB/include -isystem /build/clickhouse/contrib/s2geometry/src -isystem /build/clickhouse/contrib/rocksdb/include -isystem /build/clickhouse/contrib/ulid-c/include -isystem /build/clickhouse/contrib/consistent-hashing -isystem /build/clickhouse/contrib/annoy/src -isystem /build/clickhouse/contrib/datasketches-cpp/common/include -isystem /build/clickhouse/contrib/datasketches-cpp/theta/include -DSANITIZER -DNDEBUG -D OS_LINUX -nostdinc++ -c -MD -MT src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o -MF src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o.d -fcolor-diagnostics -o src/AggregateFunctions/CMakeFiles/clickhouse_aggregate_functions.dir/AggregateFunctionSumMap.cpp.o /build/clickhouse/src/AggregateFunctions/AggregateFunctionSumMap.cpp
1.	<eof> parser at end of file
2.	Code generation
 #0 0x00007effbd3218f1 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0xf568f1)
 #1 0x00007effbd31f63e llvm::sys::RunSignalHandlers() (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0xf5463e)
 #2 0x00007effbd320cb6 llvm::sys::CleanupOnSignal(unsigned long) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0xf55cb6)
 #3 0x00007effbd2404df (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0xe754df)
 #4 0x00007effc6ec5140 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x13140)
 #5 0x00007effbbf02ce1 raise (/lib/x86_64-linux-gnu/libc.so.6+0x38ce1)
 #6 0x00007effbbeec537 abort (/lib/x86_64-linux-gnu/libc.so.6+0x22537)
 #7 0x00007effbd252349 llvm::report_bad_alloc_error(char const*, bool) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0xe87349)
 #8 0x00007effbd252382 (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0xe87382)
 #9 0x00007effbc2a40cc operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xa60cc)
#10 0x00007effbe8f18e8 (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0x25268e8)
#11 0x00007effbe90852d llvm::MCAssembler::layout(llvm::MCAsmLayout&) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0x253d52d)
#12 0x00007effbe908d57 llvm::MCAssembler::Finish() (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0x253dd57)
#13 0x00007effbe92687c llvm::MCELFStreamer::finishImpl() (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0x255b87c)
#14 0x00007effbe94bc97 llvm::MCStreamer::finish(llvm::SMLoc) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0x2580c97)
#15 0x00007effbdbcfa20 llvm::AsmPrinter::doFinalization(llvm::Module&) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0x1804a20)
#16 0x00007effbd465411 llvm::FPPassManager::doFinalization(llvm::Module&) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0x109a411)
#17 0x00007effbd45f452 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0x1094452)
#18 0x00007effc4d538e3 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>) (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x19568e3)
#19 0x00007effc50b4241 (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x1cb7241)
#20 0x00007effc3eb860b clang::ParseAST(clang::Sema&, bool, bool) (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0xabb60b)
#21 0x00007effc50b0325 clang::CodeGenAction::ExecuteAction() (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x1cb3325)
#22 0x00007effc5ae93a7 clang::FrontendAction::Execute() (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x26ec3a7)
#23 0x00007effc5a5cfb6 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x265ffb6)
#24 0x00007effc5b673aa clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x276a3aa)
#25 0x00005643846d350f cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/bin/clang++-15+0x1450f)
#26 0x00005643846d170b (/usr/bin/clang++-15+0x1270b)
#27 0x00007effc56bea62 (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x22c1a62)
#28 0x00007effbd24023e llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/usr/lib/x86_64-linux-gnu/libLLVM-15.so.1+0xe7523e)
#29 0x00007effc56be528 clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x22c1528)
#30 0x00007effc568081a clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x228381a)
#31 0x00007effc5680a6e clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x2283a6e)
#32 0x00007effc569d80f clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (/usr/lib/llvm-15/bin/../lib/libclang-cpp.so.15+0x22a080f)
#33 0x00005643846d0e71 clang_main(int, char**) (/usr/bin/clang++-15+0x11e71)
#34 0x00007effbbeedd0a __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x23d0a)
#35 0x00005643846cdf1a _start (/usr/bin/clang++-15+0xef1a)
clang: error: clang frontend command failed with exit code 134 (use -v to see invocation)
Debian clang version 15.0.7
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
clang: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/AggregateFunctionSumMap-a4162e.cpp
clang: note: diagnostic msg: /tmp/AggregateFunctionSumMap-a4162e.sh
clang: note: diagnostic msg:

********************

Which ClickHouse server version to use:
Can try v23.3.8.21-lts / latest 23.3 branch with clang-15 (also reproduces with clang-16).

Hardware
On CI we build it on a Debian based linux with 120Gi Memory and 32CPUs (bumped from an initial 60Gi memory and 16cpus) on dev (where I reproduced), Debian Linux with 32Gi Memory and 32CPUs.

I tired the following:

• Changed built type to Debug from RelWithDebInfo [worked]
• Tried adjusting number of parallel compile jobs and linker jobs (did not make a difference) using -DPARALLEL_COMPILE_JOBS & -DPARALLEL_LINK_JOBS but did not make a difference.

Expected behavior
I would expect the ASAN build to pass under RelDebInfo build type.

Additional context
Reverting this change seems to make the error go away: https://github.com/ClickHouse/ClickHouse/pull/48556
I'm unable to confirm exactly what in that PR is causing the ASAN build to fail for us.

Do let me know if you need any other information. Thank you in advance!

[alexey-milovidov]

The function sumMap is garbage. It will be fixed in #56350