[Bug] Crashing caused by serialization in streaming aggregation

[mrhhsg]

Search before asking

• I had searched in the issues and found no similar issues.

Version

master

What's Wrong?

#0  0x000056065b70a801 in doris::vectorized::AggregateFunctionsSingleValue<doris::vectorized::AggregateFunctionMinData<doris::vectorized::SingleValueDataDecimal<doris::vectorized::Decimal<__int128> > >, false>::streaming_agg_serialize_to_column (this=0x606002ec2980, columns=0x602003b15b90, dst=..., num_rows=829, arena=0x61d001c17c00)
    at /home/zcp/repo_center/doris_master/be/src/vec/aggregate_functions/aggregate_function_min_max.h:531
#1  0x000056065d685c11 in doris::vectorized::AggFnEvaluator::streaming_agg_serialize_to_column (this=0x619002749880, block=0x61d001c17c68, dst=..., num_rows=829, arena=0x61d001c17c00) at /home/zcp/repo_center/doris_master/be/src/vec/exprs/vectorized_agg_fn.cpp:177
#2  0x000056065c85fcd9 in operator()<doris::vectorized::AggregationMethodOneNumber<unsigned int, PHHashMap<unsigned int, char*, HashCRC32<unsigned int> > >&> (__closure=0x7fda77c694e0, agg_method=...) at /home/zcp/repo_center/doris_master/be/src/vec/exec/vaggregation_node.cpp:928
#3  0x000056065c7d7e9b in std::__invoke_impl<void, doris::vectorized::AggregationNode::_pre_agg_with_serialized_key(doris::vectorized::Block*, doris::vectorized::Block*)::<lambda(auto:35&&)>, doris::vectorized::AggregationMethodOneNumber<unsigned int, PHHashMap<unsigned int, char*, HashCRC32<unsigned int>, false>, false>&>(std::__invoke_other, struct {...} &&)
    (__f=...) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:61
#4  0x000056065c7d38c5 in std::__invoke<doris::vectorized::AggregationNode::_pre_agg_with_serialized_key(doris::vectorized::Block*, doris::vectorized::Block*)::<lambda(auto:35&&)>, doris::vectorized::AggregationMethodOneNumber<unsigned int, PHHashMap<unsigned int, char*, HashCRC32<unsigned int>, false>, false>&>(struct {...} &&) (__fn=...)
    at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:96

#7  0x000056065c7c86fd in std::visit<doris::vectorized::AggregationNode::_pre_agg_with_serialized_key(doris::vectorized::Block*, doris::vectorized::Block*)::<lambda(auto:35&&)>, std::variant<doris::vectorized::AggregationMethodSerialized<PHHashMap<StringRef, char*, DefaultHash<StringRef, void>, false> >, doris::vectorized::AggregationMethodOneNumber<unsigned char, FixedHashMap<unsigned char, char*, FixedHashMapImplicitZeroCell<unsigned char, char*, HashTableNoState>, FixedHashTableCalculatedSize<FixedHashMapImplicitZeroCell<unsigned char, char*, HashTableNoState> >, Allocator<true, true> >, false>, doris::vectorized::AggregationMethodOneNumber<short unsigned int, FixedHashMap<short unsigned int, char*, FixedHashMapImplicitZeroCell<short unsigned int, char*, HashTableNoState>, FixedHashTableStoredSize<FixedHashMapImplicitZeroCell<short unsigned int, char*, HashTableNoState> >, Allocator<true, true> >, false>, doris::vectorized::AggregationMethodOneNumber<unsigned int, PHHashMap<unsigned int, char*, HashCRC32<unsigned int>, false>, false>, doris::vectorized::AggregationMethodOneNumber<long unsigned int, PHHashMap<long unsigned int, char*, HashCRC32<long unsigned int>, false>, false>, doris::vectorized::AggregationMethodStringNoCache<StringHashMap<char*, Allocator<true, true> > >, doris::vectorized::AggregationMethodOneNumber<doris::vectorized::UInt128, PHHashMap<doris::vectorized::UInt128, char*, HashCRC32<doris::vectorized::UInt128>, false>, false>, doris::vectorized::AggregationMethodOneNumber<unsigned int, PHHashMap<unsigned int, char*, HashMixWrapper<unsigned int, HashCRC32<unsigned int> >, false>, false>, doris::vectorized::AggregationMethodOneNumber<long unsigned int, PHHashMap<long unsigned int, char*, HashMixWrapper<long unsigned int, HashCRC32<long unsigned int> >, false>, false>, doris::vectorized::AggregationMethodOneNumber<doris::vectorized::UInt128, PHHashMap<doris::vectorized::UInt128, char*, HashMixWrapper<doris::vectorized::UInt128, HashCRC32<doris::vectorized::UInt128> >, false>, false>, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodOneNumber<unsigned char, doris::vectorized::AggregationDataWithNullKey<FixedHashMap<unsigned char, char*, FixedHashMapImplicitZeroCell<unsigned char, char*, HashTableNoState>, FixedHashTableCalculatedSize<FixedHashMapImplicitZeroCell<unsigned char, char*, HashTableNoState> >, Allocator<true, true> > >, false> >, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodOneNumber<short unsigned int, doris::vectorized::AggregationDataWithNullKey<FixedHashMap<short unsigned int, char*, FixedHashMapImplicitZeroCell<short unsigned int, char*, HashTableNoState>, FixedHashTableStoredSize<FixedHashMapImplicitZeroCell<short unsigned int, char*, HashTableNoState> >, Allocator<true, true> > >, false> >, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodOneNumber<unsigned int, doris::vectorized::AggregationDataWithNullKey<PHHashMap<unsigned int, char*, HashCRC32<unsigned int>, false> >, false> >, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodOneNumber<long unsigned int, doris::vectorized::AggregationDataWithNullKey<PHHashMap<long unsigned int, char*, HashCRC32<long unsigned int>, false> >, false> >, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodOneNumber<unsigned int, doris::vectorized::AggregationDataWithNullKey<PHHashMap<unsigned int, char*, HashMixWrapper<unsigned int, HashCRC32<unsigned int> >, false> >, false> >, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodOneNumber<long unsigned int, doris::vectorized::AggregationDataWithNullKey<PHHashMap<long unsigned int, char*, HashMixWrapper<long unsigned int, HashCRC32<long unsigned int> >, false> >, false> >, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodOneNumber<doris::vectorized::UInt128, doris::vectorized::AggregationDataWithNullKey<PHHashMap<doris::vectorized::UInt128, char*, HashCRC32<doris::vectorized::UInt128>, false> >, false> >, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodOneNumber<doris::vectorized::UInt128, doris::vectorized::AggregationDataWithNullKey<PHHashMap<doris::vectorized::UInt128, char*, HashMixWrapper<doris::vectorized::UInt128, HashCRC32<doris::vectorized::UInt128> >, false> >, false> >, doris::vectorized::AggregationMethodSingleNullableColumn<doris::vectorized::AggregationMethodStringNoCache<doris::vectorized::AggregationDataWithNullKey<StringHashMap<char*, Allocator<true, true> > > > >, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<long unsigned int, char*, HashCRC32<long unsigned int>, false>, false>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<long unsigned int, char*, HashCRC32<long unsigned int>, false>, true>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<doris::vectorized::UInt128, char*, HashCRC32<doris::vectorized::UInt128>, false>, false>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<doris::vectorized::UInt128, char*, HashCRC32<doris::vectorized::UInt128>, false>, true>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<doris::vectorized::UInt256, char*, HashCRC32<doris::vectorized::UInt256>, false>, false>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<doris::vectorized::UInt256, char*, HashCRC32<doris::vectorized::UInt256>, false>, true>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<long unsigned int, char*, HashMixWrapper<long unsigned int, HashCRC32<long unsigned int> >, false>, false>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<long unsigned int, char*, HashMixW--Type <RET> for more, q to quit, c to continue without paging--
rapper<long unsigned int, HashCRC32<long unsigned int> >, false>, true>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<doris::vectorized::UInt128, char*, HashMixWrapper<doris::vectorized::UInt128, HashCRC32<doris::vectorized::UInt128> >, false>, false>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<doris::vectorized::UInt128, char*, HashMixWrapper<doris::vectorized::UInt128, HashCRC32<doris::vectorized::UInt128> >, false>, true>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<doris::vectorized::UInt256, char*, HashMixWrapper<doris::vectorized::UInt256, HashCRC32<doris::vectorized::UInt256> >, false>, false>, doris::vectorized::AggregationMethodKeysFixed<PHHashMap<doris::vectorized::UInt256, char*, HashMixWrapper<doris::vectorized::UInt256, HashCRC32<doris::vectorized::UInt256> >, false>, true> >&>(struct {...} &&) (__visitor=...) at /var/local/ldb_toolchain/include/c++/11/variant:1764
#8  0x000056065c7c9277 in doris::vectorized::AggregationNode::_pre_agg_with_serialized_key (this=0x61d001c17480, in_block=0x61d001c17c68, out_block=0x60b001213100) at /home/zcp/repo_center/doris_master/be/src/vec/exec/vaggregation_node.cpp:896
#9  0x000056065c96f4f5 in std::__invoke_impl<doris::Status, doris::Status (doris::vectorized::AggregationNode::*&)(doris::vectorized::Block*, doris::vectorized::Block*), doris::vectorized::AggregationNode*&, doris::vectorized::Block*, doris::vectorized::Block*> (__f=
    @0x603007a43190: (doris::Status (doris::vectorized::AggregationNode::*)(doris::vectorized::AggregationNode * const, doris::vectorized::Block *, doris::vectorized::Block *)) 0x56065c7c8704 <doris::vectorized::AggregationNode::_pre_agg_with_serialized_key(doris::vectorized::Block*, doris::vectorized::Block*)>, __t=@0x603007a431a0: 0x61d001c17480)
    at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:74
#10 0x000056065c963d9c in std::__invoke_r<doris::Status, doris::Status (doris::vectorized::AggregationNode::*&)(doris::vectorized::Block*, doris::vectorized::Block*), doris::vectorized::AggregationNode*&, doris::vectorized::Block*, doris::vectorized::Block*> (__fn=
    @0x603007a43190: (doris::Status (doris::vectorized::AggregationNode::*)(doris::vectorized::AggregationNode * const, doris::vectorized::Block *, doris::vectorized::Block *)) 0x56065c7c8704 <doris::vectorized::AggregationNode::_pre_agg_with_serialized_key(doris::vectorized::Block*, doris::vectorized::Block*)>)
    at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:116
#11 0x000056065c957c61 in std::_Bind_result<doris::Status, doris::Status (doris::vectorized::AggregationNode::*(doris::vectorized::AggregationNode*, std::_Placeholder<1>, std::_Placeholder<2>))(doris::vectorized::Block*, doris::vectorized::Block*)>::__call<doris::Status, doris::vectorized::Block*&&, doris::vectorized::Block*&&, 0ul, 1ul, 2ul>(std::tuple<doris::vectorized::Block*&&, doris::vectorized::Block*&&>&&, std::_Index_tuple<0ul, 1ul, 2ul>) (this=0x603007a43190, __args=...) at /var/local/ldb_toolchain/include/c++/11/functional:571
#12 0x000056065c94fe23 in std::_Bind_result<doris::Status, doris::Status (doris::vectorized::AggregationNode::*(doris::vectorized::AggregationNode*, std::_Placeholder<1>, std::_Placeholder<2>))(doris::vectorized::Block*, doris::vectorized::Block*)>::operator()<doris::vectorized::Block*, doris::vectorized::Block*>(doris::vectorized::Block*&&, doris::vectorized::Block*&&) (this=0x603007a43190) at /var/local/ldb_toolchain/include/c++/11/functional:631
#13 0x000056065c93aa41 in std::__invoke_impl<doris::Status, std::_Bind_result<doris::Status, doris::Status (doris::vectorized::AggregationNode::*(doris::vectorized::AggregationNode*, std::_Placeholder<1>, std::_Placeholder<2>))(doris::vectorized::Block*, doris::vectorized::Block*)>&, doris::vectorized::Block*, doris::vectorized::Block*>(std::__invoke_other, std::_Bind_result<doris::Status, doris::Status (doris::vectorized::AggregationNode::*(doris::vectorized::AggregationNode*, std::_Placeholder<1>, std::_Placeholder<2>))(doris::vectorized::Block*, doris::vectorized::Block*)>&, doris::vectorized::Block*&&, doris::vectorized::Block*&&) (__f=...) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:61
#14 0x000056065c92c50d in std::__invoke_r<doris::Status, std::_Bind_result<doris::Status, doris::Status (doris::vectorized::AggregationNode::*(doris::vectorized::AggregationNode*, std::_Placeholder<1>, std::_Placeholder<2>))(doris::vectorized::Block*, doris::vectorized::Block*)>&, doris::vectorized::Block*, doris::vectorized::Block*>(std::_Bind_result<doris::Status, doris::Status (doris::vectorized::AggregationNode::*(doris::vectorized::AggregationNode*, std::_Placeholder<1>, std::_Placeholder<2>))(doris::vectorized::Block*, doris::vectorized::Block*)>&, doris::vectorized::Block*&&, doris::vectorized::Block*&&) (__fn=...) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:116
#15 0x000056065c927ab8 in std::_Function_handler<doris::Status (doris::vectorized::Block*, doris::vectorized::Block*), std::_Bind_result<doris::Status, doris::Status (doris::vectorized::AggregationNode::*(doris::vectorized::AggregationNode*, std::_Placeholder<1>, std::_Placeholder<2>))(doris::vectorized::Block*, doris::vectorized::Block*)> >::_M_invoke(std::_Any_data const&, doris::vectorized::Block*&&, doris::vectorized::Block*&&) (__functor=..., __args#0=@0x7fda77c699b0: 0x61d001c17c68, __args#1=@0x7fda77c699d0: 0x60b001213100) at /var/local/ldb_toolchain/include/c++/11/bits/std_function.h:292
#16 0x000056065c915b40 in std::function<doris::Status (doris::vectorized::Block*, doris::vectorized::Block*)>::operator()(doris::vectorized::Block*, doris::vectorized::Block*) const (this=0x61d001c17cf8, __args#0=0x61d001c17c68, __args#1=0x60b001213100) at /var/local/ldb_toolchain/include/c++/11/bits/std_function.h:560
#17 0x000056065c7bafee in doris::vectorized::AggregationNode::get_next (this=0x61d001c17480, state=0x61c000997880, block=0x60b001213100, eos=0x617000412049) at /home/zcp/repo_center/doris_master/be/src/vec/exec/vaggregation_node.cpp:470
#18 0x000056065a4d645e in doris::PlanFragmentExecutor::get_vectorized_internal (this=0x617000411f70, block=0x7fda77c6a150) at /home/zcp/repo_center/doris_master/be/src/runtime/plan_fragment_executor.cpp:346
#19 0x000056065a4d5222 in doris::PlanFragmentExecutor::open_vectorized_internal (this=0x617000411f70) at /home/zcp/repo_center/doris_master/be/src/runtime/plan_fragment_executor.cpp:294
#20 0x000056065a4d40f8 in doris::PlanFragmentExecutor::open (this=0x617000411f70) at /home/zcp/repo_center/doris_master/be/src/runtime/plan_fragment_executor.cpp:249
#21 0x000056065a4869d5 in doris::FragmentExecState::execute (this=0x617000411f00) at /home/zcp/repo_center/doris_master/be/src/runtime/fragment_mgr.cpp:246
#22 0x000056065a48eed1 in doris::FragmentMgr::_exec_actual(std::shared_ptr<doris::FragmentExecState>, std::function<void (doris::PlanFragmentExecutor*)>) (this=0x614000053a40, exec_state=..., cb=...) at /home/zcp/repo_center/doris_master/be/src/runtime/fragment_mgr.cpp:500
#23 0x000056065a4910bb in operator() (__closure=0x60800098b520) at /home/zcp/repo_center/doris_master/be/src/runtime/fragment_mgr.cpp:704
#24 0x000056065a49bd84 in std::__invoke_impl<void, doris::FragmentMgr::exec_plan_fragment(const doris::TExecPlanFragmentParams&, doris::FragmentMgr::FinishCallback)::<lambda()>&>(std::__invoke_other, struct {...} &) (__f=...) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:61
#25 0x000056065a49b95a in std::__invoke_r<void, doris::FragmentMgr::exec_plan_fragment(const doris::TExecPlanFragmentParams&, doris::FragmentMgr::FinishCallback)::<lambda()>&>(struct {...} &) (__fn=...) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:111
#26 0x000056065a49b284 in std::_Function_handler<void(), doris::FragmentMgr::exec_plan_fragment(const doris::TExecPlanFragmentParams&, doris::FragmentMgr::FinishCallback)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /var/local/ldb_toolchain/include/c++/11/bits/std_function.h:291
#27 0x000056065a399468 in std::function<void ()>::operator()() const (this=0x606002082b18) at /var/local/ldb_toolchain/include/c++/11/bits/std_function.h:560
#28 0x000056065abd71c6 in doris::FunctionRunnable::run (this=0x606002082b10) at /home/zcp/repo_center/doris_master/be/src/util/threadpool.cpp:45
#29 0x000056065abd2534 in doris::ThreadPool::dispatch_thread (this=0x6150001f8b80) at /home/zcp/repo_center/doris_master/be/src/util/threadpool.cpp:540
#30 0x000056065abf38d4 in std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&> (__f=@0x603000305860: (void (doris::ThreadPool::*)(doris::ThreadPool * const)) 0x56065abd1200 <doris::ThreadPool::dispatch_thread()>, __t=@0x603000305870: 0x6150001f8b80) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:74
#31 0x000056065abf3173 in std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&> (__fn=@0x603000305860: (void (doris::ThreadPool::*)(doris::ThreadPool * const)) 0x56065abd1200 <doris::ThreadPool::dispatch_thread()>) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:96
#32 0x000056065abf2512 in std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) (this=0x603000305860, __args=...) at /var/local/ldb_toolchain/include/c++/11/functional:420
#33 0x000056065abf1015 in std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::operator()<, void>() (this=0x603000305860) at /var/local/ldb_toolchain/include/c++/11/functional:503
#34 0x000056065abedb7c in std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) (__f=...) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:61
#35 0x000056065abeb0d8 in std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) (__fn=...) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:111
#36 0x000056065abe63db in std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) (__functor=...) at /var/local/ldb_toolchain/include/c++/11/bits/std_function.h:291
#37 0x000056065a399468 in std::function<void ()>::operator()() const (this=0x61100002fc18) at /var/local/ldb_toolchain/include/c++/11/bits/std_function.h:560
#38 0x000056065abb2090 in doris::Thread::supervise_thread (arg=0x61100002fc00) at /home/zcp/repo_center/doris_master/be/src/util/thread.cpp:425
#39 0x00007fdaa40f2609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#40 0x00007fdaa422c163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

What You Expected?

Do not crash

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct