-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix](struct)Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump #49485
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…ne more subfield causing BE coredump
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
| 4 \N \N {"f1":null, "f2":null, "f3":null, "f4":null} {"f1":"abc", "f2":"def", "f3":"hij"} | ||
|
|
||
| -- !select -- | ||
| 1 {"a":1} {"a":1, "b":"a"} {"a":1, "s":{"a":1}} {"a":1, "s":{"a":1, "b":"a"}} {"a":1, "s":{"b":1, "s":{"c":1}}} {"a":1, "b":{"c":1, "d":"a"}, "e":1} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not using beut to test this case?
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
eldenmoon
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…iteral with one more subfield causing BE coredump (#49485) Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump some situation like this blow will make BE core ``` create table t(a int, b int, s struct<a:int>) PROPERTIES ("replication_allocation" = "tag.location.default: 1"); insert into t values(1,1,'{1,2}'); ``` core info: ``` [WARNING!] /sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never, Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. Turn off THP: `echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled` start BE in local mode ================================================================= ==2818976==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000f17a90 at pc 0x55eba7fdd69c bp 0x7f03243dbd30 sp 0x7f03243dbd28 READ of size 8 at 0x603000f17a90 thread T928 (brpc_light) #0 0x55eba7fdd69b in std::__shared_ptr<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2>::get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291:16 #1 0x55eba7fdd649 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:990:66 #2 0x55eba7fda316 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::operator->() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:984:9 #3 0x55ebcfc8b19e in doris::vectorized::DataTypeStructSerDe::deserialize_one_cell_from_json(doris::vectorized::IColumn&, doris::Slice&, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_struct_serde.cpp:200:25 #4 0x55ebdac49c8b in doris::vectorized::ConvertImplGenericFromString::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:618:36 #5 0x55ebda2234d0 in doris::Status std::__invoke_impl<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #6 0x55ebda22323a in std::enable_if<is_invocable_r_v<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>, doris::Status>::type std::__invoke_r<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:114:9 #7 0x55ebda222e71 in std::_Function_handler<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long), doris::Status (*)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)>::_M_invoke(std::_Any_data const&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #8 0x55ebd9b3f1d2 in std::function<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long)>::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long) const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 #9 0x55ebda1ad7c7 in doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:2241:17 #10 0x55ebda1acee2 in doris::Status std::__invoke_impl<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bo@@@ ```
…iteral with one more subfield causing BE coredump (#49485) Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump some situation like this blow will make BE core ``` create table t(a int, b int, s struct<a:int>) PROPERTIES ("replication_allocation" = "tag.location.default: 1"); insert into t values(1,1,'{1,2}'); ``` core info: ``` [WARNING!] /sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never, Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. Turn off THP: `echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled` start BE in local mode ================================================================= ==2818976==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000f17a90 at pc 0x55eba7fdd69c bp 0x7f03243dbd30 sp 0x7f03243dbd28 READ of size 8 at 0x603000f17a90 thread T928 (brpc_light) #0 0x55eba7fdd69b in std::__shared_ptr<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2>::get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291:16 #1 0x55eba7fdd649 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:990:66 #2 0x55eba7fda316 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::operator->() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:984:9 #3 0x55ebcfc8b19e in doris::vectorized::DataTypeStructSerDe::deserialize_one_cell_from_json(doris::vectorized::IColumn&, doris::Slice&, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_struct_serde.cpp:200:25 #4 0x55ebdac49c8b in doris::vectorized::ConvertImplGenericFromString::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:618:36 #5 0x55ebda2234d0 in doris::Status std::__invoke_impl<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #6 0x55ebda22323a in std::enable_if<is_invocable_r_v<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>, doris::Status>::type std::__invoke_r<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:114:9 #7 0x55ebda222e71 in std::_Function_handler<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long), doris::Status (*)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)>::_M_invoke(std::_Any_data const&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #8 0x55ebd9b3f1d2 in std::function<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long)>::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long) const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 #9 0x55ebda1ad7c7 in doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:2241:17 #10 0x55ebda1acee2 in doris::Status std::__invoke_impl<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bo@@@ ```
…iteral with one more subfield causing BE coredump (#49485) Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump some situation like this blow will make BE core ``` create table t(a int, b int, s struct<a:int>) PROPERTIES ("replication_allocation" = "tag.location.default: 1"); insert into t values(1,1,'{1,2}'); ``` core info: ``` [WARNING!] /sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never, Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. Turn off THP: `echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled` start BE in local mode ================================================================= ==2818976==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000f17a90 at pc 0x55eba7fdd69c bp 0x7f03243dbd30 sp 0x7f03243dbd28 READ of size 8 at 0x603000f17a90 thread T928 (brpc_light) #0 0x55eba7fdd69b in std::__shared_ptr<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2>::get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291:16 #1 0x55eba7fdd649 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:990:66 #2 0x55eba7fda316 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::operator->() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:984:9 #3 0x55ebcfc8b19e in doris::vectorized::DataTypeStructSerDe::deserialize_one_cell_from_json(doris::vectorized::IColumn&, doris::Slice&, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_struct_serde.cpp:200:25 #4 0x55ebdac49c8b in doris::vectorized::ConvertImplGenericFromString::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:618:36 #5 0x55ebda2234d0 in doris::Status std::__invoke_impl<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #6 0x55ebda22323a in std::enable_if<is_invocable_r_v<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>, doris::Status>::type std::__invoke_r<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:114:9 #7 0x55ebda222e71 in std::_Function_handler<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long), doris::Status (*)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)>::_M_invoke(std::_Any_data const&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #8 0x55ebd9b3f1d2 in std::function<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long)>::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long) const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 #9 0x55ebda1ad7c7 in doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:2241:17 #10 0x55ebda1acee2 in doris::Status std::__invoke_impl<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bo@@@ ```
…iteral with one more subfield causing BE coredump (apache#49485) Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump some situation like this blow will make BE core ``` create table t(a int, b int, s struct<a:int>) PROPERTIES ("replication_allocation" = "tag.location.default: 1"); insert into t values(1,1,'{1,2}'); ``` core info: ``` [WARNING!] /sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never, Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. Turn off THP: `echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled` start BE in local mode ================================================================= ==2818976==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000f17a90 at pc 0x55eba7fdd69c bp 0x7f03243dbd30 sp 0x7f03243dbd28 READ of size 8 at 0x603000f17a90 thread T928 (brpc_light) #0 0x55eba7fdd69b in std::__shared_ptr<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2>::get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291:16 apache#1 0x55eba7fdd649 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:990:66 apache#2 0x55eba7fda316 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::operator->() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:984:9 apache#3 0x55ebcfc8b19e in doris::vectorized::DataTypeStructSerDe::deserialize_one_cell_from_json(doris::vectorized::IColumn&, doris::Slice&, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_struct_serde.cpp:200:25 apache#4 0x55ebdac49c8b in doris::vectorized::ConvertImplGenericFromString::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:618:36 apache#5 0x55ebda2234d0 in doris::Status std::__invoke_impl<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 apache#6 0x55ebda22323a in std::enable_if<is_invocable_r_v<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>, doris::Status>::type std::__invoke_r<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:114:9 apache#7 0x55ebda222e71 in std::_Function_handler<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long), doris::Status (*)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)>::_M_invoke(std::_Any_data const&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 apache#8 0x55ebd9b3f1d2 in std::function<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long)>::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long) const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 apache#9 0x55ebda1ad7c7 in doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:2241:17 apache#10 0x55ebda1acee2 in doris::Status std::__invoke_impl<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bo@@@ ```
Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump
some situation like this blow will make BE core
core info:
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)