Skip to content

Conversation

@jacktengg
Copy link
Contributor

@jacktengg jacktengg commented Jun 30, 2023

Proposed changes

Issue Number: close #xxx

For pipeline engine, ExecNode::create_tree may throw exception sometimes, e.g. SELECT MIN(-3.40282347e+38) FROM t0; will throw a exception bacause of invalid decimal precision.

*** Query id: 346886bf48494e77-96eeea5361233618 ***
*** Aborted at 1688101183 (unix time) try "date -d @1688101183" if you are using GNU date ***
*** Current BE git commitID: 2fcb0e090b ***
*** SIGABRT unknown detail explain (@0x13ef42) received by PID 1306434 (TID 1306918 OR 0x7ff0763e1700) from PID 1306434; stack trace: ***
terminate called recursively
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:413
 1# 0x00007FFA8780E090 in /lib/x86_64-linux-gnu/libc.so.6
 2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
 3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
 4# __gnu_cxx::__verbose_terminate_handler() [clone .cold] at ../../../../libstdc++-v3/libsupc++/vterminate.cc:75
 5# __cxxabiv1::__terminate(void (*)()) at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48
 6# 0x000055B6C30C7401 in /mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
 7# 0x000055B6C30C7554 in /mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
 8# doris::vectorized::create_decimal(unsigned long, unsigned long, bool) at /home/zcp/repo_center/doris_master/doris/be/src/vec/data_types/data_type_decimal.cpp:167
 9# doris::vectorized::DataTypeFactory::create_data_type(doris::TypeDescriptor const&, bool) at /home/zcp/repo_center/doris_master/doris/be/src/vec/data_types/data_type_factory.cpp:185
10# doris::vectorized::AggFnEvaluator::AggFnEvaluator(doris::TExprNode const&) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vectorized_agg_fn.cpp:79
11# std::unique_ptr<doris::vectorized::AggFnEvaluator, std::default_delete<doris::vectorized::AggFnEvaluator> > doris::vectorized::AggFnEvaluator::create_unique<doris::TExprNode const&>(doris::TExprNode const&) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vectorized_agg_fn.h:49
12# doris::vectorized::AggFnEvaluator::create(doris::ObjectPool*, doris::TExpr const&, doris::TSortInfo const&, doris::vectorized::AggFnEvaluator**) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vectorized_agg_fn.cpp:92
13# doris::vectorized::AggregationNode::init(doris::TPlanNode const&, doris::RuntimeState*) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/vaggregation_node.cpp:158
14# doris::ExecNode::create_tree_helper(doris::RuntimeState*, doris::ObjectPool*, std::vector<doris::TPlanNode, std::allocator<doris::TPlanNode> > const&, doris::DescriptorTbl const&, doris::ExecNode*, int*, doris::ExecNode**) at /home/zcp/repo_center/doris_master/doris/be/src/exec/exec_node.cpp:276
15# doris::ExecNode::create_tree(doris::RuntimeState*, doris::ObjectPool*, doris::TPlan const&, doris::DescriptorTbl const&, doris::ExecNode**) at /home/zcp/repo_center/doris_master/doris/be/src/exec/exec_node.cpp:231
16# doris::pipeline::PipelineFragmentContext::prepare(doris::TPipelineFragmentParams const&, unsigned long) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_fragment_context.cpp:253
17# doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_1::operator()(int) const at /home/zcp/repo_center/doris_master/doris/be/src/runtime/fragment_mgr.cpp:895
18# doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_0::operator()() const at /home/zcp/repo_center/doris_master/doris/be/src/runtime/fragment_mgr.cpp:926
19# void std::__invoke_impl<void, doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_0&>(std::__invoke_other, doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_0&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@jacktengg
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 39.45 seconds
stream load tsv: 457 seconds loaded 74807831229 Bytes, about 156 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select: 69.4 seconds inserted 10000000 Rows, about 144K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230630101221_clickbench_pr_170464.html

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 1, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2023

PR approved by at least one committer and no changes requested.

@yiguolei yiguolei added dev/2.0.0 2.0.0 release and removed approved Indicates a PR has been approved by one committer. labels Jul 1, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2023

PR approved by anyone and no changes requested.

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 3, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jul 3, 2023

PR approved by at least one committer and no changes requested.

Copy link
Contributor

@Gabriel39 Gabriel39 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@xinyiZzz xinyiZzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 7e02566 into apache:master Jul 3, 2023
@xiaokang xiaokang added dev/2.0.0-merged and removed dev/2.0.0 2.0.0 release labels Jul 3, 2023
xiaokang pushed a commit that referenced this pull request Jul 3, 2023
For pipeline engine, ExecNode::create_tree may throw exception sometimes, e.g. SELECT MIN(-3.40282347e+38) FROM t0; will throw a exception bacause of invalid decimal precision.

*** Query id: 346886bf48494e77-96eeea5361233618 ***
*** Aborted at 1688101183 (unix time) try "date -d @1688101183" if you are using GNU date ***
*** Current BE git commitID: 2fcb0e0 ***
*** SIGABRT unknown detail explain (@0x13ef42) received by PID 1306434 (TID 1306918 OR 0x7ff0763e1700) from PID 1306434; stack trace: ***
terminate called recursively
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:413
 1# 0x00007FFA8780E090 in /lib/x86_64-linux-gnu/libc.so.6
 2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
 3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
 4# __gnu_cxx::__verbose_terminate_handler() [clone .cold] at ../../../../libstdc++-v3/libsupc++/vterminate.cc:75
 5# __cxxabiv1::__terminate(void (*)()) at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48
 6# 0x000055B6C30C7401 in /mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
 7# 0x000055B6C30C7554 in /mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
 8# doris::vectorized::create_decimal(unsigned long, unsigned long, bool) at /home/zcp/repo_center/doris_master/doris/be/src/vec/data_types/data_type_decimal.cpp:167
 9# doris::vectorized::DataTypeFactory::create_data_type(doris::TypeDescriptor const&, bool) at /home/zcp/repo_center/doris_master/doris/be/src/vec/data_types/data_type_factory.cpp:185
10# doris::vectorized::AggFnEvaluator::AggFnEvaluator(doris::TExprNode const&) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vectorized_agg_fn.cpp:79
11# std::unique_ptr<doris::vectorized::AggFnEvaluator, std::default_delete<doris::vectorized::AggFnEvaluator> > doris::vectorized::AggFnEvaluator::create_unique<doris::TExprNode const&>(doris::TExprNode const&) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vectorized_agg_fn.h:49
12# doris::vectorized::AggFnEvaluator::create(doris::ObjectPool*, doris::TExpr const&, doris::TSortInfo const&, doris::vectorized::AggFnEvaluator**) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vectorized_agg_fn.cpp:92
13# doris::vectorized::AggregationNode::init(doris::TPlanNode const&, doris::RuntimeState*) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/vaggregation_node.cpp:158
14# doris::ExecNode::create_tree_helper(doris::RuntimeState*, doris::ObjectPool*, std::vector<doris::TPlanNode, std::allocator<doris::TPlanNode> > const&, doris::DescriptorTbl const&, doris::ExecNode*, int*, doris::ExecNode**) at /home/zcp/repo_center/doris_master/doris/be/src/exec/exec_node.cpp:276
15# doris::ExecNode::create_tree(doris::RuntimeState*, doris::ObjectPool*, doris::TPlan const&, doris::DescriptorTbl const&, doris::ExecNode**) at /home/zcp/repo_center/doris_master/doris/be/src/exec/exec_node.cpp:231
16# doris::pipeline::PipelineFragmentContext::prepare(doris::TPipelineFragmentParams const&, unsigned long) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_fragment_context.cpp:253
17# doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_1::operator()(int) const at /home/zcp/repo_center/doris_master/doris/be/src/runtime/fragment_mgr.cpp:895
18# doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_0::operator()() const at /home/zcp/repo_center/doris_master/doris/be/src/runtime/fragment_mgr.cpp:926
19# void std::__invoke_impl<void, doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_0&>(std::__invoke_other, doris::FragmentMgr::exec_plan_fragment(doris::TPipelineFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_0&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. area/pipeline dev/2.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants