Skip to content

[Bug](pipelineX) init runtime filter profile at first#24106

Merged
Gabriel39 merged 1 commit intoapache:masterfrom
Gabriel39:fix_0908
Sep 8, 2023
Merged

[Bug](pipelineX) init runtime filter profile at first#24106
Gabriel39 merged 1 commit intoapache:masterfrom
Gabriel39:fix_0908

Conversation

@Gabriel39
Copy link
Contributor

@Gabriel39 Gabriel39 commented Sep 8, 2023

Proposed changes

*** Current BE git commitID: eea0a33 ***
15:31:00  *** SIGSEGV address not mapped to object (@0x1f0) received by PID 30487 (TID 30812 OR 0x7ff5bf110700) from PID 496; stack trace: ***
15:31:00  0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:413
15:31:00  1# os::Linux::chained_handler(int, siginfo*, void*) in /usr/local/software/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so
15:31:00  2# JVM_handle_linux_signal in /usr/local/software/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so
15:31:00  3# signalHandler(int, siginfo*, void*) in /usr/local/software/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so
15:31:00  4# 0x00007FF79D7A3090 in /lib/x86_64-linux-gnu/libc.so.6
15:31:00  5# __pthread_mutex_lock at ../nptl/pthread_mutex_lock.c:67
15:31:00  6# pthread_mutex_lock in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P1/Cluster0/be/lib/doris_be
15:31:00  7# doris::RuntimeProfile::add_info_string(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) at /root/doris/be/src/util/runtime_profile.cpp:340
15:31:00  8# doris::IRuntimeFilter::signal() at /root/doris/be/src/exprs/runtime_filter.cpp:1278
15:31:00  9# doris::IRuntimeFilter::publish() at /root/doris/be/src/exprs/runtime_filter.cpp:1140
15:31:00  10# doris::VRuntimeFilterSlots::publish() at /root/doris/be/src/exprs/runtime_filter_slots.h:223
15:31:00  11# doris::vectorized::ProcessRuntimeFilterBuild<doris::vectorized::PrimaryTypeHashTableContext<unsigned int, doris::vectorized::RowRefList> >::operator()(doris::RuntimeState*, doris::vectorized::PrimaryTypeHashTableContext<unsigned int, doris::vectorized::RowRefList>&) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P1/Cluster0/be/lib/doris_be
15:31:00  12# _ZNSt8__detail9__variant17__gen_vtable_implINS0_12_Multi_arrayIPFNS0_21__deduce_visit_resultIN5doris6StatusEEEONS4_8pipeline8OverloadIJZNS7_26HashJoinBuildSinkOperatorX4sinkEPNS4_12RuntimeStateEPNS4_10vectorized5BlockENS7_11SourceStateEE3$_0ZNS9_4sinkESB_SE_SF_E3$1EEERSt7variantIJSt9monostateNSC_26SerializedHashTableContextINSC_10RowRefListEEENSC_27PrimaryTypeHashTableContextIhSN_EENSP_ItSN_EENSP_IjSN_EENSP_ImSN_EENSP_INSC_7UInt128ESN_EENSP_INSC_7UInt256ESN_EENSC_24FixedKeyHashTableContextImLb1ESN_EENSY_ImLb0ESN_EENSY_ISU_Lb1ESN_EENSY_ISU_Lb0ESN_EENSY_ISW_Lb1ESN_EENSY_ISW_Lb0ESN_EENSM_INSC_18RowRefListWithFlagEEENSP_IhS15_EENSP_ItS15_EENSP_IjS15_EENSP_ImS15_EENSP_ISU_S15_EENSP_ISW_S15_EENSY_ImLb1ES15_EENSY_ImLb0ES15_EENSY_ISU_Lb1ES15_EENSY_ISU_Lb0ES15_EENSY_ISW_Lb1ES15_EENSY_ISW_Lb0ES15_EENSM_INSC_19RowRefListWithFlagsEEENSP_IhS1J_EENSP_ItS1J_EENSP_IjS1J_EENSP_ImS1J_EENSP_ISU_S1J_EENSP_ISW_S1J_EENSY_ImLb1ES1J_EENSY_ImLb0ES1J_EENSY_ISU_Lb1ES1J_EENSY_ISU_Lb0ES1J_EENSY_ISW_Lb1ES1J_EENSY_ISW_Lb0ES1J_EEEEEJEEESt16integer_sequenceImJLm4EEEE14__visit_invokeESJ_S1Y at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/variant:1013
15:31:00  13# doris::pipeline::HashJoinBuildSinkOperatorX::sink(doris::RuntimeState*, doris::vectorized::Block*, doris::pipeline::SourceState) at /root/doris/be/src/pipeline/exec/hashjoin_build_sink.cpp:497
15:31:00  14# doris::pipeline::PipelineXTask::execute(bool*) at /root/doris/be/src/pipeline/pipeline_x/pipeline_x_task.cpp:198
15:31:00  15# doris::pipeline::TaskScheduler::_do_work(unsigned long) at /root/doris/be/src/pipeline/task_scheduler.cpp:265
15:31:00  16# doris::ThreadPool::dispatch_thread() at /root/doris/be/src/util/threadpool.cpp:539
15:31:00  17# doris::Thread::supervise_thread(void*) at /root/doris/be/src/util/thread.cpp:466
15:31:00  18# start_thread at /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478
15:31:00  19# __clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Gabriel39
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2023

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.91% (7895/21387)
Line Coverage: 28.93% (63358/219007)
Region Coverage: 27.87% (32896/118019)
Branch Coverage: 24.46% (16897/69078)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a2f6850fa0731d1e38a5160fef46d5858893d0cd_a2f6850fa0731d1e38a5160fef46d5858893d0cd/report/index.html

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.54 seconds
stream load tsv: 527 seconds loaded 74807831229 Bytes, about 135 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162186316 Bytes

Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 8, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2023

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2023

PR approved by anyone and no changes requested.

@Gabriel39 Gabriel39 merged commit 0b24bd6 into apache:master Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments