[Feature](runtime-filter) support rf on set operator: be part#49679
[Feature](runtime-filter) support rf on set operator: be part#49679BiteTheDDDDt merged 4 commits intoapache:masterfrom
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
run beut |
| : RuntimeFilterWrapper::State::READY; | ||
| if (!wake_up_early) { | ||
| // Hash table is completed and runtime filter has a global size now. | ||
| uint64_t hash_table_size = block ? block->rows() : 0; |
There was a problem hiding this comment.
why use set hash table size. more correct
| return Status::OK(); | ||
| } | ||
|
|
||
| if (!_runtime_filter_producer_helper || state->is_cancelled() || !_eos) { |
There was a problem hiding this comment.
if (_runtime_filter_producer_helper && state->ok() && _eos) {
try {
RETURN_IF_ERROR(
_runtime_filter_producer_helper->process(state, &_shared_state->build_block));
} catch (Exception& e) {
return Status::InternalError(
"rf process meet error: {}, wake_up_early: {}, _finish_dependency: {}",
e.to_string(), state->get_task()->wake_up_early(),
_finish_dependency->debug_string());
}
}
return Base::close(state, exec_status);
|
run buildall |
|
run buildall |
|
run buildall |
Gabriel39
left a comment
There was a problem hiding this comment.
I strongly suggest to use a common interface to unify RuntimeFilterProducerHelper/RuntimeFilterProducerHelperCross/RuntimeFilterProducerHelperSet
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
…#49679) ### What problem does this PR solve? support rf on set operator: be part ```sql create table tx ( k1 int null, k2 int not null, k3 bigint null, k4 varchar(100) null ) duplicate key (k1) properties("replication_num" = "1"); insert into tx select e1,e1,e1,e1 from (select 1 k1) as t lateral view explode_numbers(x) tmp1 as e1; select k1 from t1000 intersect select k1 from t10000000; 10.03 sec -> 0.13 sec select k1 from t1000 intersect select k1 from t1000000; 1.17 sec -> 0.13 sec select k1 from t10000 intersect select k1 from t100000; 0.23 sec -> 0.18 sec select k1 from t10000 intersect select * from( select k1 from t100000 where k1 > 10000)t; 0.15 sec -> 0.16 sec select k1 from t10000 intersect select * from( select k1 from t100000 where k1 < 10000)t; 0.13sec -> 0.18sec ``` part of apache#49573 ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [x] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [x] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [x] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [x] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
What problem does this PR solve?
support rf on set operator: be part
part of #49573
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)