Skip to content

[Fix]Fix TopN pushdown not select be by tag#46154

Merged
wangbo merged 1 commit intoapache:masterfrom
wangbo:1227_fix_topn_tag
Dec 30, 2024
Merged

[Fix]Fix TopN pushdown not select be by tag#46154
wangbo merged 1 commit intoapache:masterfrom
wangbo:1227_fix_topn_tag

Conversation

@wangbo
Copy link
Contributor

@wangbo wangbo commented Dec 30, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
For TopN pushdown feature, it should select backends to fetch two phase result by user's tag.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 30, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@wangbo
Copy link
Contributor Author

wangbo commented Dec 30, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32348 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 38f7a21583e8a141c9a09a1cee3272627cb6df71, data reload: false

------ Round 1 ----------------------------------
q1	17568	6133	5984	5984
q2	2055	299	169	169
q3	10425	1214	738	738
q4	10264	867	424	424
q5	8139	2136	1970	1970
q6	205	184	147	147
q7	889	759	607	607
q8	9248	1350	1171	1171
q9	5286	4873	4883	4873
q10	6747	2284	1868	1868
q11	480	277	257	257
q12	345	357	216	216
q13	17753	3542	2952	2952
q14	225	230	206	206
q15	567	506	515	506
q16	646	632	593	593
q17	579	865	340	340
q18	6899	6601	6348	6348
q19	2059	960	543	543
q20	297	304	188	188
q21	2777	2149	1933	1933
q22	356	341	315	315
Total cold run time: 103809 ms
Total hot run time: 32348 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6255	6204	6147	6147
q2	229	323	224	224
q3	2239	2686	2311	2311
q4	1406	1842	1381	1381
q5	4338	4726	4756	4726
q6	180	182	139	139
q7	2086	1943	1833	1833
q8	2669	2797	2676	2676
q9	7333	7231	7333	7231
q10	3070	3348	2763	2763
q11	584	517	499	499
q12	648	773	649	649
q13	3310	3759	3166	3166
q14	286	299	271	271
q15	577	524	520	520
q16	659	697	658	658
q17	1217	1731	1261	1261
q18	7638	7617	7399	7399
q19	828	1208	1073	1073
q20	2013	2019	1931	1931
q21	5679	5199	4926	4926
q22	618	607	607	607
Total cold run time: 53862 ms
Total hot run time: 52391 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196936 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 38f7a21583e8a141c9a09a1cee3272627cb6df71, data reload: false

query1	1303	979	906	906
query2	6467	2347	2275	2275
query3	10966	4831	4559	4559
query4	33197	23957	23385	23385
query5	4814	601	451	451
query6	288	201	204	201
query7	3999	491	306	306
query8	305	259	251	251
query9	9445	2747	2756	2747
query10	472	306	245	245
query11	18043	15479	15174	15174
query12	160	113	104	104
query13	1582	555	422	422
query14	9566	7929	7019	7019
query15	245	213	184	184
query16	7915	589	433	433
query17	1511	754	592	592
query18	2084	397	297	297
query19	219	183	167	167
query20	124	113	116	113
query21	215	133	115	115
query22	4441	4592	4444	4444
query23	34461	33964	33773	33773
query24	6373	2339	2335	2335
query25	475	462	383	383
query26	1166	279	152	152
query27	2135	456	328	328
query28	5667	2522	2489	2489
query29	654	560	422	422
query30	205	192	154	154
query31	1007	928	842	842
query32	64	60	58	58
query33	480	349	292	292
query34	785	844	524	524
query35	807	828	774	774
query36	1011	1062	967	967
query37	115	105	76	76
query38	4292	4243	4150	4150
query39	1548	1451	1458	1451
query40	215	116	105	105
query41	53	46	47	46
query42	128	108	115	108
query43	560	515	486	486
query44	1338	826	856	826
query45	184	182	178	178
query46	873	1076	658	658
query47	2014	2008	2001	2001
query48	401	437	341	341
query49	761	495	385	385
query50	621	666	394	394
query51	7209	7290	7292	7290
query52	100	100	91	91
query53	223	281	187	187
query54	476	506	431	431
query55	80	81	79	79
query56	257	271	241	241
query57	1238	1266	1187	1187
query58	247	230	220	220
query59	3276	3287	3251	3251
query60	281	262	258	258
query61	122	111	125	111
query62	863	817	740	740
query63	225	183	187	183
query64	3830	1030	642	642
query65	3341	3219	3272	3219
query66	948	429	303	303
query67	16568	15967	15496	15496
query68	9326	762	522	522
query69	474	300	259	259
query70	1245	1143	1161	1143
query71	429	295	253	253
query72	6214	3852	3891	3852
query73	663	763	356	356
query74	10482	9060	9050	9050
query75	4543	3158	2643	2643
query76	5378	1195	790	790
query77	1046	376	282	282
query78	10055	10412	9496	9496
query79	2362	906	613	613
query80	730	509	436	436
query81	485	274	234	234
query82	484	147	121	121
query83	196	175	148	148
query84	276	93	73	73
query85	760	343	303	303
query86	345	320	296	296
query87	4511	4502	4430	4430
query88	3418	2256	2211	2211
query89	426	336	296	296
query90	1906	183	186	183
query91	130	147	149	147
query92	64	54	52	52
query93	1073	915	543	543
query94	656	407	289	289
query95	332	288	252	252
query96	492	605	281	281
query97	2773	2781	2730	2730
query98	222	201	198	198
query99	1717	1556	1437	1437
Total cold run time: 298251 ms
Total hot run time: 196936 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 38f7a21583e8a141c9a09a1cee3272627cb6df71, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.41	0.42	0.41
query6	1.16	0.65	0.66
query7	0.02	0.01	0.02
query8	0.03	0.03	0.03
query9	0.60	0.48	0.50
query10	0.56	0.58	0.55
query11	0.14	0.10	0.10
query12	0.16	0.11	0.11
query13	0.61	0.59	0.59
query14	2.82	2.74	2.83
query15	0.90	0.84	0.82
query16	0.39	0.38	0.37
query17	1.00	1.00	1.06
query18	0.23	0.20	0.21
query19	2.00	1.78	2.01
query20	0.02	0.01	0.01
query21	15.38	0.93	0.55
query22	0.74	0.74	0.72
query23	15.29	1.48	0.61
query24	3.09	1.74	1.60
query25	0.24	0.17	0.11
query26	0.29	0.14	0.14
query27	0.04	0.04	0.04
query28	14.06	1.47	1.05
query29	12.60	3.90	3.25
query30	0.25	0.09	0.06
query31	2.82	0.60	0.39
query32	3.22	0.54	0.46
query33	3.08	3.09	3.14
query34	16.57	5.06	4.49
query35	4.57	4.57	4.46
query36	0.66	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.42 s
Total hot run time: 31.89 s

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 30, 2024
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@xinyiZzz xinyiZzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wangbo wangbo merged commit e57ccfb into apache:master Dec 30, 2024
15 of 20 checks passed
@wm1581066 wm1581066 added the usercase Important user case type label label Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants