Skip to content

[opt](column) Optimize the execution of the append_data_by_selector function in ColumnNullable.#49421

Closed
Mryange wants to merge 1 commit intoapache:masterfrom
Mryange:opt-column_nullable_append_data_by_selector
Closed

[opt](column) Optimize the execution of the append_data_by_selector function in ColumnNullable.#49421
Mryange wants to merge 1 commit intoapache:masterfrom
Mryange:opt-column_nullable_append_data_by_selector

Conversation

@Mryange
Copy link
Contributor

@Mryange Mryange commented Mar 24, 2025

What problem does this PR solve?

CPU Caches:
  L1 Data 32 KiB (x48)
  L1 Instruction 32 KiB (x48)
  L2 Unified 1024 KiB (x48)
  L3 Unified 36608 KiB (x2)
Load Average: 251.11, 123.23, 74.40
---------------------------------------------------------
Benchmark               Time             CPU   Iterations
---------------------------------------------------------
Example_origin      11730 ns        11730 ns        59580
Example_opt          3984 ns         3984 ns       173992

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Mryange
Copy link
Contributor Author

Mryange commented Mar 24, 2025

run buildall

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

TPC-H: Total hot run time: 34607 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f9aa3285e8378961305d515bec9c2af0daaf4ab7, data reload: false

------ Round 1 ----------------------------------
q1	26199	5269	5128	5128
q2	2079	288	166	166
q3	10484	1271	713	713
q4	10233	1015	541	541
q5	7670	2405	2409	2405
q6	186	167	136	136
q7	929	761	617	617
q8	9321	1332	1130	1130
q9	6827	5188	5191	5188
q10	6823	2299	1911	1911
q11	484	288	270	270
q12	345	351	220	220
q13	17754	3692	3136	3136
q14	236	232	206	206
q15	531	479	485	479
q16	622	607	587	587
q17	582	879	358	358
q18	7987	7194	7116	7116
q19	1325	972	573	573
q20	325	322	207	207
q21	4043	3434	2524	2524
q22	1064	1019	996	996
Total cold run time: 116049 ms
Total hot run time: 34607 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5208	5177	5188	5177
q2	247	328	234	234
q3	2163	2696	2359	2359
q4	1409	1898	1536	1536
q5	4579	4476	4389	4389
q6	210	167	128	128
q7	1985	1917	1711	1711
q8	2587	2554	2684	2554
q9	7247	7230	7105	7105
q10	3004	3205	2756	2756
q11	571	491	483	483
q12	682	760	598	598
q13	3557	3952	3332	3332
q14	292	310	270	270
q15	520	465	474	465
q16	653	696	650	650
q17	1183	1553	1431	1431
q18	7860	7609	7551	7551
q19	826	892	993	892
q20	1958	1992	1858	1858
q21	5243	4729	4783	4729
q22	1069	1034	982	982
Total cold run time: 53053 ms
Total hot run time: 51190 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187091 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f9aa3285e8378961305d515bec9c2af0daaf4ab7, data reload: false

query1	1005	464	487	464
query2	6567	1963	1950	1950
query3	6801	220	223	220
query4	26611	24112	23360	23360
query5	4425	662	467	467
query6	306	206	207	206
query7	4614	498	273	273
query8	304	250	237	237
query9	8631	2580	2582	2580
query10	475	326	253	253
query11	15578	15221	14933	14933
query12	164	121	101	101
query13	1651	530	383	383
query14	8950	5990	6104	5990
query15	201	185	164	164
query16	7149	640	471	471
query17	1186	692	551	551
query18	1973	388	294	294
query19	189	171	163	163
query20	115	111	116	111
query21	207	120	98	98
query22	4212	4293	4165	4165
query23	34096	33219	33074	33074
query24	8433	2368	2364	2364
query25	537	435	416	416
query26	1244	261	141	141
query27	2778	512	313	313
query28	4362	2415	2396	2396
query29	765	565	434	434
query30	281	217	192	192
query31	936	836	737	737
query32	75	63	62	62
query33	564	411	312	312
query34	768	837	494	494
query35	803	803	720	720
query36	966	986	890	890
query37	117	103	77	77
query38	4188	4339	4109	4109
query39	1445	1389	1395	1389
query40	206	115	104	104
query41	55	53	51	51
query42	116	103	100	100
query43	505	496	470	470
query44	1286	787	786	786
query45	176	173	178	173
query46	843	1043	613	613
query47	1761	1820	1713	1713
query48	398	407	296	296
query49	805	499	423	423
query50	677	726	422	422
query51	4140	4113	4146	4113
query52	110	108	99	99
query53	221	249	183	183
query54	484	492	415	415
query55	90	81	88	81
query56	267	280	267	267
query57	1157	1151	1097	1097
query58	269	253	246	246
query59	2731	2672	2616	2616
query60	288	280	278	278
query61	157	153	148	148
query62	813	725	658	658
query63	226	188	185	185
query64	4514	998	661	661
query65	4450	4391	4360	4360
query66	1145	400	293	293
query67	15931	15652	15459	15459
query68	7916	879	504	504
query69	468	297	269	269
query70	1209	1144	1106	1106
query71	441	297	249	249
query72	5522	5171	5108	5108
query73	683	597	345	345
query74	9248	9126	8891	8891
query75	3840	3244	2711	2711
query76	3552	1185	742	742
query77	794	380	285	285
query78	9969	10204	9379	9379
query79	2011	808	566	566
query80	598	518	436	436
query81	481	254	217	217
query82	460	125	95	95
query83	172	178	159	159
query84	245	93	72	72
query85	804	355	300	300
query86	336	310	293	293
query87	4467	4718	4401	4401
query88	3780	2236	2275	2236
query89	385	313	278	278
query90	1940	211	206	206
query91	143	140	111	111
query92	71	60	58	58
query93	1714	1068	579	579
query94	666	423	298	298
query95	355	279	262	262
query96	479	569	272	272
query97	3279	3353	3303	3303
query98	225	212	212	212
query99	1347	1397	1284	1284
Total cold run time: 275093 ms
Total hot run time: 187091 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.73 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f9aa3285e8378961305d515bec9c2af0daaf4ab7, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.10	0.10
query3	0.25	0.19	0.19
query4	1.58	0.19	0.20
query5	0.60	0.57	0.59
query6	1.20	0.72	0.71
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.54	0.51
query10	0.59	0.58	0.56
query11	0.15	0.11	0.11
query12	0.15	0.11	0.12
query13	0.61	0.61	0.60
query14	2.81	2.69	2.70
query15	0.93	0.86	0.84
query16	0.38	0.39	0.39
query17	1.02	1.00	1.03
query18	0.21	0.19	0.19
query19	1.92	1.85	1.84
query20	0.02	0.01	0.01
query21	15.36	0.87	0.54
query22	0.78	1.05	0.64
query23	15.05	1.38	0.60
query24	7.20	1.40	0.33
query25	0.35	0.20	0.11
query26	0.67	0.18	0.15
query27	0.05	0.04	0.05
query28	8.99	0.88	0.44
query29	12.55	4.03	3.34
query30	0.25	0.08	0.07
query31	2.82	0.58	0.38
query32	3.23	0.55	0.48
query33	3.05	3.14	3.11
query34	15.67	5.15	4.52
query35	4.57	4.49	4.54
query36	0.65	0.50	0.49
query37	0.09	0.07	0.07
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.94 s
Total hot run time: 30.73 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 100.00% (10/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 50.23% (13443/26762)
Line Coverage 39.67% (116432/293488)
Region Coverage 38.39% (59190/154164)
Branch Coverage 33.53% (29900/89182)

@Mryange Mryange marked this pull request as draft April 2, 2025 09:44
@Mryange Mryange closed this Apr 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants