Skip to content

[fix](iceberg)fix iceberg v3 row lineage count distinct error result#63826

Open
hubgeter wants to merge 1 commit into
apache:masterfrom
hubgeter:fix_iceberg_agg
Open

[fix](iceberg)fix iceberg v3 row lineage count distinct error result#63826
hubgeter wants to merge 1 commit into
apache:masterfrom
hubgeter:fix_iceberg_agg

Conversation

@hubgeter
Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hubgeter
Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 32032 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7c24abc3ea60d23f79fe8638c82b2c2f0edecdfa, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17824	4138	4054	4054
q2	q3	10864	1431	807	807
q4	4692	478	345	345
q5	7588	2288	2123	2123
q6	249	177	138	138
q7	952	814	636	636
q8	9386	1687	1646	1646
q9	5178	5020	4985	4985
q10	6382	2242	1891	1891
q11	458	274	253	253
q12	625	423	294	294
q13	18093	3369	2767	2767
q14	266	257	243	243
q15	q16	833	769	711	711
q17	927	941	963	941
q18	7050	5894	5631	5631
q19	1350	1236	1263	1236
q20	553	452	291	291
q21	6349	2898	2723	2723
q22	476	373	317	317
Total cold run time: 100095 ms
Total hot run time: 32032 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4957	4845	4766	4766
q2	q3	4858	5308	4648	4648
q4	2098	2223	1383	1383
q5	5086	4782	4698	4698
q6	241	178	131	131
q7	1921	1756	1585	1585
q8	2453	2145	2197	2145
q9	7950	7585	7477	7477
q10	4770	4703	4228	4228
q11	541	391	359	359
q12	730	748	525	525
q13	3032	3337	2838	2838
q14	282	283	272	272
q15	q16	691	701	613	613
q17	1292	1274	1277	1274
q18	7396	6715	6876	6715
q19	1137	1092	1096	1092
q20	2233	2226	1970	1970
q21	5364	4678	4516	4516
q22	542	503	414	414
Total cold run time: 57574 ms
Total hot run time: 51649 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 173042 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7c24abc3ea60d23f79fe8638c82b2c2f0edecdfa, data reload: false

query5	4342	669	524	524
query6	333	221	209	209
query7	4423	576	307	307
query8	327	244	222	222
query9	8806	4113	4072	4072
query10	473	352	301	301
query11	5782	2514	2257	2257
query12	184	130	122	122
query13	1293	630	422	422
query14	6114	5477	5188	5188
query14_1	4524	4513	4496	4496
query15	211	205	194	194
query16	1009	450	433	433
query17	1144	764	614	614
query18	2497	516	372	372
query19	220	205	170	170
query20	142	137	133	133
query21	225	144	121	121
query22	13772	13646	13390	13390
query23	17427	16651	16322	16322
query23_1	16305	16324	16485	16324
query24	7764	1824	1360	1360
query24_1	1330	1370	1350	1350
query25	610	511	465	465
query26	1336	333	175	175
query27	2667	564	349	349
query28	4474	2023	1977	1977
query29	1019	669	535	535
query30	315	252	211	211
query31	1137	1093	950	950
query32	89	80	80	80
query33	565	369	323	323
query34	1191	1149	667	667
query35	797	825	714	714
query36	1443	1384	1285	1285
query37	169	109	95	95
query38	3234	3197	3090	3090
query39	944	941	913	913
query39_1	906	877	883	877
query40	254	155	132	132
query41	72	70	71	70
query42	119	115	114	114
query43	355	352	310	310
query44	
query45	219	211	202	202
query46	1103	1287	796	796
query47	2393	2344	2253	2253
query48	424	435	314	314
query49	652	515	418	418
query50	1033	366	265	265
query51	4340	4327	4292	4292
query52	113	114	99	99
query53	266	300	208	208
query54	344	288	290	288
query55	97	96	91	91
query56	334	330	324	324
query57	1437	1483	1323	1323
query58	294	280	280	280
query59	1665	1699	1481	1481
query60	321	334	309	309
query61	154	157	155	155
query62	703	652	584	584
query63	241	201	210	201
query64	2454	790	618	618
query65	
query66	1731	479	358	358
query67	29942	29114	29792	29114
query68	
query69	478	343	314	314
query70	1092	1048	1047	1047
query71	313	273	277	273
query72	3031	2695	2497	2497
query73	878	766	414	414
query74	5154	4947	4853	4853
query75	2698	2643	2289	2289
query76	2279	1159	796	796
query77	419	433	346	346
query78	12480	12598	12032	12032
query79	1470	1070	775	775
query80	687	558	469	469
query81	465	286	247	247
query82	1369	168	129	129
query83	358	283	250	250
query84	260	149	112	112
query85	895	531	453	453
query86	410	331	304	304
query87	3443	3405	3261	3261
query88	3650	2728	2733	2728
query89	444	411	350	350
query90	1992	201	206	201
query91	180	175	145	145
query92	82	82	76	76
query93	1501	1528	919	919
query94	539	344	302	302
query95	691	502	357	357
query96	1147	775	353	353
query97	2743	2706	2615	2615
query98	242	243	243	243
query99	1172	1148	1043	1043
Total cold run time: 255737 ms
Total hot run time: 173042 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (11/11) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.82% (28091/38055)
Line Coverage 57.74% (305101/528412)
Region Coverage 54.97% (255758/465240)
Branch Coverage 56.41% (110342/195614)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants