Skip to content

[fix](parquet) Fix wrong encoding for parquet page v2#63305

Open
Gabriel39 wants to merge 1 commit into
apache:masterfrom
Gabriel39:fix_0515
Open

[fix](parquet) Fix wrong encoding for parquet page v2#63305
Gabriel39 wants to merge 1 commit into
apache:masterfrom
Gabriel39:fix_0515

Conversation

@Gabriel39
Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223
Copy link
Copy Markdown
Member

run external

@Gabriel39
Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31130 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7a527c7d811d800a4d48236ef1f8fc93c38e1e3c, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17680	3913	3965	3913
q2	q3	10802	1448	814	814
q4	4700	474	342	342
q5	7542	2287	2105	2105
q6	250	182	140	140
q7	957	781	625	625
q8	9408	1714	1541	1541
q9	5182	4935	4907	4907
q10	6381	2080	1781	1781
q11	438	279	250	250
q12	632	429	305	305
q13	18121	3387	2785	2785
q14	265	261	236	236
q15	q16	822	784	713	713
q17	940	994	1047	994
q18	7026	5640	5593	5593
q19	1326	1346	1081	1081
q20	511	398	260	260
q21	5881	2612	2436	2436
q22	438	368	309	309
Total cold run time: 99302 ms
Total hot run time: 31130 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4322	4275	4283	4275
q2	q3	4551	4913	4318	4318
q4	2149	2209	1432	1432
q5	4399	4322	4328	4322
q6	237	186	137	137
q7	2212	1963	1621	1621
q8	2783	2279	2205	2205
q9	8029	8039	7858	7858
q10	4555	4497	3996	3996
q11	595	432	391	391
q12	777	730	533	533
q13	3375	3624	3010	3010
q14	290	291	282	282
q15	q16	736	736	652	652
q17	1404	1343	1401	1343
q18	7809	7384	7301	7301
q19	1207	1153	1161	1153
q20	2219	2219	1959	1959
q21	5403	4667	4579	4579
q22	523	479	422	422
Total cold run time: 57575 ms
Total hot run time: 51789 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 169421 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7a527c7d811d800a4d48236ef1f8fc93c38e1e3c, data reload: false

query5	4322	650	514	514
query6	342	217	200	200
query7	4262	573	303	303
query8	336	235	221	221
query9	8818	3992	3977	3977
query10	442	346	289	289
query11	5819	2366	2215	2215
query12	193	137	125	125
query13	1273	596	444	444
query14	6014	5382	5088	5088
query14_1	4389	4417	4350	4350
query15	211	205	184	184
query16	997	486	451	451
query17	1171	762	619	619
query18	2768	477	345	345
query19	226	194	155	155
query20	139	131	131	131
query21	216	143	119	119
query22	13602	13666	13310	13310
query23	17205	16321	16088	16088
query23_1	16126	16167	16250	16167
query24	7607	1769	1288	1288
query24_1	1307	1317	1291	1291
query25	546	474	427	427
query26	1321	331	177	177
query27	2638	558	338	338
query28	4440	1938	1923	1923
query29	981	627	504	504
query30	313	243	197	197
query31	1109	1079	942	942
query32	87	77	72	72
query33	518	356	290	290
query34	1169	1125	635	635
query35	791	783	685	685
query36	1302	1349	1221	1221
query37	156	112	93	93
query38	3218	3137	3083	3083
query39	953	940	903	903
query39_1	870	903	891	891
query40	228	145	126	126
query41	67	65	65	65
query42	109	110	108	108
query43	321	325	296	296
query44	
query45	214	209	197	197
query46	1093	1187	725	725
query47	2295	2416	2231	2231
query48	411	406	294	294
query49	634	502	386	386
query50	929	363	254	254
query51	4348	4301	4254	4254
query52	110	106	93	93
query53	253	282	210	210
query54	323	286	272	272
query55	96	93	85	85
query56	311	311	310	310
query57	1403	1410	1486	1410
query58	310	272	266	266
query59	1613	1617	1431	1431
query60	334	332	352	332
query61	160	161	159	159
query62	663	641	552	552
query63	250	201	204	201
query64	2383	826	639	639
query65	
query66	1669	511	360	360
query67	30086	30018	29792	29792
query68	
query69	459	341	308	308
query70	1054	993	981	981
query71	300	279	276	276
query72	3021	2701	2420	2420
query73	839	757	443	443
query74	5087	4924	4775	4775
query75	2702	2641	2277	2277
query76	2325	1137	736	736
query77	410	428	358	358
query78	12122	12098	11581	11581
query79	1276	1031	736	736
query80	646	585	499	499
query81	451	286	259	259
query82	247	160	129	129
query83	282	290	264	264
query84	277	149	115	115
query85	945	635	616	616
query86	366	360	321	321
query87	3382	3404	3233	3233
query88	3489	2659	2662	2659
query89	429	384	340	340
query90	2201	181	186	181
query91	177	171	139	139
query92	81	81	72	72
query93	1479	1461	936	936
query94	522	348	312	312
query95	662	385	352	352
query96	1080	778	355	355
query97	2684	2743	2549	2549
query98	236	227	228	227
query99	1106	1109	1003	1003
Total cold run time: 252108 ms
Total hot run time: 169421 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants