Skip to content

Revert "[fix](simd-json-reader) fix SIMD json reader lose data and support stream parser (#35781) #36043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jun 7, 2024
Merged

Revert "[fix](simd-json-reader) fix SIMD json reader lose data and support stream parser (#35781) #36043

dataroaring merged 1 commit into from
Jun 7, 2024

Conversation

@sollhui
Copy link
Contributor

@sollhui sollhui commented Jun 7, 2024

#35781

This reverts commit cb7e8bf.

@doris-robot
Copy link

doris-robot commented Jun 7, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@sollhui
Copy link
Contributor Author

sollhui commented Jun 7, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Jun 7, 2024

TeamCity be ut coverage result:
Function Coverage: 36.44% (8974/24630)
Line Coverage: 27.97% (73332/262199)
Region Coverage: 27.37% (38005/138860)
Branch Coverage: 23.99% (19301/80452)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f8c51851d8d91b92bbcf508b20a7ec88210d9169_f8c51851d8d91b92bbcf508b20a7ec88210d9169/report/index.html

@doris-robot
Copy link

doris-robot commented Jun 7, 2024

TPC-H: Total hot run time: 41025 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f8c51851d8d91b92bbcf508b20a7ec88210d9169, data reload: false

------ Round 1 ----------------------------------
q1	17627	4343	4258	4258
q2	2017	191	191	191
q3	10468	1135	1215	1135
q4	10182	821	855	821
q5	7485	2738	2664	2664
q6	226	137	141	137
q7	951	637	627	627
q8	9223	2110	2117	2110
q9	9330	6668	6679	6668
q10	9180	3827	3955	3827
q11	454	250	244	244
q12	459	242	244	242
q13	17473	3225	3219	3219
q14	275	235	232	232
q15	527	485	476	476
q16	508	401	397	397
q17	1002	782	684	684
q18	8404	7848	7744	7744
q19	4690	1366	1486	1366
q20	666	336	342	336
q21	5093	3307	4096	3307
q22	404	340	357	340
Total cold run time: 116644 ms
Total hot run time: 41025 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4570	4459	4473	4459
q2	377	279	274	274
q3	3124	2870	2872	2870
q4	2104	1690	1625	1625
q5	5333	5551	5541	5541
q6	225	127	128	127
q7	2290	1869	1854	1854
q8	3278	3395	3406	3395
q9	8732	8626	8651	8626
q10	4065	3840	3833	3833
q11	569	477	488	477
q12	773	610	606	606
q13	15777	3065	3153	3065
q14	311	299	266	266
q15	535	472	475	472
q16	490	424	437	424
q17	1786	1487	1513	1487
q18	8079	7731	7362	7362
q19	1763	1575	1548	1548
q20	3038	1809	1803	1803
q21	4953	4842	4634	4634
q22	590	560	544	544
Total cold run time: 72762 ms
Total hot run time: 55292 ms

@doris-robot
Copy link

doris-robot commented Jun 7, 2024

TPC-DS: Total hot run time: 171611 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f8c51851d8d91b92bbcf508b20a7ec88210d9169, data reload: false

query1	928	379	377	377
query2	6483	2523	2388	2388
query3	6650	205	207	205
query4	19045	17440	17456	17440
query5	4178	461	442	442
query6	255	154	152	152
query7	4579	301	291	291
query8	333	299	288	288
query9	8553	2378	2372	2372
query10	449	313	263	263
query11	10437	9962	9856	9856
query12	140	82	85	82
query13	1634	347	367	347
query14	8486	7666	6990	6990
query15	228	187	177	177
query16	7211	268	258	258
query17	1352	557	518	518
query18	1892	295	284	284
query19	201	157	156	156
query20	90	84	80	80
query21	209	132	129	129
query22	4449	3942	3889	3889
query23	33918	33132	33106	33106
query24	11245	2830	2784	2784
query25	624	354	351	351
query26	1373	173	151	151
query27	3077	313	311	311
query28	7603	2022	2006	2006
query29	937	601	591	591
query30	293	147	146	146
query31	937	726	712	712
query32	94	50	59	50
query33	758	269	274	269
query34	980	468	482	468
query35	743	610	653	610
query36	1139	974	901	901
query37	153	89	68	68
query38	2887	2754	2726	2726
query39	839	775	784	775
query40	205	122	122	122
query41	56	53	51	51
query42	118	97	97	97
query43	581	546	555	546
query44	1226	722	745	722
query45	193	165	163	163
query46	1055	731	705	705
query47	1876	1762	1749	1749
query48	363	301	294	294
query49	1093	408	405	405
query50	770	397	393	393
query51	6829	6573	6672	6573
query52	101	90	91	90
query53	356	293	289	289
query54	856	452	442	442
query55	73	71	73	71
query56	328	248	255	248
query57	1128	1036	1090	1036
query58	254	238	261	238
query59	3642	3280	3145	3145
query60	287	274	284	274
query61	88	88	87	87
query62	644	433	446	433
query63	311	285	290	285
query64	8968	2177	1728	1728
query65	3177	3152	3148	3148
query66	1084	332	332	332
query67	15714	14840	15050	14840
query68	4995	549	531	531
query69	582	424	347	347
query70	1147	1165	1035	1035
query71	419	277	284	277
query72	7930	5075	5449	5075
query73	775	316	320	316
query74	5870	5438	5454	5438
query75	3393	2666	2639	2639
query76	3020	911	918	911
query77	653	301	304	301
query78	10186	9691	10001	9691
query79	2123	510	514	510
query80	908	469	453	453
query81	572	222	222	222
query82	794	100	97	97
query83	255	166	194	166
query84	241	84	83	83
query85	1588	272	371	272
query86	497	313	317	313
query87	3266	3088	3080	3080
query88	3967	2320	2322	2320
query89	483	388	377	377
query90	1769	187	192	187
query91	127	97	98	97
query92	63	49	52	49
query93	1880	507	505	505
query94	1252	181	182	181
query95	405	305	311	305
query96	576	266	269	266
query97	3152	3040	3009	3009
query98	216	202	200	200
query99	1318	871	843	843
Total cold run time: 271180 ms
Total hot run time: 171611 ms

@doris-robot
Copy link

doris-robot commented Jun 7, 2024

ClickBench: Total hot run time: 30.89 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f8c51851d8d91b92bbcf508b20a7ec88210d9169, data reload: false

query1	0.03	0.03	0.04
query2	0.09	0.04	0.03
query3	0.23	0.06	0.04
query4	1.68	0.07	0.07
query5	0.50	0.50	0.49
query6	1.12	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.54	0.49	0.50
query10	0.56	0.55	0.53
query11	0.15	0.11	0.11
query12	0.14	0.12	0.12
query13	0.58	0.59	0.59
query14	0.77	0.78	0.78
query15	0.83	0.80	0.80
query16	0.37	0.36	0.36
query17	0.99	0.99	0.98
query18	0.23	0.26	0.25
query19	1.78	1.67	1.68
query20	0.02	0.01	0.00
query21	15.74	0.65	0.64
query22	4.54	7.01	2.24
query23	18.33	1.39	1.33
query24	2.12	0.21	0.22
query25	0.15	0.08	0.09
query26	0.26	0.18	0.17
query27	0.08	0.08	0.08
query28	13.37	1.11	1.06
query29	13.13	3.27	3.31
query30	0.24	0.06	0.06
query31	2.87	0.40	0.39
query32	3.25	0.45	0.46
query33	2.85	2.91	2.87
query34	17.05	4.41	4.47
query35	4.52	4.62	4.47
query36	0.65	0.48	0.47
query37	0.18	0.15	0.15
query38	0.16	0.14	0.14
query39	0.05	0.03	0.04
query40	0.15	0.15	0.16
query41	0.09	0.05	0.04
query42	0.05	0.04	0.05
query43	0.04	0.04	0.03
Total cold run time: 110.55 s
Total hot run time: 30.89 s

liaoxin01
liaoxin01 approved these changes Jun 7, 2024
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jun 7, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 7, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 7, 2024

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Jun 7, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit ad4912f into apache:master Jun 7, 2024
dataroaring pushed a commit that referenced this pull request Jun 8, 2024
@sollhui @dataroaring
Revert "[fix](simd-json-reader) fix SIMD json reader lose data and su…
ca83531 
…pport stream parser (#35781) (#36043)

#35781

This reverts commit cb7e8bf.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sollhui @doris-robot @liaoxin01 @dataroaring