Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](load) Fix uninitiated load_type (not a valid value for type) of StreamLoadContext when operate_txn_2pc #38033

Merged
merged 1 commit into from
Jul 18, 2024
Merged

[Fix](load) Fix uninitiated load_type (not a valid value for type) of StreamLoadContext when operate_txn_2pc #38033

merged 1 commit into from
Jul 18, 2024

Conversation

liaoxin01
Copy link
Contributor

/root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:384:14: runtime error: load of value 3200171710, which is not a valid value for type 'TLoadType::type'
#0 0x55be4e2cf808 in doris::StreamLoadExecutor::collect_load_stat(doris::StreamLoadContext*, doris::TTxnCommitAttachment*) /root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:384:14
#1 0x55be4e2d2666 in doris::StreamLoadExecutor::rollback_txn(doris::StreamLoadContext*) /root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:364:9
#2 0x55be80413748 in doris::CloudStreamLoadExecutor::operate_txn_2pc(doris::StreamLoadContext*) /root/doris/be/src/cloud/cloud_stream_load_executor.cpp:80:33
#3 0x55be50cad893 in doris::StreamLoad2PCAction::handle(doris::HttpRequest*) /root/doris/be/src/http/action/stream_load_2pc.cpp:85:49

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@liaoxin01
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@gavinchou gavinchou changed the title [Fix](load) Fix load_type of StreamLoadContext is not a valid value for type when operate_txn_2pc [Fix](load) Fix uninitiated load_type (not a valid value for type) of StreamLoadContext when operate_txn_2pc Jul 17, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 17, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39853 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9b78d58dd101feee2ac4b9cfd9d53d7ff8bf1d74, data reload: false

------ Round 1 ----------------------------------
q1	17611	5310	4332	4332
q2	2015	193	187	187
q3	10628	1331	1113	1113
q4	10556	800	763	763
q5	8128	2768	2687	2687
q6	228	144	143	143
q7	973	612	609	609
q8	9249	2082	2113	2082
q9	9261	6673	6569	6569
q10	8819	3768	3823	3768
q11	449	237	247	237
q12	396	222	218	218
q13	17800	2986	2979	2979
q14	287	241	246	241
q15	536	482	477	477
q16	516	390	377	377
q17	962	661	694	661
q18	8187	7447	7310	7310
q19	6361	1402	1302	1302
q20	705	323	339	323
q21	4881	3200	3316	3200
q22	342	285	275	275
Total cold run time: 118890 ms
Total hot run time: 39853 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4365	4255	4233	4233
q2	367	270	274	270
q3	2990	2751	2763	2751
q4	1926	1629	1637	1629
q5	5317	5273	5329	5273
q6	215	129	131	129
q7	2070	1708	1742	1708
q8	3170	3344	3279	3279
q9	8475	8423	8407	8407
q10	3899	3710	3634	3634
q11	586	474	493	474
q12	799	605	620	605
q13	16349	2949	2992	2949
q14	302	269	276	269
q15	514	472	473	472
q16	458	404	422	404
q17	1753	1482	1471	1471
q18	7536	7440	7355	7355
q19	1657	1432	1468	1432
q20	1973	1784	1792	1784
q21	4835	4717	4722	4717
q22	553	489	490	489
Total cold run time: 70109 ms
Total hot run time: 53734 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173020 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9b78d58dd101feee2ac4b9cfd9d53d7ff8bf1d74, data reload: false

query1	912	369	363	363
query2	6469	1801	1870	1801
query3	6647	205	219	205
query4	28245	17604	17191	17191
query5	4296	493	499	493
query6	271	183	170	170
query7	4592	291	280	280
query8	239	190	183	183
query9	8638	2435	2424	2424
query10	443	296	273	273
query11	10770	9967	10088	9967
query12	131	95	82	82
query13	1652	377	363	363
query14	10175	7778	7779	7778
query15	224	170	176	170
query16	7916	320	314	314
query17	1821	560	547	547
query18	1962	298	282	282
query19	207	158	152	152
query20	92	82	85	82
query21	203	131	125	125
query22	4183	3977	4026	3977
query23	33872	33205	33199	33199
query24	11110	2880	2889	2880
query25	639	387	386	386
query26	1330	147	151	147
query27	2874	271	276	271
query28	7671	2015	1996	1996
query29	911	657	647	647
query30	287	149	157	149
query31	966	740	758	740
query32	98	55	55	55
query33	795	388	287	287
query34	889	469	485	469
query35	683	588	579	579
query36	1091	955	922	922
query37	150	78	77	77
query38	2871	2725	2716	2716
query39	888	831	810	810
query40	271	122	122	122
query41	50	45	45	45
query42	124	102	103	102
query43	504	466	477	466
query44	1274	756	738	738
query45	193	161	156	156
query46	1086	739	742	739
query47	1818	1771	1797	1771
query48	360	292	292	292
query49	1096	423	417	417
query50	769	391	390	390
query51	6879	6810	6774	6774
query52	100	93	91	91
query53	361	292	295	292
query54	961	445	439	439
query55	77	75	74	74
query56	316	283	271	271
query57	1145	1036	1056	1036
query58	251	264	267	264
query59	2661	2692	2583	2583
query60	314	276	279	276
query61	97	103	98	98
query62	825	650	659	650
query63	326	299	293	293
query64	10661	2279	1677	1677
query65	3180	3087	3111	3087
query66	1385	351	333	333
query67	15602	15066	15041	15041
query68	5843	533	544	533
query69	696	457	342	342
query70	1111	1161	1131	1131
query71	455	276	279	276
query72	8993	5687	5575	5575
query73	792	335	322	322
query74	6167	5656	5712	5656
query75	4267	2663	2666	2663
query76	3788	1016	952	952
query77	701	311	312	311
query78	9852	8961	9055	8961
query79	3932	517	521	517
query80	2323	477	475	475
query81	594	231	222	222
query82	1455	133	138	133
query83	275	172	169	169
query84	278	89	85	85
query85	1582	316	301	301
query86	421	312	308	308
query87	3322	3181	3151	3151
query88	4519	2386	2374	2374
query89	508	397	382	382
query90	1866	195	193	193
query91	141	100	104	100
query92	59	48	51	48
query93	5044	507	496	496
query94	1162	214	214	214
query95	412	321	312	312
query96	604	268	271	268
query97	3204	3026	3039	3026
query98	215	201	188	188
query99	1611	1267	1281	1267
Total cold run time: 296091 ms
Total hot run time: 173020 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.05 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9b78d58dd101feee2ac4b9cfd9d53d7ff8bf1d74, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.03
query3	0.22	0.04	0.05
query4	1.67	0.07	0.08
query5	0.50	0.49	0.50
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.56	0.51	0.50
query10	0.55	0.54	0.55
query11	0.15	0.12	0.11
query12	0.15	0.13	0.13
query13	0.59	0.59	0.59
query14	0.76	0.77	0.77
query15	0.85	0.81	0.80
query16	0.35	0.37	0.36
query17	1.03	1.02	1.04
query18	0.23	0.21	0.21
query19	1.86	1.67	1.71
query20	0.01	0.00	0.00
query21	15.42	0.73	0.66
query22	4.11	6.90	2.33
query23	18.29	1.38	1.31
query24	2.10	0.23	0.22
query25	0.15	0.09	0.08
query26	0.30	0.21	0.21
query27	0.45	0.23	0.23
query28	13.22	1.03	0.99
query29	12.62	3.33	3.31
query30	0.25	0.06	0.05
query31	2.87	0.42	0.40
query32	3.24	0.47	0.48
query33	2.94	2.88	2.89
query34	17.08	4.36	4.39
query35	4.42	4.45	4.40
query36	0.65	0.46	0.47
query37	0.21	0.16	0.15
query38	0.16	0.15	0.15
query39	0.05	0.03	0.03
query40	0.15	0.12	0.12
query41	0.10	0.05	0.04
query42	0.06	0.04	0.05
query43	0.05	0.04	0.03
Total cold run time: 109.69 s
Total hot run time: 31.05 s

dataroaring
dataroaring approved these changes Jul 18, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit ec762f2 into apache:master Jul 18, 2024
28 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Jul 19, 2024
@liaoxin01 @dataroaring
[Fix](load) Fix uninitiated load_type (not a valid value for type) of…
1ded643 
… StreamLoadContext when operate_txn_2pc (#38033)

/root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:384:14:
runtime error: load of value 3200171710, which is not a valid value for
type 'TLoadType::type'
#0 0x55be4e2cf808 in
doris::StreamLoadExecutor::collect_load_stat(doris::StreamLoadContext*,
doris::TTxnCommitAttachment*)
/root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:384:14
#1 0x55be4e2d2666 in
doris::StreamLoadExecutor::rollback_txn(doris::StreamLoadContext*)
/root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:364:9
#2 0x55be80413748 in
doris::CloudStreamLoadExecutor::operate_txn_2pc(doris::StreamLoadContext*)
/root/doris/be/src/cloud/cloud_stream_load_executor.cpp:80:33
#3 0x55be50cad893 in
doris::StreamLoad2PCAction::handle(doris::HttpRequest*)
/root/doris/be/src/http/action/stream_load_2pc.cpp:85:49
liaoxin01 added a commit to liaoxin01/doris that referenced this pull request Jul 22, 2024
@liaoxin01
[Fix](load) Fix uninitiated load_type (not a valid value for type) of…
21991dc 
… StreamLoadContext when operate_txn_2pc (apache#38033)

/root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:384:14:
runtime error: load of value 3200171710, which is not a valid value for
type 'TLoadType::type'
#0 0x55be4e2cf808 in
doris::StreamLoadExecutor::collect_load_stat(doris::StreamLoadContext*,
doris::TTxnCommitAttachment*)
/root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:384:14
apache#1 0x55be4e2d2666 in
doris::StreamLoadExecutor::rollback_txn(doris::StreamLoadContext*)
/root/doris/be/src/runtime/stream_load/stream_load_executor.cpp:364:9
apache#2 0x55be80413748 in
doris::CloudStreamLoadExecutor::operate_txn_2pc(doris::StreamLoadContext*)
/root/doris/be/src/cloud/cloud_stream_load_executor.cpp:80:33
apache#3 0x55be50cad893 in
doris::StreamLoad2PCAction::handle(doris::HttpRequest*)
/root/doris/be/src/http/action/stream_load_2pc.cpp:85:49
@liaoxin01 liaoxin01 mentioned this pull request Jul 22, 2024
Merged
dataroaring pushed a commit that referenced this pull request Jul 22, 2024
@liaoxin01
[Fix](load) Fix uninitiated load_type (not a valid value for type) of…
ce1c268 
… StreamLoadContext when operate_txn_2pc #38033 (#38187)

cherry pick from #38033
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@gavinchou gavinchou gavinchou approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@liaoxin01 @doris-robot @gavinchou @dataroaring