Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](group commit) Fix group commit forward required param fault #38265

Merged

Conversation

Yukang-Lian
Copy link
Collaborator

Proposed changes

The error message is:

2024-07-23 17:30:09.300 INFO [suite-thread-1] (StreamLoadAction.groovy:201) - Stream load elapsed 2050 ms, is http stream: true, response: nulljava.lang.IllegalStateException: Expect frontend stream load response code is 307, but meet 200
body: {"status":"FAILED","msg":"errCode = 2, detailMessage = Required field 'user' was not present! Struct: TMasterOpRequest(user:null, db:, sql:, clientNodeHost:172.20.48.85, clientNodePort:9010, groupCommitInfo:TGroupCommitInfo(getGroupCommitLoadBeId:true, groupCommitLoadTableId:10321, isCloud:false))"}

The forward param TMasterOpRequest(FrontendService.thrift) has a required field user, which get from ConnectContext. So we set admin_user to ConnectContext to solve this problem.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yukang-Lian
Copy link
Collaborator Author

run buildall

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 23, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39765 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fba2ef323155b9bad80898e9221c3aa9166aadfa, data reload: false

------ Round 1 ----------------------------------
q1	17663	4360	4275	4275
q2	2016	188	186	186
q3	10455	1220	1175	1175
q4	10192	782	786	782
q5	7539	2698	2639	2639
q6	227	140	137	137
q7	945	599	590	590
q8	9220	2083	2060	2060
q9	8958	6587	6581	6581
q10	8730	3731	3737	3731
q11	472	233	238	233
q12	414	223	223	223
q13	17924	2970	2959	2959
q14	274	235	232	232
q15	520	483	495	483
q16	509	393	378	378
q17	964	686	634	634
q18	7955	7574	7368	7368
q19	6577	1342	1445	1342
q20	713	317	312	312
q21	4902	3228	3166	3166
q22	346	294	279	279
Total cold run time: 117515 ms
Total hot run time: 39765 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4371	4224	4272	4224
q2	363	270	284	270
q3	3025	2836	2884	2836
q4	2006	1642	1708	1642
q5	5654	5548	5501	5501
q6	218	132	133	132
q7	2215	1933	1834	1834
q8	3292	3417	3447	3417
q9	8802	8846	8820	8820
q10	4087	3831	3796	3796
q11	604	527	499	499
q12	827	646	633	633
q13	16602	3193	3191	3191
q14	330	282	299	282
q15	529	490	487	487
q16	499	438	421	421
q17	1805	1514	1535	1514
q18	8124	7828	7840	7828
q19	1734	1583	1596	1583
q20	2976	1900	1844	1844
q21	8038	4867	4906	4867
q22	582	528	531	528
Total cold run time: 76683 ms
Total hot run time: 56149 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174835 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fba2ef323155b9bad80898e9221c3aa9166aadfa, data reload: false

query1	912	372	363	363
query2	6365	1938	1889	1889
query3	6650	206	218	206
query4	23530	17600	17528	17528
query5	3626	473	476	473
query6	272	196	164	164
query7	4585	288	283	283
query8	238	194	193	193
query9	8418	2436	2390	2390
query10	442	293	271	271
query11	10491	10056	10090	10056
query12	118	84	85	84
query13	1650	384	372	372
query14	10321	8389	7824	7824
query15	223	167	165	165
query16	7585	484	495	484
query17	1509	565	560	560
query18	1713	285	297	285
query19	201	151	159	151
query20	93	82	84	82
query21	211	136	136	136
query22	4258	4181	4043	4043
query23	34094	33693	33694	33693
query24	11116	2867	3008	2867
query25	642	420	464	420
query26	1247	155	154	154
query27	2340	297	282	282
query28	6695	2071	2068	2068
query29	882	649	647	647
query30	303	155	155	155
query31	989	778	770	770
query32	96	55	58	55
query33	723	341	338	338
query34	909	490	510	490
query35	882	775	802	775
query36	1116	962	971	962
query37	151	80	86	80
query38	2939	2859	2814	2814
query39	873	804	838	804
query40	194	116	122	116
query41	45	42	45	42
query42	118	106	97	97
query43	522	442	487	442
query44	1254	727	758	727
query45	194	163	161	161
query46	1087	752	736	736
query47	1888	1788	1769	1769
query48	372	308	292	292
query49	907	404	414	404
query50	780	408	389	389
query51	6694	6700	6710	6700
query52	104	89	95	89
query53	367	293	296	293
query54	905	452	445	445
query55	74	74	77	74
query56	285	270	264	264
query57	1130	1093	1034	1034
query58	239	243	279	243
query59	2710	2977	2618	2618
query60	307	294	297	294
query61	99	93	95	93
query62	803	671	640	640
query63	319	301	295	295
query64	9522	2239	1666	1666
query65	3157	3101	3122	3101
query66	754	330	327	327
query67	15640	15128	15107	15107
query68	5509	558	551	551
query69	714	453	378	378
query70	1218	1204	1158	1158
query71	456	286	286	286
query72	8677	5401	5602	5401
query73	783	329	332	329
query74	6281	5663	5722	5663
query75	4186	2718	2683	2683
query76	3720	1029	941	941
query77	728	330	299	299
query78	9761	9048	8992	8992
query79	5997	527	524	524
query80	1225	483	483	483
query81	577	224	222	222
query82	953	139	142	139
query83	312	173	171	171
query84	323	87	89	87
query85	1347	360	308	308
query86	446	287	304	287
query87	3324	3166	3099	3099
query88	4534	2373	2388	2373
query89	505	391	400	391
query90	1918	201	198	198
query91	132	102	100	100
query92	64	50	49	49
query93	5739	530	518	518
query94	1221	284	282	282
query95	419	321	321	321
query96	611	273	270	270
query97	3234	3037	3073	3037
query98	218	247	207	207
query99	1617	1261	1254	1254
Total cold run time: 286627 ms
Total hot run time: 174835 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.84 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fba2ef323155b9bad80898e9221c3aa9166aadfa, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.03	0.04
query3	0.22	0.06	0.06
query4	1.66	0.09	0.08
query5	0.50	0.50	0.49
query6	1.12	0.74	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.56	0.50	0.49
query10	0.55	0.55	0.54
query11	0.16	0.11	0.11
query12	0.14	0.13	0.13
query13	0.62	0.59	0.60
query14	0.78	0.80	0.80
query15	0.88	0.83	0.85
query16	0.37	0.36	0.35
query17	1.04	0.96	0.98
query18	0.22	0.21	0.22
query19	1.79	1.72	1.71
query20	0.01	0.01	0.02
query21	15.46	0.81	0.66
query22	4.01	6.86	2.72
query23	18.30	1.51	1.33
query24	2.09	0.24	0.23
query25	0.16	0.09	0.09
query26	0.29	0.22	0.21
query27	0.44	0.23	0.23
query28	13.24	1.09	1.03
query29	12.61	3.33	3.31
query30	0.25	0.06	0.06
query31	2.86	0.39	0.40
query32	3.26	0.49	0.50
query33	2.96	2.93	2.96
query34	16.96	4.41	4.54
query35	4.44	4.52	4.46
query36	0.66	0.50	0.49
query37	0.18	0.16	0.16
query38	0.15	0.15	0.14
query39	0.04	0.04	0.04
query40	0.16	0.13	0.12
query41	0.10	0.05	0.04
query42	0.06	0.05	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.54 s
Total hot run time: 31.84 s

@dataroaring dataroaring merged commit 8e17d59 into apache:master Jul 24, 2024
28 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Jul 24, 2024
…8265)

## Proposed changes

<!--Describe your changes.-->

The error message is:

2024-07-23 17:30:09.300 INFO [suite-thread-1]
(StreamLoadAction.groovy:201) - Stream load elapsed 2050 ms, is http
stream: true, response: nulljava.lang.IllegalStateException: Expect
frontend stream load response code is 307, but meet 200
body: {"status":"FAILED","msg":"errCode = 2, detailMessage = Required
field 'user' was not present! Struct: TMasterOpRequest(user:null, db:,
sql:, clientNodeHost:172.20.48.85, clientNodePort:9010,
groupCommitInfo:TGroupCommitInfo(getGroupCommitLoadBeId:true,
groupCommitLoadTableId:10321, isCloud:false))"}

The forward param `TMasterOpRequest`(FrontendService.thrift) has a
required field `user`, which get from ConnectContext. So we set
admin_user to ConnectContext to solve this problem.
Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Jul 25, 2024
…ache#38265)

## Proposed changes

<!--Describe your changes.-->

The error message is:

2024-07-23 17:30:09.300 INFO [suite-thread-1]
(StreamLoadAction.groovy:201) - Stream load elapsed 2050 ms, is http
stream: true, response: nulljava.lang.IllegalStateException: Expect
frontend stream load response code is 307, but meet 200
body: {"status":"FAILED","msg":"errCode = 2, detailMessage = Required
field 'user' was not present! Struct: TMasterOpRequest(user:null, db:,
sql:, clientNodeHost:172.20.48.85, clientNodePort:9010,
groupCommitInfo:TGroupCommitInfo(getGroupCommitLoadBeId:true,
groupCommitLoadTableId:10321, isCloud:false))"}

The forward param `TMasterOpRequest`(FrontendService.thrift) has a
required field `user`, which get from ConnectContext. So we set
admin_user to ConnectContext to solve this problem.
yiguolei pushed a commit that referenced this pull request Jul 25, 2024
…blems (#38345)

## Proposed changes

<!--Describe your changes.-->

Pick #38228 #38265
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants