Skip to content

[fix](variable) modify @@auto_commit column type to BIGINT #36584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 2 commits into from
Jun 20, 2024
Merged

[fix](variable) modify @@auto_commit column type to BIGINT #36584

morningman merged 2 commits into from
Jun 20, 2024

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Jun 20, 2024

@morningman
[fix](variables) change column type of @@autocommit to BIGINT (apache…
5c86400 
…#33282)

Some of mysql connector (eg, dotnet MySQL.Data) rely on variable's column type to make connection.
eg, `select @@autocommit` should with column type `BIGINT`, not `BIT`, otherwise it will throw error like:

```
System.FormatException: The input string 'True' was not in a correct format.
   at System.Number.ThrowFormatException[TChar](ReadOnlySpan`1 value)
   at System.Convert.ToInt32(String value)
   at MySql.Data.MySqlClient.Driver.LoadCharacterSetsAsync(MySqlConnection connection, Boolean execAsync, CancellationToken cancellationToken)
   at MySql.Data.MySqlClient.Driver.ConfigureAsync(MySqlConnection connection, Boolean execAsync, CancellationToken cancellationToken)
   at MySql.Data.MySqlClient.MySqlConnection.OpenAsync(Boolean execAsync, CancellationToken cancellationToken)
   at MySql.Data.MySqlClient.MySqlConnection.Open()
```

In this PR, I add a new field of `VarAttr`: `convertBoolToLongMethod`, if set, it will convert boolean to long.
And set it for `autocommit`
@morningman
[fix](variable) modify @@auto_commit column type to BIGINT in Nereids. …
0469dd6 
…apache#33887
@doris-robot
Copy link

doris-robot commented Jun 20, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Contributor Author

morningman commented Jun 20, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 20, 2024

TPC-H: Total hot run time: 49658 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0469dd6eab880e6f88376a7d0fc08ccb94ec843a, data reload: false

------ Round 1 ----------------------------------
q1	18098	4372	4337	4337
q2	2072	163	149	149
q3	10438	1928	1923	1923
q4	10335	1230	1318	1230
q5	9181	3842	3878	3842
q6	242	150	125	125
q7	2104	1598	1587	1587
q8	9307	2732	2706	2706
q9	10674	10301	10128	10128
q10	8611	3510	3516	3510
q11	410	247	249	247
q12	473	310	305	305
q13	18357	3935	3998	3935
q14	353	327	322	322
q15	507	450	456	450
q16	685	577	560	560
q17	1138	978	985	978
q18	7240	6900	6973	6900
q19	1808	1637	1559	1559
q20	546	307	294	294
q21	4539	4186	4117	4117
q22	524	456	454	454
Total cold run time: 117642 ms
Total hot run time: 49658 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4379	4388	4344	4344
q2	328	227	225	225
q3	4203	4128	4162	4128
q4	2767	2767	2758	2758
q5	7208	7103	7102	7102
q6	241	123	120	120
q7	3242	2863	2804	2804
q8	4376	4490	4515	4490
q9	17191	17072	17086	17072
q10	4414	4337	4272	4272
q11	760	696	681	681
q12	1034	876	857	857
q13	8964	3766	3795	3766
q14	465	447	421	421
q15	511	461	464	461
q16	727	674	671	671
q17	3887	3829	3879	3829
q18	8679	8770	8817	8770
q19	1722	1727	1659	1659
q20	2433	2200	2120	2120
q21	8443	8378	8392	8378
q22	1072	963	944	944
Total cold run time: 87046 ms
Total hot run time: 79872 ms

@doris-robot
Copy link

doris-robot commented Jun 20, 2024

TPC-DS: Total hot run time: 204113 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0469dd6eab880e6f88376a7d0fc08ccb94ec843a, data reload: false

query1	932	416	376	376
query2	6535	2864	2817	2817
query3	6917	206	201	201
query4	20678	18211	18146	18146
query5	19732	6502	6592	6502
query6	332	224	228	224
query7	4503	306	316	306
query8	400	430	429	429
query9	3156	2756	2674	2674
query10	427	296	317	296
query11	11478	10664	10717	10664
query12	127	80	76	76
query13	5592	693	707	693
query14	18149	13162	13632	13162
query15	367	244	252	244
query16	6446	292	269	269
query17	1445	1483	880	880
query18	2265	420	417	417
query19	204	161	155	155
query20	82	83	89	83
query21	198	93	95	93
query22	5177	4979	4946	4946
query23	32473	31897	32080	31897
query24	6827	6494	6553	6494
query25	537	434	435	434
query26	474	173	169	169
query27	1554	294	300	294
query28	6169	2372	2374	2372
query29	2825	2753	2852	2753
query30	247	167	164	164
query31	898	745	719	719
query32	73	65	63	63
query33	384	292	267	267
query34	858	474	489	474
query35	1116	939	918	918
query36	1337	1257	1184	1184
query37	90	59	61	59
query38	3103	2957	2944	2944
query39	1397	1346	1318	1318
query40	205	96	92	92
query41	47	44	43	43
query42	79	86	87	86
query43	709	615	705	615
query44	1139	722	726	722
query45	251	242	233	233
query46	1257	968	1005	968
query47	2136	1695	1824	1695
query48	1050	719	734	719
query49	608	378	372	372
query50	860	613	596	596
query51	4807	4599	4689	4599
query52	99	75	93	75
query53	457	335	327	327
query54	2649	2471	2473	2471
query55	87	85	88	85
query56	232	226	231	226
query57	1118	1078	1263	1078
query58	224	211	208	208
query59	4152	4046	3716	3716
query60	212	212	222	212
query61	99	96	100	96
query62	770	429	461	429
query63	481	338	350	338
query64	2188	1547	1511	1511
query65	3635	3557	3527	3527
query66	772	374	383	374
query67	17839	15569	15760	15569
query68	8277	685	670	670
query69	563	355	359	355
query70	1540	1518	1362	1362
query71	410	309	311	309
query72	6579	3512	3523	3512
query73	746	324	321	321
query74	6341	5821	5842	5821
query75	4643	3724	3666	3666
query76	4720	1143	1160	1143
query77	659	261	265	261
query78	12747	11758	12757	11758
query79	8926	643	649	643
query80	2159	413	405	405
query81	499	238	231	231
query82	686	97	102	97
query83	172	144	136	136
query84	259	70	71	70
query85	1134	317	310	310
query86	362	286	341	286
query87	3180	2987	3031	2987
query88	4781	2336	2331	2331
query89	387	288	298	288
query90	1797	219	217	217
query91	188	152	145	145
query92	58	52	52	52
query93	3752	614	549	549
query94	870	212	217	212
query95	1129	1068	1051	1051
query96	649	349	332	332
query97	6471	6412	6454	6412
query98	196	173	175	173
query99	3002	914	850	850
Total cold run time: 311548 ms
Total hot run time: 204113 ms

@doris-robot
Copy link

doris-robot commented Jun 20, 2024

ClickBench: Total hot run time: 30.37 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0469dd6eab880e6f88376a7d0fc08ccb94ec843a, data reload: false

query1	0.02	0.02	0.01
query2	0.06	0.02	0.03
query3	0.24	0.05	0.05
query4	1.80	0.06	0.08
query5	0.54	0.52	0.52
query6	1.27	0.62	0.63
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.49	0.48
query10	0.54	0.54	0.54
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.62	0.62	0.61
query14	0.80	0.79	0.80
query15	0.79	0.77	0.76
query16	0.37	0.36	0.37
query17	1.01	1.03	1.01
query18	0.26	0.25	0.25
query19	1.91	1.89	1.85
query20	0.02	0.01	0.01
query21	15.49	0.55	0.58
query22	2.28	2.02	1.47
query23	16.95	0.95	0.86
query24	3.84	5.93	0.57
query25	0.40	0.11	0.04
query26	0.80	0.16	0.16
query27	0.04	0.05	0.04
query28	4.86	0.76	0.78
query29	12.68	2.35	2.35
query30	0.58	0.50	0.53
query31	2.81	0.39	0.38
query32	3.36	0.50	0.50
query33	3.04	3.04	3.09
query34	15.69	4.83	4.83
query35	4.90	4.91	4.90
query36	1.08	1.05	1.03
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.15	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.01
Total cold run time: 100.23 s
Total hot run time: 30.37 s

@doris-robot
Copy link

doris-robot commented Jun 20, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 0469dd6eab880e6f88376a7d0fc08ccb94ec843a with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 20, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 20, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 20, 2024

PR approved by anyone and no changes requested.

@morningman morningman merged commit 132b9d6 into apache:branch-2.0 Jun 20, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@morningman
[fix](variable) modify @@auto_commit column type to BIGINT (apache#36584
e187329 
)

bp apache#33887 apache#33282
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zy-kkk zy-kkk zy-kkk approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@morningman @doris-robot @zy-kkk