Skip to content

[fix](restore) update is_being_synced properties #40194#40557

Merged
w41ter merged 1 commit intoapache:branch-2.0from
w41ter:pick_40194_to_origin_branch-2.0
Sep 10, 2024
Merged

[fix](restore) update is_being_synced properties #40194#40557
w41ter merged 1 commit intoapache:branch-2.0from
w41ter:pick_40194_to_origin_branch-2.0

Conversation

@w41ter
Copy link
Contributor

@w41ter w41ter commented Sep 9, 2024

cherry pick from #40194

Some properties, like storage_policy, are not supported by the ccr
syncer. This PR maintains those properties for both local and remote
tables while restoring.
@w41ter
Copy link
Contributor Author

w41ter commented Sep 9, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@doris-robot
Copy link

TPC-H: Total hot run time: 49122 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3ffe18f6b53cbde2b13be9a0e41705121a8aeb74, data reload: false

------ Round 1 ----------------------------------
q1	22790	4619	4523	4523
q2	2415	166	156	156
q3	10245	1895	1954	1895
q4	10243	1266	1322	1266
q5	8565	3940	3936	3936
q6	230	124	125	124
q7	2083	1645	1581	1581
q8	9311	2733	2694	2694
q9	10310	9708	9864	9708
q10	8655	3547	3520	3520
q11	432	261	239	239
q12	472	301	299	299
q13	18351	3931	4030	3931
q14	354	338	335	335
q15	510	453	463	453
q16	537	449	456	449
q17	1139	937	932	932
q18	7213	6875	6906	6875
q19	1698	1579	1482	1482
q20	525	321	302	302
q21	4385	4116	4037	4037
q22	487	392	385	385
Total cold run time: 120950 ms
Total hot run time: 49122 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4300	4311	4278	4278
q2	321	232	220	220
q3	4189	4163	4110	4110
q4	2743	2744	2768	2744
q5	7165	7122	7091	7091
q6	240	119	120	119
q7	3229	2789	2803	2789
q8	4343	4413	4413	4413
q9	13701	13692	13546	13546
q10	4236	4221	4255	4221
q11	750	708	685	685
q12	1024	833	858	833
q13	6998	3740	3699	3699
q14	449	426	426	426
q15	506	461	448	448
q16	642	611	617	611
q17	3748	3765	3774	3765
q18	8854	8837	8831	8831
q19	1698	1612	1647	1612
q20	2396	2126	2138	2126
q21	8433	8432	8368	8368
q22	1001	949	960	949
Total cold run time: 80966 ms
Total hot run time: 75884 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 213288 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3ffe18f6b53cbde2b13be9a0e41705121a8aeb74, data reload: false

query1	923	424	385	385
query2	6530	2172	1893	1893
query3	6921	214	208	208
query4	23364	21670	21371	21371
query5	19739	6493	6550	6493
query6	295	220	220	220
query7	4328	300	318	300
query8	255	247	255	247
query9	3093	2646	2565	2565
query10	459	294	303	294
query11	15715	15193	15566	15193
query12	132	77	72	72
query13	1022	441	429	429
query14	17178	13579	13755	13579
query15	371	219	219	219
query16	6485	291	262	262
query17	1871	931	887	887
query18	899	311	319	311
query19	210	149	155	149
query20	82	78	75	75
query21	195	99	98	98
query22	5052	5071	4977	4977
query23	34247	33702	33439	33439
query24	7863	6289	6297	6289
query25	535	434	423	423
query26	1275	159	165	159
query27	2406	302	297	297
query28	6067	2229	2210	2210
query29	2774	2838	2783	2783
query30	244	175	172	172
query31	943	729	738	729
query32	70	60	58	58
query33	456	250	270	250
query34	869	487	477	477
query35	1159	900	912	900
query36	1374	1164	1081	1081
query37	175	62	62	62
query38	3034	2928	2895	2895
query39	1374	1315	1324	1315
query40	313	98	96	96
query41	40	36	36	36
query42	81	84	83	83
query43	610	680	585	585
query44	1169	717	720	717
query45	248	234	230	230
query46	1223	952	970	952
query47	2010	1782	1759	1759
query48	517	406	399	399
query49	634	369	364	364
query50	878	613	609	609
query51	4734	4659	4643	4643
query52	94	73	83	73
query53	236	192	180	180
query54	2664	2478	2500	2478
query55	94	83	95	83
query56	229	200	205	200
query57	1184	1133	1130	1130
query58	219	219	227	219
query59	3452	3576	3360	3360
query60	221	204	212	204
query61	95	95	97	95
query62	900	486	527	486
query63	204	186	179	179
query64	3540	1597	1507	1507
query65	3639	3583	3580	3580
query66	763	440	433	433
query67	17457	16651	15370	15370
query68	7159	661	648	648
query69	488	274	280	274
query70	1609	1735	1456	1456
query71	400	309	316	309
query72	6691	4787	4822	4787
query73	754	326	328	326
query74	6337	5877	5880	5877
query75	4603	3692	3692	3692
query76	4162	1149	1210	1149
query77	550	275	264	264
query78	12456	11801	12081	11801
query79	8343	641	670	641
query80	2389	398	403	398
query81	530	241	244	241
query82	1378	102	102	102
query83	172	136	133	133
query84	256	71	68	68
query85	1392	328	311	311
query86	330	287	326	287
query87	3244	3044	3059	3044
query88	4840	2395	2396	2395
query89	448	302	304	302
query90	1693	222	213	213
query91	166	133	134	133
query92	62	51	53	51
query93	5426	555	588	555
query94	685	210	209	209
query95	1928	2012	2048	2012
query96	649	334	331	331
query97	6434	6323	6377	6323
query98	216	204	205	204
query99	2942	989	936	936
Total cold run time: 316164 ms
Total hot run time: 213288 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3ffe18f6b53cbde2b13be9a0e41705121a8aeb74, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.25	0.05	0.05
query4	1.78	0.07	0.07
query5	0.54	0.53	0.53
query6	1.30	0.63	0.60
query7	0.01	0.01	0.02
query8	0.04	0.03	0.02
query9	0.52	0.49	0.49
query10	0.55	0.53	0.54
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.62	0.61	0.62
query14	0.80	0.78	0.81
query15	0.79	0.75	0.78
query16	0.37	0.37	0.39
query17	1.03	1.00	1.00
query18	0.22	0.28	0.24
query19	1.92	1.84	1.73
query20	0.02	0.01	0.01
query21	15.48	0.58	0.56
query22	1.97	2.26	2.00
query23	16.94	1.01	0.93
query24	4.23	1.36	5.71
query25	0.36	0.12	0.05
query26	0.98	0.16	0.15
query27	0.05	0.05	0.04
query28	4.06	0.76	0.75
query29	12.61	2.19	2.35
query30	0.55	0.54	0.52
query31	2.82	0.38	0.38
query32	3.38	0.51	0.49
query33	3.05	3.08	3.08
query34	15.25	4.78	4.78
query35	4.86	4.84	4.83
query36	1.05	1.03	1.00
query37	0.06	0.05	0.05
query38	0.03	0.03	0.03
query39	0.02	0.01	0.02
query40	0.15	0.13	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 99.09 s
Total hot run time: 31.38 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3ffe18f6b53cbde2b13be9a0e41705121a8aeb74 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@w41ter w41ter merged commit 70b39a2 into apache:branch-2.0 Sep 10, 2024
@w41ter w41ter deleted the pick_40194_to_origin_branch-2.0 branch September 10, 2024 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants