[fix](statistics) Skip statistics cache for system dbs by yujun777 · Pull Request #63089 · apache/doris

yujun777 · 2026-05-08T12:36:26Z

What problem does this PR solve?

Statistics cache may still try to load column statistics, partition column statistics, or histograms for system databases. This is not expected for information_schema, mysql, or the dynamically assigned __internal_schema database.

Solution

Add a system database check in StatisticConstants.
Reuse the check in StatisticsCache for column statistics, partition statistics, and histograms.
Resolve __internal_schema by looking up the internal catalog DB object so the dynamic db id is covered.

Check List

FE UT

Test

StatisticsCacheTest

Result: BUILD SUCCESS. StatisticsCacheTest ran 6 tests with 0 failures and 0 errors.

Avoid loading column, partition column, and histogram statistics for system databases, including the dynamically assigned __internal_schema database. Key changes: - Add a system database check in StatisticConstants. - Reuse the check in StatisticsCache for column statistics, partition statistics, and histograms. - Keep table-level system table checks for OlapTableStatistics. Unit Test: - Add StatisticsCacheTest coverage for information_schema and dynamic __internal_schema db ids.

hello-stephen · 2026-05-08T12:36:32Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

morrySnow · 2026-05-09T02:41:48Z

run buildall

hello-stephen · 2026-05-09T03:10:27Z

TPC-H: Total hot run time: 29393 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 111668f7cc2528b3f7a933cb107eadc4bb645b0b, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17792	3971	3927	3927
q2	q3	10733	868	600	600
q4	4656	461	343	343
q5	7456	1306	1112	1112
q6	190	163	136	136
q7	918	929	758	758
q8	9410	1354	1257	1257
q9	6111	5330	5294	5294
q10	6312	2056	1783	1783
q11	470	269	251	251
q12	698	424	288	288
q13	18221	3290	2759	2759
q14	297	285	261	261
q15	q16	896	865	782	782
q17	1032	1073	734	734
q18	6459	5670	5641	5641
q19	1387	1227	1060	1060
q20	504	394	260	260
q21	4573	2262	1850	1850
q22	409	352	297	297
Total cold run time: 98524 ms
Total hot run time: 29393 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4178	4057	4063	4057
q2	q3	4629	4755	4190	4190
q4	2082	2161	1375	1375
q5	4983	4985	5261	4985
q6	189	162	130	130
q7	2041	1753	2027	1753
q8	3367	3172	3128	3128
q9	8366	8419	8471	8419
q10	4460	4492	4278	4278
q11	620	433	407	407
q12	681	737	519	519
q13	3159	3638	2879	2879
q14	307	309	269	269
q15	q16	766	785	674	674
q17	1395	1314	1426	1314
q18	8086	7166	7024	7024
q19	1133	1155	1179	1155
q20	2220	2222	1939	1939
q21	6043	5332	4918	4918
q22	540	519	428	428
Total cold run time: 59245 ms
Total hot run time: 53841 ms

hello-stephen · 2026-05-09T03:21:28Z

TPC-DS: Total hot run time: 172037 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 111668f7cc2528b3f7a933cb107eadc4bb645b0b, data reload: false

query5	4289	685	512	512
query6	339	223	201	201
query7	4227	565	302	302
query8	324	228	212	212
query9	8821	3963	3999	3963
query10	494	341	297	297
query11	6045	2396	2158	2158
query12	211	137	130	130
query13	1289	583	431	431
query14	6932	5351	5095	5095
query14_1	4376	4370	4379	4370
query15	217	204	189	189
query16	1009	469	429	429
query17	1403	740	604	604
query18	2725	483	349	349
query19	329	204	166	166
query20	140	133	131	131
query21	229	149	114	114
query22	13588	14098	14392	14098
query23	17286	16540	16320	16320
query23_1	16323	16303	16299	16299
query24	7964	1802	1373	1373
query24_1	1384	1350	1401	1350
query25	566	481	456	456
query26	1298	301	170	170
query27	2670	598	338	338
query28	4336	1951	1967	1951
query29	1012	655	532	532
query30	306	237	194	194
query31	1118	1084	948	948
query32	84	72	73	72
query33	552	357	298	298
query34	1145	1127	640	640
query35	785	782	674	674
query36	1362	1368	1162	1162
query37	150	108	88	88
query38	3300	3219	3040	3040
query39	932	899	902	899
query39_1	882	916	873	873
query40	233	160	138	138
query41	67	66	64	64
query42	112	113	111	111
query43	318	323	281	281
query44	
query45	215	209	191	191
query46	1083	1239	713	713
query47	2330	2348	2204	2204
query48	391	405	304	304
query49	650	553	449	449
query50	702	288	222	222
query51	4357	4426	4154	4154
query52	105	107	96	96
query53	266	285	208	208
query54	322	303	266	266
query55	94	90	87	87
query56	327	315	319	315
query57	1465	1413	1338	1338
query58	306	317	267	267
query59	1618	1709	1447	1447
query60	353	334	326	326
query61	157	158	155	155
query62	708	638	556	556
query63	239	205	210	205
query64	2365	822	685	685
query65	
query66	1682	513	390	390
query67	30241	30013	29839	29839
query68	
query69	460	342	308	308
query70	992	974	974	974
query71	307	277	270	270
query72	2917	2748	2481	2481
query73	823	756	437	437
query74	5093	4928	4710	4710
query75	2772	2656	2335	2335
query76	2297	1160	747	747
query77	415	407	345	345
query78	12977	12972	12337	12337
query79	1525	1023	754	754
query80	1387	580	485	485
query81	522	279	259	259
query82	936	161	123	123
query83	352	273	246	246
query84	269	138	108	108
query85	919	538	442	442
query86	461	354	338	338
query87	3390	3361	3211	3211
query88	3535	2671	2667	2667
query89	439	386	335	335
query90	1949	185	174	174
query91	181	177	146	146
query92	75	72	68	68
query93	1078	961	558	558
query94	724	343	305	305
query95	651	463	369	369
query96	1065	781	358	358
query97	2694	2758	2557	2557
query98	240	226	248	226
query99	1124	1136	991	991
Total cold run time: 256442 ms
Total hot run time: 172037 ms

yujun777 · 2026-05-09T03:41:37Z

run buildall

hello-stephen · 2026-05-09T04:20:57Z

FE UT Coverage Report

Increment line coverage 57.14% (8/14) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-05-09T05:33:35Z

TPC-H: Total hot run time: 29524 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55f4e284ce15ef4c45adf826f54492a8214cc3ab, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17712	3885	3899	3885
q2	q3	10716	887	611	611
q4	4663	466	354	354
q5	7450	1312	1148	1148
q6	184	169	141	141
q7	908	956	754	754
q8	9310	1397	1247	1247
q9	5722	5384	5351	5351
q10	6296	2086	1831	1831
q11	466	271	255	255
q12	688	426	301	301
q13	18209	3354	2740	2740
q14	295	286	263	263
q15	q16	903	889	789	789
q17	1021	1102	697	697
q18	6502	5661	5612	5612
q19	1187	1140	1011	1011
q20	524	384	263	263
q21	4705	2387	1937	1937
q22	471	395	334	334
Total cold run time: 97932 ms
Total hot run time: 29524 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4605	4605	4516	4516
q2	q3	4694	4809	4215	4215
q4	2118	2203	1425	1425
q5	5030	5097	5267	5097
q6	207	172	144	144
q7	2054	1793	1675	1675
q8	3374	3105	3081	3081
q9	8418	8638	8470	8470
q10	4477	4464	4263	4263
q11	644	413	387	387
q12	707	767	528	528
q13	3263	3637	2965	2965
q14	316	324	271	271
q15	q16	784	796	695	695
q17	1340	1302	1247	1247
q18	8074	7160	7193	7160
q19	1159	1140	1170	1140
q20	2264	2234	1982	1982
q21	6154	5443	4888	4888
q22	518	470	396	396
Total cold run time: 60200 ms
Total hot run time: 54545 ms

hello-stephen · 2026-05-09T05:44:31Z

TPC-DS: Total hot run time: 171110 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55f4e284ce15ef4c45adf826f54492a8214cc3ab, data reload: false

query5	4352	663	529	529
query6	337	228	208	208
query7	4311	570	317	317
query8	337	244	231	231
query9	8863	4108	4102	4102
query10	475	353	310	310
query11	5912	2397	2224	2224
query12	183	135	140	135
query13	1284	621	425	425
query14	6774	5377	5061	5061
query14_1	4404	4396	4363	4363
query15	211	201	189	189
query16	1007	450	458	450
query17	1236	803	620	620
query18	2713	476	355	355
query19	222	202	161	161
query20	141	136	130	130
query21	212	140	118	118
query22	13668	13645	13410	13410
query23	17094	16418	15923	15923
query23_1	16254	16204	16225	16204
query24	7410	1754	1371	1371
query24_1	1360	1351	1353	1351
query25	562	493	449	449
query26	1316	324	168	168
query27	2656	614	349	349
query28	4389	1984	1968	1968
query29	996	634	520	520
query30	302	246	199	199
query31	1125	1076	953	953
query32	96	74	75	74
query33	533	397	286	286
query34	1148	1148	639	639
query35	768	791	670	670
query36	1367	1305	1111	1111
query37	148	101	93	93
query38	3208	3162	3028	3028
query39	931	928	892	892
query39_1	875	871	877	871
query40	235	163	135	135
query41	64	64	60	60
query42	114	114	112	112
query43	320	324	294	294
query44	
query45	240	200	196	196
query46	1059	1220	744	744
query47	2235	2367	2211	2211
query48	386	426	300	300
query49	643	530	426	426
query50	725	283	225	225
query51	4372	4281	4286	4281
query52	104	107	94	94
query53	263	282	208	208
query54	311	267	261	261
query55	96	88	86	86
query56	308	299	295	295
query57	1436	1432	1326	1326
query58	326	288	264	264
query59	1608	1641	1456	1456
query60	352	348	334	334
query61	154	149	151	149
query62	679	633	580	580
query63	246	209	215	209
query64	2246	831	672	672
query65	
query66	1665	514	400	400
query67	29989	30021	29871	29871
query68	
query69	450	350	345	345
query70	1059	985	990	985
query71	297	278	266	266
query72	2863	2784	2417	2417
query73	859	726	440	440
query74	5051	4898	4717	4717
query75	2782	2649	2329	2329
query76	2318	1139	814	814
query77	409	422	349	349
query78	12833	12781	12346	12346
query79	1461	975	745	745
query80	674	586	511	511
query81	449	279	247	247
query82	1329	158	123	123
query83	355	272	246	246
query84	265	144	112	112
query85	861	497	434	434
query86	411	366	339	339
query87	3387	3361	3226	3226
query88	3563	2703	2676	2676
query89	436	384	339	339
query90	1884	181	179	179
query91	174	167	141	141
query92	87	79	74	74
query93	956	969	571	571
query94	519	346	264	264
query95	672	397	359	359
query96	1022	734	345	345
query97	2699	2696	2562	2562
query98	241	236	240	236
query99	1125	1143	986	986
Total cold run time: 253637 ms
Total hot run time: 171110 ms

github-actions · 2026-05-09T07:02:37Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-05-09T07:02:40Z

PR approved by anyone and no changes requested.

…3089 (#63198) cherry-pick: #63089

…3089 (#63197) cherry-pick: #63089

fix fe ut

55f4e28

morrySnow approved these changes May 9, 2026

View reviewed changes

github-actions Bot added approved Indicates a PR has been approved by one committer. reviewed labels May 9, 2026

morrySnow merged commit 75176c0 into apache:master May 9, 2026
29 of 31 checks passed

morrySnow added dev/4.0.x dev/4.1.x labels May 9, 2026

github-actions Bot added dev/4.0.x-conflict dev/4.1.x-conflict labels May 9, 2026

morrySnow added dev/4.0.x dev/4.1.x and removed dev/4.0.x dev/4.0.x-conflict dev/4.1.x dev/4.1.x-conflict labels May 13, 2026

github-actions Bot added dev/4.1.x-conflict dev/4.0.x-conflict labels May 13, 2026

This was referenced May 13, 2026

branch-4.0: [fix](statistics) Skip statistics cache for system dbs #63089 #63197

Merged

branch-4.1: [fix](statistics) Skip statistics cache for system dbs #63089 #63198

Merged

yiguolei pushed a commit that referenced this pull request May 15, 2026

branch-4.1: [fix](statistics) Skip statistics cache for system dbs #6…

e752545

…3089 (#63198) cherry-pick: #63089

yiguolei added dev/4.1.1-merged and removed dev/4.1.x dev/4.1.x-conflict labels May 15, 2026

yiguolei mentioned this pull request May 20, 2026

4.1.1 Release Notes #63426

Open

yiguolei pushed a commit that referenced this pull request May 20, 2026

branch-4.0: [fix](statistics) Skip statistics cache for system dbs #6…

aaa0df1

…3089 (#63197) cherry-pick: #63089

yiguolei added dev/4.0.6-merged and removed dev/4.0.x dev/4.0.x-conflict labels May 20, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](statistics) Skip statistics cache for system dbs#63089

[fix](statistics) Skip statistics cache for system dbs#63089
morrySnow merged 2 commits into
apache:masterfrom
yujun777:fix-system-db-stats-guard

yujun777 commented May 8, 2026 •

edited

Loading

Uh oh!

hello-stephen commented May 8, 2026

Uh oh!

morrySnow commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

yujun777 commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

github-actions Bot commented May 9, 2026

Uh oh!

github-actions Bot commented May 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

yujun777 commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Solution

Check List

Test

Uh oh!

hello-stephen commented May 8, 2026

Uh oh!

morrySnow commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

yujun777 commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

github-actions Bot commented May 9, 2026

Uh oh!

github-actions Bot commented May 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yujun777 commented May 8, 2026 •

edited

Loading