[fix](mtmv)Fix the mapping issue where base tables could still reference a dropped MTMV #53383

zddr · 2025-07-16T09:18:03Z

What problem does this PR solve?

Background:

The MTMVRelationManager maintains a mapping relationship between base tables and materialized views (MTMVs).

When an MTMV is dropped, this relationship is maintained by removing the relevant mappings.

When an MTMV is successfully refreshed, the mapping is updated (old mappings are removed and new ones are added).

Suspected Issue Scenario:

The MTMV is refreshed successfully.

The MTMV is dropped.

The mapping relationship is refreshed (potentially causing inconsistent state).

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
Fix the mapping issue where base tables could still reference a dropped MTMV

Release note

Fix the mapping issue where base tables could still reference a dropped MTMV

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-07-16T09:18:09Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zddr · 2025-07-16T09:18:49Z

run buildall

zddr · 2025-07-18T11:12:47Z

run buildall

doris-robot · 2025-07-18T11:47:03Z

TPC-H: Total hot run time: 34264 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2a24ec2bee3c1bbb707b6dcd16356af9421f60c9, data reload: false

------ Round 1 ----------------------------------
q1	17590	5164	5150	5150
q2	1931	289	202	202
q3	10346	1262	689	689
q4	10317	1018	530	530
q5	8705	2348	2385	2348
q6	189	163	135	135
q7	918	801	616	616
q8	9335	1292	1123	1123
q9	6776	5075	5096	5075
q10	6887	2407	2000	2000
q11	516	294	286	286
q12	351	363	226	226
q13	17775	3629	3057	3057
q14	238	235	233	233
q15	600	514	526	514
q16	433	439	384	384
q17	610	861	360	360
q18	7541	7199	7187	7187
q19	1315	972	574	574
q20	349	355	236	236
q21	3711	2541	2331	2331
q22	1056	1008	1013	1008
Total cold run time: 107489 ms
Total hot run time: 34264 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5188	5131	5148	5131
q2	268	346	245	245
q3	2180	2652	2329	2329
q4	1370	1786	1331	1331
q5	4310	4353	4534	4353
q6	238	176	141	141
q7	1980	2024	1797	1797
q8	2620	2477	2545	2477
q9	7277	7082	7154	7082
q10	3106	3383	2840	2840
q11	557	555	519	519
q12	683	780	626	626
q13	3576	3925	3300	3300
q14	293	306	284	284
q15	558	506	497	497
q16	466	504	441	441
q17	1158	1520	1403	1403
q18	7795	7601	7545	7545
q19	777	787	860	787
q20	1996	2130	1961	1961
q21	4955	4658	4325	4325
q22	1084	1061	985	985
Total cold run time: 52435 ms
Total hot run time: 50399 ms

doris-robot · 2025-07-18T11:58:51Z

TPC-DS: Total hot run time: 190290 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2a24ec2bee3c1bbb707b6dcd16356af9421f60c9, data reload: false

query1	995	405	420	405
query2	6550	1714	1725	1714
query3	6806	226	218	218
query4	26286	23745	23406	23406
query5	4473	653	533	533
query6	328	248	242	242
query7	4636	511	314	314
query8	390	342	330	330
query9	8645	2659	2654	2654
query10	486	357	312	312
query11	15296	15147	15372	15147
query12	196	132	128	128
query13	1706	568	462	462
query14	10041	5980	6023	5980
query15	218	202	170	170
query16	7652	605	442	442
query17	1191	748	605	605
query18	2052	417	363	363
query19	204	195	166	166
query20	137	126	128	126
query21	236	143	127	127
query22	4263	4376	4272	4272
query23	34177	33220	33183	33183
query24	8342	2402	2409	2402
query25	557	467	421	421
query26	1250	285	156	156
query27	2714	516	361	361
query28	4350	2164	2125	2125
query29	740	577	466	466
query30	302	239	207	207
query31	959	870	797	797
query32	113	75	72	72
query33	583	406	364	364
query34	809	859	533	533
query35	838	852	751	751
query36	961	1021	915	915
query37	114	107	87	87
query38	4249	4155	4045	4045
query39	1541	1517	1489	1489
query40	237	135	127	127
query41	111	104	101	101
query42	127	112	113	112
query43	524	543	481	481
query44	1315	834	835	834
query45	196	185	179	179
query46	845	1018	660	660
query47	1807	1889	1779	1779
query48	401	438	347	347
query49	816	569	468	468
query50	665	710	436	436
query51	5644	5635	5591	5591
query52	125	122	108	108
query53	250	281	203	203
query54	597	599	530	530
query55	88	89	93	89
query56	328	352	336	336
query57	1213	1235	1192	1192
query58	318	322	295	295
query59	2592	2707	2518	2518
query60	347	355	338	338
query61	155	143	155	143
query62	815	765	692	692
query63	250	216	212	212
query64	4343	1082	726	726
query65	4288	4176	4179	4176
query66	1241	587	498	498
query67	15897	15897	15496	15496
query68	8493	906	542	542
query69	487	335	364	335
query70	1232	1163	1171	1163
query71	490	334	340	334
query72	5285	4624	4603	4603
query73	709	572	368	368
query74	9372	8921	8721	8721
query75	3921	3320	2751	2751
query76	3640	1177	743	743
query77	844	434	463	434
query78	9963	10280	9445	9445
query79	1533	868	597	597
query80	706	573	597	573
query81	481	281	244	244
query82	437	142	107	107
query83	313	290	275	275
query84	304	121	96	96
query85	828	396	358	358
query86	357	340	316	316
query87	4527	4605	4275	4275
query88	3014	2415	2454	2415
query89	436	362	312	312
query90	1952	227	234	227
query91	150	161	127	127
query92	89	78	68	68
query93	1123	947	606	606
query94	694	444	325	325
query95	389	316	310	310
query96	504	583	295	295
query97	2739	2784	2636	2636
query98	246	216	210	210
query99	1497	1426	1322	1322
Total cold run time: 277676 ms
Total hot run time: 190290 ms

doris-robot · 2025-07-18T12:04:23Z

ClickBench: Total hot run time: 33.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2a24ec2bee3c1bbb707b6dcd16356af9421f60c9, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.25	0.07	0.08
query4	1.62	0.11	0.11
query5	0.42	0.42	0.40
query6	1.16	0.67	0.67
query7	0.02	0.02	0.02
query8	0.06	0.04	0.04
query9	0.59	0.53	0.50
query10	0.59	0.57	0.57
query11	0.16	0.12	0.12
query12	0.16	0.12	0.12
query13	0.65	0.61	0.61
query14	0.80	0.81	0.81
query15	0.92	0.89	0.88
query16	0.39	0.38	0.40
query17	1.07	1.07	1.08
query18	0.23	0.22	0.21
query19	1.96	1.87	1.89
query20	0.01	0.02	0.02
query21	15.37	0.96	0.59
query22	0.79	1.22	0.81
query23	14.77	1.36	0.66
query24	7.41	1.16	0.82
query25	0.48	0.34	0.09
query26	0.73	0.17	0.16
query27	0.06	0.06	0.05
query28	8.98	0.89	0.44
query29	12.55	4.05	3.36
query30	3.15	3.08	3.03
query31	2.82	0.58	0.40
query32	3.23	0.55	0.49
query33	3.07	3.10	3.16
query34	16.16	5.41	4.79
query35	4.79	4.84	4.86
query36	0.67	0.52	0.50
query37	0.10	0.08	0.07
query38	0.07	0.06	0.05
query39	0.04	0.04	0.03
query40	0.18	0.15	0.16
query41	0.09	0.04	0.03
query42	0.05	0.03	0.04
query43	0.05	0.05	0.04
Total cold run time: 106.79 s
Total hot run time: 33.08 s

hello-stephen · 2025-07-18T12:29:54Z

FE UT Coverage Report

Increment line coverage 100.00% (14/14) 🎉
Increment coverage report
Complete coverage report

zddr · 2025-07-29T07:18:45Z

run buildall

doris-robot · 2025-07-29T07:56:03Z

TPC-H: Total hot run time: 33893 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5d85c73f7e8d847c21cf551d049dcb6667035c02, data reload: false

------ Round 1 ----------------------------------
q1	17603	5774	5556	5556
q2	1929	281	204	204
q3	10300	1278	678	678
q4	10227	963	521	521
q5	8330	2330	2257	2257
q6	182	162	128	128
q7	875	750	601	601
q8	9288	1266	1045	1045
q9	6862	5081	5067	5067
q10	6944	2342	1964	1964
q11	473	279	273	273
q12	343	369	228	228
q13	17779	3490	2972	2972
q14	237	242	219	219
q15	538	471	462	462
q16	421	441	381	381
q17	571	831	359	359
q18	7580	7109	6950	6950
q19	2337	996	543	543
q20	319	310	216	216
q21	3353	2470	2217	2217
q22	1073	1070	1052	1052
Total cold run time: 107564 ms
Total hot run time: 33893 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5892	5743	5710	5710
q2	229	309	214	214
q3	2055	2571	2184	2184
q4	1309	1719	1321	1321
q5	4329	4497	4305	4305
q6	215	172	139	139
q7	2014	1991	1801	1801
q8	2389	2472	2486	2472
q9	7433	7073	7589	7073
q10	3208	3340	2900	2900
q11	598	506	486	486
q12	702	780	647	647
q13	3466	3693	3344	3344
q14	433	303	274	274
q15	512	447	430	430
q16	418	485	445	445
q17	1136	1449	1324	1324
q18	7980	7871	8021	7871
q19	6497	911	1041	911
q20	2007	2001	1791	1791
q21	14453	4203	4288	4203
q22	1029	1066	988	988
Total cold run time: 68304 ms
Total hot run time: 50833 ms

doris-robot · 2025-07-29T08:08:02Z

TPC-DS: Total hot run time: 173510 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5d85c73f7e8d847c21cf551d049dcb6667035c02, data reload: false

============================================
query1	1018	388	433	388
query2	6545	1871	1604	1604
query3	6738	233	223	223
query4	27046	23715	22838	22838
query5	4357	642	530	530
query6	323	265	225	225
query7	4626	520	308	308
query8	262	231	211	211
query9	8567	3169	3126	3126
query10	474	346	296	296
query11	15844	15039	14839	14839
query12	191	140	131	131
query13	1675	591	428	428
query14	9656	7756	7831	7756
query15	218	206	168	168
query16	7822	638	454	454
query17	1643	823	653	653
query18	2117	449	343	343
query19	287	214	207	207
query20	155	151	143	143
query21	218	130	116	116
query22	4049	4041	3810	3810
query23	34382	33997	34023	33997
query24	7925	2443	2484	2443
query25	641	503	443	443
query26	722	291	162	162
query27	2588	494	355	355
query28	4155	2315	2280	2280
query29	709	607	495	495
query30	297	237	207	207
query31	898	809	701	701
query32	87	75	74	74
query33	572	473	382	382
query34	824	839	529	529
query35	823	838	742	742
query36	1014	1063	968	968
query37	140	111	98	98
query38	4010	4059	4060	4059
query39	1463	1406	1523	1406
query40	238	143	135	135
query41	61	55	58	55
query42	140	125	129	125
query43	516	494	463	463
query44	1397	877	865	865
query45	210	187	189	187
query46	944	1066	679	679
query47	1788	1799	1730	1730
query48	412	438	312	312
query49	670	506	439	439
query50	679	698	434	434
query51	5571	5600	5532	5532
query52	137	137	122	122
query53	264	285	217	217
query54	671	670	564	564
query55	98	91	96	91
query56	365	366	359	359
query57	1204	1200	1154	1154
query58	341	333	357	333
query59	2628	2576	2496	2496
query60	405	398	419	398
query61	129	123	125	123
query62	814	732	670	670
query63	255	214	213	213
query64	3052	1125	774	774
query65	4260	4099	4088	4088
query66	905	449	389	389
query67	query68	18054	921	597	597
query69	1006	320	321	320
query70	1395	1102	1138	1102
query71	732	361	357	357
query72	9162	2302	2385	2302
query73	3392	661	374	374
query74	8984	9034	8469	8469
query75	7566	3128	2682	2682
query76	8842	1264	813	813
query77	1159	426	365	365
query78	9660	10202	9546	9546
query79	15318	611	590	590
query80	1914	613	504	504
query81	560	280	242	242
query82	473	153	119	119
query83	354	301	277	277
query84	292	101	90	90
query85	1241	366	335	335
query86	373	322	330	322
query87	4328	4301	4249	4249
query88	5592	2331	2319	2319
query89	512	412	320	320
query90	2544	241	238	238
query91	148	141	111	111
query92	89	73	69	69
query93	6483	982	663	663
query94	1134	397	281	281
query95	441	337	333	333
query96	523	611	293	293
query97	2740	2756	2640	2640
query98	288	243	233	233
query99	1529	1402	1270	1270
Total cold run time: 310063 ms
Total hot run time: 173510 ms

doris-robot · 2025-07-29T08:13:32Z

ClickBench: Total hot run time: 33.41 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5d85c73f7e8d847c21cf551d049dcb6667035c02, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.05	0.04
query3	0.24	0.08	0.07
query4	1.63	0.11	0.11
query5	0.40	0.42	0.45
query6	1.15	0.69	0.69
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.56	0.48	0.47
query10	0.53	0.54	0.54
query11	0.16	0.11	0.11
query12	0.16	0.11	0.11
query13	0.65	0.67	0.66
query14	0.93	1.17	1.01
query15	0.92	0.89	0.91
query16	0.39	0.39	0.38
query17	1.07	1.05	1.10
query18	0.21	0.21	0.20
query19	2.03	1.81	1.87
query20	0.01	0.02	0.01
query21	15.39	0.85	0.54
query22	0.77	1.24	0.69
query23	14.90	1.16	0.62
query24	6.41	0.91	1.43
query25	0.52	0.18	0.27
query26	0.55	0.16	0.14
query27	0.06	0.05	0.05
query28	10.53	0.85	0.46
query29	12.72	3.85	3.36
query30	3.01	2.95	2.91
query31	2.81	0.57	0.40
query32	3.25	0.57	0.50
query33	3.04	3.24	3.28
query34	15.81	5.27	4.90
query35	4.85	5.01	5.01
query36	0.70	0.51	0.50
query37	0.10	0.07	0.08
query38	0.06	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.07 s
Total hot run time: 33.41 s

github-actions · 2025-07-30T02:37:52Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-07-30T02:37:55Z

PR approved by anyone and no changes requested.

…nce a dropped MTMV (#53383) ### What problem does this PR solve? Background: The MTMVRelationManager maintains a mapping relationship between base tables and materialized views (MTMVs). When an MTMV is dropped, this relationship is maintained by removing the relevant mappings. When an MTMV is successfully refreshed, the mapping is updated (old mappings are removed and new ones are added). Suspected Issue Scenario: The MTMV is refreshed successfully. The MTMV is dropped. The mapping relationship is refreshed (potentially causing inconsistent state).

…still reference a dropped MTMV #53383 (#54082) Cherry-picked from #53383 Co-authored-by: zhangdong <zhangdong@selectdb.com>

…still reference a dropped MTMV #53383 (#54080) Cherry-picked from #53383 Co-authored-by: zhangdong <zhangdong@selectdb.com>

…nce a dropped MTMV (apache#53383) ### What problem does this PR solve? Background: The MTMVRelationManager maintains a mapping relationship between base tables and materialized views (MTMVs). When an MTMV is dropped, this relationship is maintained by removing the relevant mappings. When an MTMV is successfully refreshed, the mapping is updated (old mappings are removed and new ones are added). Suspected Issue Scenario: The MTMV is refreshed successfully. The MTMV is dropped. The mapping relationship is refreshed (potentially causing inconsistent state).

…till refere… (apache#5154) …nce a dropped MTMV (apache#53383) pick: apache#53383

1

2a24ec2

morrySnow added usercase Important user case type label dev/2.1.x dev/3.0.x dev/3.1.x labels Jul 18, 2025

add ut

5d85c73

morrySnow approved these changes Jul 30, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 30, 2025

github-actions bot added the reviewed label Jul 30, 2025

seawinde approved these changes Jul 30, 2025

View reviewed changes

morrySnow merged commit d288cc0 into apache:master Jul 30, 2025
30 checks passed

github-actions bot mentioned this pull request Jul 30, 2025

branch-3.0: [fix](mtmv)Fix the mapping issue where base tables could still reference a dropped MTMV #53383 #54080

Merged

github-actions bot mentioned this pull request Jul 30, 2025

branch-2.1: [fix](mtmv)Fix the mapping issue where base tables could still reference a dropped MTMV #53383 #54081

Open

github-actions bot mentioned this pull request Jul 30, 2025

branch-3.1: [fix](mtmv)Fix the mapping issue where base tables could still reference a dropped MTMV #53383 #54082

Merged

morrySnow pushed a commit that referenced this pull request Jul 30, 2025

branch-3.1: [fix](mtmv)Fix the mapping issue where base tables could …

6326937

…still reference a dropped MTMV #53383 (#54082) Cherry-picked from #53383 Co-authored-by: zhangdong <zhangdong@selectdb.com>

morrySnow added dev/3.1.0-merged and removed dev/3.1.x labels Jul 30, 2025

dataroaring pushed a commit that referenced this pull request Aug 12, 2025

branch-3.0: [fix](mtmv)Fix the mapping issue where base tables could …

b389dcc

…still reference a dropped MTMV #53383 (#54080) Cherry-picked from #53383 Co-authored-by: zhangdong <zhangdong@selectdb.com>

dataroaring added dev/3.0.8-merged and removed dev/3.0.x labels Aug 12, 2025

gavinchou mentioned this pull request Sep 1, 2025

Release Note 3.0.8 #55553

Open

yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Dec 19, 2025

branch-2.1:[fix](mtmv)Fix the mapping issue where base tables could s…

c5c5c9b

…till refere… (apache#5154) …nce a dropped MTMV (apache#53383) pick: apache#53383

[fix](mtmv)Fix the mapping issue where base tables could still reference a dropped MTMV #53383

[fix](mtmv)Fix the mapping issue where base tables could still reference a dropped MTMV #53383

Uh oh!

Conversation

zddr commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jul 16, 2025

Uh oh!

zddr commented Jul 16, 2025

Uh oh!

zddr commented Jul 18, 2025

Uh oh!

doris-robot commented Jul 18, 2025

Uh oh!

doris-robot commented Jul 18, 2025

Uh oh!

doris-robot commented Jul 18, 2025

Uh oh!

hello-stephen commented Jul 18, 2025

FE UT Coverage Report

Uh oh!

zddr commented Jul 29, 2025

Uh oh!

doris-robot commented Jul 29, 2025

Uh oh!

doris-robot commented Jul 29, 2025

Uh oh!

doris-robot commented Jul 29, 2025

Uh oh!

github-actions bot commented Jul 30, 2025

Uh oh!

github-actions bot commented Jul 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

zddr commented Jul 16, 2025 •

edited

Loading