Skip to content

[fix](mv) Isolate temporary MV pre-rewrite state - #67780

Draft
morrySnow wants to merge 1 commit into
apache:masterfrom
morrySnow:fix/isolate-pre-rewrite-context
Draft

[fix](mv) Isolate temporary MV pre-rewrite state#67780
morrySnow wants to merge 1 commit into
apache:masterfrom
morrySnow:fix/isolate-pre-rewrite-context

Conversation

@morrySnow

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

MV pre-rewrite normalizes CTE children before the main rewrite. It reused the main statement context, allowing temporary CTE producer and consumer caches to affect later privilege checking and leaving partial CTE maps that could fail with an internal error.

This change gives temporary pre-rewrite its own mutable statement state while keeping ID and generated-alias allocation in the original statement scope. It temporarily installs the isolated context for utilities that access the connection directly and restores the original context in a finally block.

The regression test creates a view for which a user can select one column but not another, then exercises direct access and CTE queries under both default and forced pre-rewrite strategies. All paths must return the same permission error.

Release note

None

Check List (For Author)

  • Test:
    • Regression test
    • Unit Test
    • Manual test
  • Behavior changed:
    • Yes. MV pre-rewrite can no longer bypass or replace view-column privilege errors.
  • Does this need documentation?
    • No.

MV pre-rewrite normalized CTE children with the main statement context, allowing temporary producer and consumer caches to leak into the formal rewrite. This could either bypass view-column privilege checks or leave partial CTE state that failed with an internal error.\n\nCreate an independent temporary statement context for pre-rewrite state while sharing statement-scoped ID and alias allocation. Install it for utilities that read the connection context, restore the original context in a finally block, and cover state isolation plus view-column authorization in unit and regression tests.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow

Copy link
Copy Markdown
Contributor Author

run buildall

@morrySnow morrySnow changed the title [fix](fe) Isolate temporary MV pre-rewrite state [fix](mt) Isolate temporary MV pre-rewrite state Sep 10, 2026
@morrySnow morrySnow changed the title [fix](mt) Isolate temporary MV pre-rewrite state [fix](mv) Isolate temporary MV pre-rewrite state Sep 10, 2026
@morrySnow
morrySnow marked this pull request as draft September 10, 2026 07:08
@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16763 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 214d90ec1da9980492a3e2f217a3d457a73904f6, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17556	3101	3092	3092
q2	2071	265	234	234
q3	10242	930	499	499
q4	4670	249	198	198
q5	7677	579	396	396
q6	132	113	93	93
q7	522	488	379	379
q8	9240	890	906	890
q9	3438	2369	2371	2369
q10	6499	877	716	716
q11	390	205	182	182
q12	621	260	198	198
q13	18094	1497	1147	1147
q14	158	145	139	139
q15	q16	432	392	370	370
q17	1342	866	775	775
q18	3011	2259	2231	2231
q19	1098	820	769	769
q20	359	283	200	200
q21	4925	1659	1911	1659
q22	330	267	227	227
Total cold run time: 92807 ms
Total hot run time: 16763 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3474	3425	3390	3390
q2	495	395	371	371
q3	2213	2284	2282	2282
q4	1174	1146	881	881
q5	2140	2119	2084	2084
q6	163	117	85	85
q7	1013	924	856	856
q8	1590	1410	1405	1405
q9	3107	3097	3057	3057
q10	1859	1788	1637	1637
q11	348	267	254	254
q12	449	431	339	339
q13	1489	1500	1138	1138
q14	177	178	161	161
q15	q16	393	392	374	374
q17	3566	3320	3264	3264
q18	4899	4415	4718	4415
q19	890	795	804	795
q20	1096	1004	860	860
q21	3869	3078	3291	3078
q22	393	338	323	323
Total cold run time: 34797 ms
Total hot run time: 31049 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81673 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 214d90ec1da9980492a3e2f217a3d457a73904f6, data reload: false

query5	4271	444	367	367
query6	396	150	138	138
query7	4859	443	239	239
query8	300	131	126	126
query9	8699	2864	2840	2840
query10	402	241	190	190
query11	5391	1054	932	932
query12	127	71	67	67
query13	1185	462	325	325
query14	6173	2168	2078	2078
query14_1	1945	1942	1949	1942
query15	174	117	106	106
query16	908	360	340	340
query17	763	438	339	339
query18	2321	322	229	229
query19	158	133	106	106
query20	71	68	66	66
query21	199	98	84	84
query22	5476	5340	5261	5261
query23	6485	6075	6135	6075
query23_1	6004	6100	5913	5913
query24	7262	1070	795	795
query24_1	764	759	782	759
query25	424	293	255	255
query26	1249	233	136	136
query27	2771	448	251	251
query28	4669	1537	1489	1489
query29	944	442	371	371
query30	255	156	129	129
query31	830	394	342	342
query32	133	79	78	78
query33	482	240	185	185
query34	992	807	486	486
query35	398	402	357	357
query36	592	565	532	532
query37	130	82	73	73
query38	998	836	815	815
query39	481	463	504	463
query39_1	459	504	477	477
query40	202	94	80	80
query41	60	58	58	58
query42	78	74	73	73
query43	242	244	212	212
query44	980	534	550	534
query45	116	108	105	105
query46	804	823	525	525
query47	746	750	712	712
query48	318	321	231	231
query49	545	248	204	204
query50	744	261	200	200
query51	8097	8158	8306	8158
query52	74	69	62	62
query53	191	197	151	151
query54	207	178	169	169
query55	76	61	58	58
query56	249	164	166	164
query57	678	673	658	658
query58	202	165	156	156
query59	1296	1230	1090	1090
query60	249	182	164	164
query61	110	119	117	117
query62	339	208	174	174
query63	172	138	137	137
query64	2762	710	562	562
query65	1637	1577	1594	1577
query66	1888	282	204	204
query67	9816	9494	9566	9494
query68	2746	1195	651	651
query69	331	208	186	186
query70	686	605	607	605
query71	252	178	164	164
query72	2294	1688	1512	1512
query73	624	536	324	324
query74	1569	1231	1143	1143
query75	1165	1103	963	963
query76	2280	702	519	519
query77	256	257	218	218
query78	4037	3680	3222	3222
query79	2885	770	588	588
query80	1597	318	278	278
query81	509	157	131	131
query82	732	128	99	99
query83	290	214	199	199
query84	311	110	86	86
query85	798	336	276	276
query86	397	174	165	165
query87	1006	993	918	918
query88	2805	2079	2119	2079
query89	280	195	173	173
query90	1939	134	131	131
query91	132	133	99	99
query92	90	69	62	62
query93	1414	1183	656	656
query94	652	231	207	207
query95	514	271	295	271
query96	812	571	273	273
query97	1040	1047	1025	1025
query98	162	140	129	129
query99	427	343	307	307
Total cold run time: 177522 ms
Total hot run time: 81673 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.6 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 214d90ec1da9980492a3e2f217a3d457a73904f6, data reload: false

query1	0.00	0.01	0.01
query2	0.07	0.04	0.03
query3	0.25	0.11	0.10
query4	1.60	0.10	0.09
query5	0.17	0.15	0.16
query6	1.28	0.68	0.68
query7	0.03	0.00	0.01
query8	0.05	0.03	0.03
query9	0.28	0.21	0.22
query10	0.35	0.36	0.35
query11	0.18	0.12	0.12
query12	0.15	0.13	0.12
query13	0.29	0.33	0.30
query14	0.45	0.45	0.44
query15	0.35	0.35	0.35
query16	0.22	0.23	0.23
query17	0.69	0.66	0.70
query18	0.18	0.16	0.17
query19	1.23	1.16	1.13
query20	0.02	0.01	0.01
query21	15.44	0.16	0.11
query22	5.06	0.04	0.05
query23	16.18	0.25	0.10
query24	3.01	0.32	0.27
query25	0.11	0.03	0.03
query26	0.81	0.17	0.12
query27	0.03	0.04	0.03
query28	3.65	0.53	0.28
query29	12.41	3.21	2.57
query30	0.26	0.12	0.12
query31	2.75	0.37	0.18
query32	3.53	0.32	0.23
query33	1.34	1.54	1.40
query34	15.37	2.17	1.76
query35	1.76	1.73	1.70
query36	0.49	0.29	0.28
query37	0.07	0.04	0.03
query38	0.03	0.02	0.02
query39	0.03	0.02	0.02
query40	0.12	0.07	0.08
query41	0.07	0.02	0.03
query42	0.02	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 90.42 s
Total hot run time: 14.6 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants