Skip to content

[fix](insert) reset skipAuth on all INSERT OVERWRITE exit paths - #66383

Merged
CalvinKirs merged 1 commit into
apache:masterfrom
CalvinKirs:fix-insert-overwrite-skipauth
Aug 4, 2026
Merged

[fix](insert) reset skipAuth on all INSERT OVERWRITE exit paths#66383
CalvinKirs merged 1 commit into
apache:masterfrom
CalvinKirs:fix-insert-overwrite-skipauth

Conversation

@CalvinKirs

@CalvinKirs CalvinKirs commented Aug 3, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

InsertOverwriteTableCommand runs its internal partition-replacement work
with ConnectContext.skipAuth=true for OLAP targets, and clears the flag
in a finally block. The flag was set before that try block, so if any
statement in between returned early or threw — for example the
@branch-on-non-iceberg guard, or recordRunningTableOrException when the
same table already has an overwrite running — the method exited without the
finally ever running, and skipAuth stayed set for the rest of that
connection.

What changed

Move setSkipAuth(true) to the first statement inside the existing try
block (keeping the same OLAP-only condition), so it is always paired with
the setSkipAuth(false) in finally. The guards that can throw before that
point now run while skipAuth is still untouched, so there is nothing to
leak. The finally is unchanged.

Test

Added InsertOverwriteSkipAuthResetTest: runs a failing
INSERT OVERWRITE ... @branch against an OLAP table and asserts
skipAuth is reset afterwards. It fails on the old code
(expected: <false> but was: <true>) and passes with this change.

Checklist

  • Regression test added
  • mvn checkstyle:check -pl fe-core passes

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs
CalvinKirs force-pushed the fix-insert-overwrite-skipauth branch 2 times, most recently from 57858ce to 373aacf Compare August 3, 2026 10:04
@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@CalvinKirs

Copy link
Copy Markdown
Member Author

/review

// Set the flag here, inside the try, so the finally below always pairs the reset even if
// an earlier step (e.g. the @branch guard) throws before we get here.
if (physicalTableSink instanceof PhysicalOlapTableSink && targetTable instanceof OlapTable) {
ConnectContext.get().setSkipAuth(true);

@924060929 924060929 Aug 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use local variable ctx instead of

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review opinion: approve. I found no additional blocking or correctness issue in this change.

Checkpoint conclusions:

  • Goal and proof: the patch closes the connection-level skipAuth leak by moving the OLAP-only enable into the existing try, after the branch and running-table guards. The new JUnit 5 regression is a valid base-fails/head-passes oracle for the @branch failure path: the base sets the flag before that exception, while the patched path reaches the same exception without touching it.
  • Scope and clarity: the change is focused to the flag boundary plus one regression test. The existing inline thread already asks to use the local ctx; I did not duplicate it because production caller tracing did not establish a separate context-identity bug.
  • Authorization and lifecycle: source analysis and the explicit target LOAD check remain before the bypass. Task registration, inner insert planning, temp-partition add/replace, cancellation returns, failure cleanup, and rollback after enablement remain inside the paired try/finally; the reset is the first finally action. Known MySQL/proxy and MTMV callers install the same context thread-locally, and no production pre-existing-true caller into this command was found.
  • Concurrency and data writes: no new concurrency, locking, or transaction protocol is introduced. The existing isRunning/task-manager synchronization and partition write/replace paths are unchanged; normal, auto-detect, cancellation, MTMV, remote, and connector paths were traced, with the bypass condition still limited to local OLAP targets.
  • Compatibility, persistence, configuration, and observability: no protocol/storage-format compatibility, EditLog/persistence, configuration, FE-BE variable propagation, or new observability requirement applies. The change adds no meaningful hot-path cost.
  • Tests and validation: the regression fixture, parser reachability, thread-local context setup, and old/new control flow were reviewed statically. Per the review-only workflow I did not run builds or tests locally. The external CheckStyle job is green; FE UT and compile were still pending at final verification.
  • Review focus and completion: no additional user-provided focus was supplied, so the full PR was reviewed. Two normal reviewers and a separate lifecycle-risk reviewer all returned NO_NEW_VALUABLE_FINDINGS in round 1 against the same frozen head, and the final changed-file/unresolved-candidate sweep was clean.

@CalvinKirs
CalvinKirs force-pushed the fix-insert-overwrite-skipauth branch from 373aacf to 0cab8bf Compare August 3, 2026 12:04
@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

INSERT OVERWRITE marks the connection with skipAuth=true so its internal
partition-replacement work can run, and clears it in a finally block. The
flag was set before that try block, so any early return or exception in
between (for example the @branch-on-non-iceberg guard, or a running-table
conflict) returned without the finally ever running and left skipAuth set
for the rest of the connection.

Set the flag as the first statement inside the try instead, under the same
OLAP-only condition as before, so it is always paired with the reset in the
finally.

Add a regression test that runs a failing INSERT OVERWRITE ... @Branch and
asserts skipAuth is reset afterwards.
@CalvinKirs
CalvinKirs force-pushed the fix-insert-overwrite-skipauth branch from 0cab8bf to ceb4d2d Compare August 3, 2026 12:16
@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28539 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ceb4d2d9be44432b53467e75bf02fd7d382797d6, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17572	4243	3940	3940
q2	1993	314	195	195
q3	10329	1341	824	824
q4	4682	466	337	337
q5	7494	830	556	556
q6	179	163	141	141
q7	723	784	593	593
q8	9758	1658	1540	1540
q9	5770	4029	3999	3999
q10	6784	1594	1355	1355
q11	520	356	327	327
q12	709	583	464	464
q13	18078	3257	2682	2682
q14	269	257	243	243
q15	q16	728	727	657	657
q17	1142	1016	939	939
q18	6675	5618	5429	5429
q19	1197	1276	971	971
q20	818	666	591	591
q21	5693	2511	2453	2453
q22	427	351	303	303
Total cold run time: 101540 ms
Total hot run time: 28539 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4241	4130	4152	4130
q2	281	304	206	206
q3	4524	4879	4353	4353
q4	2128	2234	1392	1392
q5	4220	4026	4066	4026
q6	230	178	128	128
q7	1680	1579	1417	1417
q8	2575	2107	2049	2049
q9	7267	7161	7094	7094
q10	4315	4290	3963	3963
q11	569	390	367	367
q12	688	716	499	499
q13	3123	3431	3079	3079
q14	314	305	301	301
q15	q16	679	705	613	613
q17	1266	1277	1268	1268
q18	7836	7075	7189	7075
q19	1062	1039	1045	1039
q20	2191	2169	1899	1899
q21	5192	4480	4391	4391
q22	536	441	389	389
Total cold run time: 54917 ms
Total hot run time: 49678 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 169350 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ceb4d2d9be44432b53467e75bf02fd7d382797d6, data reload: false

query5	4308	621	471	471
query6	461	215	205	205
query7	4834	592	336	336
query8	344	188	166	166
query9	8744	3959	3965	3959
query10	472	351	315	315
query11	5799	2171	2003	2003
query12	149	95	96	95
query13	1233	611	445	445
query14	6095	4643	4349	4349
query14_1	3782	3759	3756	3756
query15	207	198	177	177
query16	993	466	455	455
query17	1135	739	554	554
query18	2465	481	354	354
query19	214	191	158	158
query20	115	101	99	99
query21	242	166	141	141
query22	12960	12943	12760	12760
query23	17225	16317	15905	15905
query23_1	16000	16044	16115	16044
query24	7499	1718	1226	1226
query24_1	1249	1254	1273	1254
query25	562	466	380	380
query26	1349	358	209	209
query27	2685	628	411	411
query28	4437	2038	1997	1997
query29	1032	590	501	501
query30	345	274	224	224
query31	1150	1078	932	932
query32	98	61	58	58
query33	493	296	234	234
query34	1183	1155	646	646
query35	732	750	650	650
query36	805	794	723	723
query37	147	107	93	93
query38	1849	1681	1611	1611
query39	852	839	821	821
query39_1	818	837	825	825
query40	240	168	142	142
query41	68	61	62	61
query42	92	87	91	87
query43	312	314	272	272
query44	1462	792	764	764
query45	189	178	168	168
query46	1069	1226	736	736
query47	1566	1565	1428	1428
query48	388	420	293	293
query49	579	405	298	298
query50	1082	419	364	364
query51	10810	10533	10662	10533
query52	85	85	103	85
query53	257	277	203	203
query54	280	233	229	229
query55	74	69	64	64
query56	300	299	295	295
query57	1031	980	929	929
query58	280	235	264	235
query59	1520	1588	1391	1391
query60	304	265	254	254
query61	152	147	146	146
query62	403	325	271	271
query63	234	192	192	192
query64	2801	1007	833	833
query65	3868	3835	3799	3799
query66	1808	465	358	358
query67	28180	28186	28014	28014
query68	3179	1524	1014	1014
query69	399	292	259	259
query70	910	805	788	788
query71	374	341	340	340
query72	3009	2759	2435	2435
query73	852	801	464	464
query74	4618	4520	4285	4285
query75	2404	2367	2010	2010
query76	2345	1210	771	771
query77	355	394	283	283
query78	11274	11152	10609	10609
query79	2200	1187	817	817
query80	1289	565	450	450
query81	552	325	290	290
query82	604	147	112	112
query83	364	329	295	295
query84	278	162	132	132
query85	999	602	500	500
query86	415	235	215	215
query87	1794	1797	1753	1753
query88	3779	2830	2797	2797
query89	401	317	288	288
query90	1881	199	191	191
query91	201	187	169	169
query92	60	59	56	56
query93	1757	1555	988	988
query94	707	359	312	312
query95	778	592	481	481
query96	1085	805	361	361
query97	2464	2425	2346	2346
query98	205	193	188	188
query99	714	728	607	607
Total cold run time: 256776 ms
Total hot run time: 169350 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ceb4d2d9be44432b53467e75bf02fd7d382797d6, data reload: false

query1	0.01	0.01	0.01
query2	0.09	0.06	0.05
query3	0.26	0.14	0.13
query4	1.60	0.14	0.14
query5	0.24	0.23	0.22
query6	1.16	0.84	0.83
query7	0.04	0.01	0.01
query8	0.06	0.04	0.04
query9	0.43	0.32	0.32
query10	0.57	0.59	0.55
query11	0.19	0.13	0.13
query12	0.18	0.14	0.14
query13	0.47	0.49	0.48
query14	1.00	1.00	1.01
query15	0.62	0.58	0.59
query16	0.34	0.34	0.33
query17	1.12	1.13	1.12
query18	0.22	0.22	0.21
query19	2.03	1.94	1.92
query20	0.01	0.02	0.01
query21	15.44	0.19	0.14
query22	4.91	0.05	0.05
query23	16.15	0.30	0.13
query24	3.03	0.42	0.32
query25	0.11	0.05	0.05
query26	0.75	0.20	0.15
query27	0.04	0.05	0.04
query28	3.56	0.71	0.34
query29	12.46	4.01	3.18
query30	0.29	0.15	0.16
query31	2.77	0.56	0.34
query32	3.22	0.59	0.51
query33	3.24	3.22	3.16
query34	15.60	3.91	3.28
query35	3.28	3.26	3.23
query36	0.54	0.42	0.42
query37	0.09	0.07	0.06
query38	0.04	0.04	0.03
query39	0.04	0.03	0.03
query40	0.17	0.14	0.14
query41	0.08	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 96.53 s
Total hot run time: 24.06 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 50.00% (1/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 50.00% (1/2) 🎉
Increment coverage report
Complete coverage report

@CalvinKirs
CalvinKirs merged commit d84b04e into apache:master Aug 4, 2026
29 of 30 checks passed
@CalvinKirs
CalvinKirs deleted the fix-insert-overwrite-skipauth branch August 4, 2026 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants