Skip to content

[opt](nereids) skip null-check access path for physically non-nullable columns#63468

Open
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:fix-rqg-dev-5.21
Open

[opt](nereids) skip null-check access path for physically non-nullable columns#63468
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:fix-rqg-dev-5.21

Conversation

@Mryange
Copy link
Copy Markdown
Contributor

@Mryange Mryange commented May 21, 2026

What problem does this PR solve?

Issue Number: N/A

Problem Summary:

When collecting access path expressions for variant types, null-check paths (ACCESS_NULL) were generated even for columns that are physically non-nullable (i.e., NOT NULL in schema). This introduced unnecessary access paths that have no effect, since null checks on non-nullable columns are always no-ops.

Root cause: visitSlotReference did not check whether the slot's underlying column is physically nullable before processing function null check paths.

Fix: In visitSlotReference, skip processing when the access path is a function null check path and the slot reference is physically non-nullable. Physical nullability is determined by the original column's isAllowNull() metadata, falling back to the slot's nullable() flag.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label May 22, 2026
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@Mryange
Copy link
Copy Markdown
Contributor Author

Mryange commented May 25, 2026

run buildall

@Mryange Mryange changed the title [fix]AccessPathExpressionCollector [opt](nereids) skip null-check access path for physically non-nullable columns May 25, 2026
@Mryange Mryange marked this pull request as ready for review May 25, 2026 01:58
@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31131 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f5a56fe03cca6c250a44abda7c30c32f0ce04daa, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17913	4000	3976	3976
q2	q3	10867	1402	822	822
q4	4684	479	350	350
q5	7579	2275	2131	2131
q6	351	177	138	138
q7	937	793	650	650
q8	9433	1739	1613	1613
q9	6764	4953	4961	4953
q10	6437	2253	1889	1889
q11	449	276	244	244
q12	685	423	312	312
q13	18184	3413	2783	2783
q14	270	259	241	241
q15	q16	822	792	705	705
q17	957	822	963	822
q18	6786	5757	5524	5524
q19	1254	1319	1069	1069
q20	508	400	254	254
q21	5716	2537	2352	2352
q22	437	355	303	303
Total cold run time: 101033 ms
Total hot run time: 31131 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4385	4310	4291	4291
q2	q3	4573	4943	4338	4338
q4	2129	2236	1442	1442
q5	4467	4328	4752	4328
q6	250	200	150	150
q7	2095	1940	1616	1616
q8	2533	2191	2183	2183
q9	8059	7942	8003	7942
q10	4892	4836	4540	4540
q11	591	410	391	391
q12	753	766	556	556
q13	3295	3706	3078	3078
q14	284	318	280	280
q15	q16	733	772	682	682
q17	1380	1373	1314	1314
q18	7918	7469	6992	6992
q19	1139	1098	1115	1098
q20	2243	2241	1958	1958
q21	5345	4632	4531	4531
q22	521	460	411	411
Total cold run time: 57585 ms
Total hot run time: 52121 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 172638 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f5a56fe03cca6c250a44abda7c30c32f0ce04daa, data reload: false

query5	4310	643	525	525
query6	334	235	196	196
query7	4241	590	322	322
query8	339	229	218	218
query9	8819	4093	4115	4093
query10	445	332	294	294
query11	5783	2577	2272	2272
query12	177	126	131	126
query13	1264	637	438	438
query14	6214	5560	5245	5245
query14_1	4496	4558	4529	4529
query15	210	207	192	192
query16	990	447	408	408
query17	1146	720	592	592
query18	2521	472	353	353
query19	250	200	167	167
query20	137	130	131	130
query21	211	142	117	117
query22	13729	13662	13434	13434
query23	17462	16550	16382	16382
query23_1	16261	16233	16365	16233
query24	7518	1777	1297	1297
query24_1	1325	1330	1321	1321
query25	576	484	444	444
query26	1340	342	178	178
query27	2708	547	357	357
query28	4481	2038	2013	2013
query29	1023	635	537	537
query30	312	243	203	203
query31	1144	1083	970	970
query32	90	77	76	76
query33	573	372	330	330
query34	1206	1131	663	663
query35	767	816	705	705
query36	1503	1407	1296	1296
query37	159	109	94	94
query38	3294	3205	3142	3142
query39	962	961	928	928
query39_1	922	917	911	911
query40	234	167	133	133
query41	73	70	70	70
query42	115	115	109	109
query43	343	344	305	305
query44	
query45	219	212	200	200
query46	1091	1192	736	736
query47	2374	2381	2254	2254
query48	419	428	328	328
query49	653	519	412	412
query50	983	352	252	252
query51	4372	4411	4345	4345
query52	109	108	98	98
query53	255	293	211	211
query54	338	310	269	269
query55	93	93	86	86
query56	318	311	326	311
query57	1443	1436	1341	1341
query58	323	285	278	278
query59	1641	1667	1444	1444
query60	329	341	322	322
query61	184	185	178	178
query62	711	656	593	593
query63	280	206	207	206
query64	2408	787	626	626
query65	
query66	1756	479	358	358
query67	29416	30022	29906	29906
query68	
query69	479	342	304	304
query70	1031	1007	1035	1007
query71	307	270	269	269
query72	3038	2655	2383	2383
query73	860	756	454	454
query74	5203	5014	4817	4817
query75	2723	2626	2272	2272
query76	2302	1155	762	762
query77	417	410	332	332
query78	12406	12588	11973	11973
query79	1537	1069	767	767
query80	1341	526	444	444
query81	517	282	241	241
query82	1006	157	119	119
query83	351	277	254	254
query84	263	136	114	114
query85	928	539	439	439
query86	460	325	317	317
query87	3430	3412	3274	3274
query88	3639	2760	2709	2709
query89	453	393	342	342
query90	1930	182	177	177
query91	178	169	148	148
query92	79	78	74	74
query93	1539	1481	906	906
query94	721	354	300	300
query95	659	370	441	370
query96	994	778	344	344
query97	2748	2736	2586	2586
query98	235	232	232	232
query99	1168	1187	1057	1057
Total cold run time: 255715 ms
Total hot run time: 172638 ms

@Mryange
Copy link
Copy Markdown
Contributor Author

Mryange commented May 25, 2026

run p0

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 8.00% (6/75) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants