[BugFix]: Remove unnecessary projections in master index

[arjunsk]

User description

What type of PR is this?

• API-change
• BUG
• Improvement
• Documentation
• Feature
• Test and CI
• Code Refactoring

Which issue(s) this PR fixes:

issue # #14876

What this PR does / why we need it:

Remove unnecessary projects from Master Index.

mysql> explain analyze SELECT tbl.a100  FROM tbl  WHERE tbl.a75 = 'I2nJ0RqIQu';
+---------------------------------------------------------------------------------------------------------------------------------------------------------+
| AP QUERY PLAN ON MULTICN(10 core)                                                                                                                       |
+---------------------------------------------------------------------------------------------------------------------------------------------------------+
| Project                                                                                                                                                 |
|   Analyze: timeConsumed=0ms waitTime=6ms inputRows=2 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=0bytes                                |
|   ->  Join                                                                                                                                              |
|         Analyze: timeConsumed=2ms waitTime=44ms inputRows=2 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=0bytes                         |
|         Join Type: INDEX                                                                                                                                |
|         Join Cond: (tbl.a100 = #[1,0])                                                                                                                  |
|         Runtime Filter Build: #[-1,0]                                                                                                                   |
|         ->  Table Scan on a.tbl [ForceOneCN]                                                                                                            |
|               Analyze: timeConsumed=0ms waitTime=0ms inputBlocks=1 inputRows=1 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=50bytes     |
|               Filter Cond: (tbl.a75 = 'I2nJ0RqIQu')                                                                                                     |
|               Block Filter Cond: (tbl.a75 = 'I2nJ0RqIQu')                                                                                               |
|               Runtime Filter Probe: tbl.a100                                                                                                            |
|         ->  Table Scan on a.__mo_index_secondary_019003d8-3fd8-7455-b750-bd977ca13178 [ForceOneCN]                                                      |
|               Analyze: timeConsumed=2ms waitTime=0ms inputBlocks=6 inputRows=49152 outputRows=1 InputSize=3mb OutputSize=24bytes MemorySize=696320bytes |
|               Filter Cond: prefix_eq(#[0,0], 'F74 FI2nJ0RqIQu ')                                                                                      |
|               Block Filter Cond: prefix_eq(#[0,0], 'F74 FI2nJ0RqIQu ')                                                                                |
+---------------------------------------------------------------------------------------------------------------------------------------------------------+
16 rows in set (0.01 sec)


mysql> explain analyze SELECT tbl.a100  FROM tbl  WHERE tbl.a37 = '3Tfm6CEXy5' AND tbl.a94 = '6PRBdXpsVB';
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| AP QUERY PLAN ON MULTICN(10 core)                                                                                                                                                                         |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Project                                                                                                                                                                                                   |
|   Analyze: timeConsumed=0ms waitTime=15ms inputRows=3 outputRows=1 InputSize=72bytes OutputSize=24bytes MemorySize=0bytes                                                                                 |
|   ->  Join                                                                                                                                                                                                |
|         Analyze: timeConsumed=4ms waitTime=72ms inputRows=2 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=0bytes                                                                           |
|         Join Type: INDEX                                                                                                                                                                                  |
|         Join Cond: (tbl.a100 = #[1,0])                                                                                                                                                                    |
|         Runtime Filter Build: #[-1,0]                                                                                                                                                                     |
|         ->  Table Scan on a.tbl [ForceOneCN]                                                                                                                                                              |
|               Analyze: timeConsumed=0ms waitTime=0ms inputBlocks=1 inputRows=1 outputRows=1 InputSize=72bytes OutputSize=24bytes MemorySize=75bytes                                                       |
|               Filter Cond: (tbl.a94 = '6PRBdXpsVB'), (tbl.a37 = '3Tfm6CEXy5')                                                                                                                             |
|               Block Filter Cond: (tbl.a94 = '6PRBdXpsVB'), (tbl.a37 = '3Tfm6CEXy5')                                                                                                                       |
|               Runtime Filter Probe: tbl.a100                                                                                                                                                              |
|         ->  Join                                                                                                                                                                                          |
|               Analyze: timeConsumed=4ms probe_time=[total=0ms,min=0ms,max=0ms,dop=10] build_time=[4ms] waitTime=57ms inputRows=2 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=361187bytes |
|               Join Type: INNER                                                                                                                                                                            |
|               Join Cond: (#[0,0] = #[1,0])                                                                                                                                                                |
|               ->  Table Scan on a.__mo_index_secondary_019003d8-3fd8-7455-b750-bd977ca13178 [ForceOneCN]                                                                                                  |
|                     Analyze: timeConsumed=3ms waitTime=0ms inputBlocks=4 inputRows=32768 outputRows=1 InputSize=2mb OutputSize=24bytes MemorySize=679936bytes                                             |
|                     Filter Cond: prefix_eq(#[0,0], 'F36 F3Tfm6CEXy5 ')                                                                                                                                  |
|                     Block Filter Cond: prefix_eq(#[0,0], 'F36 F3Tfm6CEXy5 ')                                                                                                                            |
|               ->  Table Scan on a.__mo_index_secondary_019003d8-3fd8-7455-b750-bd977ca13178 [ForceOneCN]                                                                                                  |
|                     Analyze: timeConsumed=3ms waitTime=0ms inputBlocks=6 inputRows=49152 outputRows=1 InputSize=3mb OutputSize=24bytes MemorySize=696320bytes                                             |
|                     Filter Cond: prefix_eq(#[0,0], 'F93 F6PRBdXpsVB ')                                                                                                                                  |
|                     Block Filter Cond: prefix_eq(#[0,0], 'F93 F6PRBdXpsVB ')                                                                                                                            |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
24 rows in set (0.00 sec)

---

PR Type

Bug fix

---

Description

• Removed unnecessary projections from index table scans in apply_indices_master.go.
• Adjusted column positions for primary key columns in apply_indices_master.go.
• Added a check in runtime_filter.go to skip runtime filter generation for Master Index INNER Joins.
• Implemented isMasterIndexInnerJoin function to identify Master Index INNER Joins and prevent runtime filter application.

---

Changes walkthrough 📝

Relevant files

Bug fix

apply_indices_master.go Remove unnecessary projections and adjust column positions pkg/sql/plan/apply_indices_master.go Removed unnecessary projections from index table scans. Adjusted column positions for primary key columns. +5/-23    runtime_filter.go Skip runtime filters for Master Index INNER Joins                pkg/sql/plan/runtime_filter.go Added a check to skip runtime filter generation for Master Index INNER Joins. Implemented isMasterIndexInnerJoin function to identify such joins. +62/-0

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[codiumai-pr-agent-pro]

PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

PR Reviewer Guide 🔍

⏱️ Estimated effort to review [1-5]	3
🧪 Relevant tests	No
🔒 Security concerns	No
⚡ Key issues to review	Variable Naming Consistency: The variable currIdxProjTag was renamed to currIdxScanTag, and currIdxScanTag is used consistently in subsequent code. Ensure that this renaming aligns with the intended functionality and does not introduce confusion or errors.
	Column Position Adjustment: The ColPos for __mo_index_pk_col was changed from 0 to 1. Verify that this change correctly reflects the data structure and does not lead to incorrect data retrieval or processing errors.
	Removal of Projection Node: The projection node related to __mo_index_pk_col was removed, and the function now directly returns idxScanTag and scanId. Confirm that this change does not affect the expected output of the function, especially in terms of data structure and flow.

[codiumai-pr-agent-pro]

PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

PR Code Suggestions ✨

Category	Suggestion	Score
Possible issue	Add error handling for the Resolve function call to manage potential errors Consider adding error handling for the Resolve function call to ensure that any issues during the resolution of idxObjRef and idxTableDef are properly managed. pkg/sql/plan/apply_indices_master.go [44] -idxObjRef, idxTableDef := builder.compCtx.Resolve(scanNode.ObjRef.SchemaName, indexDef.IndexTableName, Snapshot{TS: &timestamp.Timestamp{}}) +idxObjRef, idxTableDef, err := builder.compCtx.Resolve(scanNode.ObjRef.SchemaName, indexDef.IndexTableName, Snapshot{TS: &timestamp.Timestamp{}}) +if err != nil { + return err +} Apply this suggestion Suggestion importance[1-10]: 9 Why: This suggestion addresses a potential error handling issue which is crucial for robustness, especially in database operations where failure to resolve indices could lead to significant application errors.	9
	Check if node is nil before accessing its properties to avoid potential issues To avoid potential issues, consider checking if node is nil before accessing its properties in the isMasterIndexInnerJoin function. pkg/sql/plan/runtime_filter.go [272-273] -if !(node.JoinType == plan.Node_INNER && len(node.Children) == 2) { +if node == nil || !(node.JoinType == plan.Node_INNER && len(node.Children) == 2) { return false } Apply this suggestion Suggestion importance[1-10]: 8 Why: This suggestion is highly relevant as it prevents a possible nil pointer dereference, which is a critical error. Adding a nil check is a best practice in Go, especially before accessing properties of a pointer.	8
Maintainability	Rename currScanId to currIdxScanId for consistency in variable naming To improve readability and maintainability, consider renaming the variable currScanId to currIdxScanId to make it consistent with the naming convention used for currIdxScanTag. pkg/sql/plan/apply_indices_master.go [47] -currIdxScanTag, currScanId := makeIndexTblScan(builder, builder.ctxByNode[nodeID], filterExp, idxTableDef, idxObjRef, scanNode.ScanSnapshot, colDefs) +currIdxScanTag, currIdxScanId := makeIndexTblScan(builder, builder.ctxByNode[nodeID], filterExp, idxTableDef, idxObjRef, scanNode.ScanSnapshot, colDefs) Apply this suggestion Suggestion importance[1-10]: 7 Why: The suggestion correctly identifies a minor inconsistency in variable naming which affects readability. Renaming currScanId to currIdxScanId aligns with the naming of currIdxScanTag, enhancing code consistency.	7
Performance	Break out of the loop early if a non-matching column is found to enhance performance To enhance performance, consider breaking out of the loop early if a non-matching column is found in the isMasterIndexInnerJoin function. pkg/sql/plan/runtime_filter.go [293-304] for _, column := range leftChild.TableDef.Cols { - if column.Name == catalog.MasterIndexTablePrimaryColName { - continue + if column.Name != catalog.MasterIndexTablePrimaryColName && column.Name != catalog.MasterIndexTableIndexColName && column.Name != catalog.Row_ID { + return false } - if column.Name == catalog.MasterIndexTableIndexColName { - continue - } - if column.Name == catalog.Row_ID { - continue - } - return false } Apply this suggestion Suggestion importance[1-10]: 6 Why: The suggestion correctly identifies a potential performance optimization by breaking out of the loop early. However, the impact on overall performance might be limited depending on the typical size of Cols.	6