planner: support using nested IN to build IndexMerge path

[time-and-fate]

What problem does this PR solve?

Issue Number: ref #65822

Problem Summary:

For queries like SELECT * FROM t1 WHERE e = 1 AND (a IN (1,2,3) OR b IN (2,3,4) OR c IN (3,4,5)), TiDB previously could not build an IndexMerge path when there are IN expressions in the nested OR list. The query would fall back to a plain IndexLookUp with a residual Selection, which is much less efficient.

This is the first optimization described in the issue.

What changed and how does it work?

• planner
  • checkAccessFilter4IdxCol() (pkg/planner/core/indexmerge_path.go): Add support for ast.In expressions in the non-virtual column branch. Previously only ast.EQ was recognized, so IN expressions like a IN (1,2,3) could not be collected as partial access filters in the "gradual collection" path (case 3 in initUnfinishedPathsFromExpr()). Now they are collected and later combined with top-level AND conditions (e.g., e = 1) by handleTopLevelANDList() to build valid ranges for composite indexes.
  • Rename eqOnNonMVColTp to eqOrInOnNonMVColTp to reflect that it now covers both EQ and IN expressions.

After this fix, the plan becomes:

IndexMerge
├─IndexRangeScan  index:iea(e, a)  range:[1 1,1 1], [1 2,1 2], [1 3,1 3]
├─IndexRangeScan  index:ieb(e, b)  range:[1 2,1 2], [1 3,1 3], [1 4,1 4]
├─IndexRangeScan  index:iec(e, c)  range:[1 3,1 3], [1 4,1 4], [1 5,1 5]
└─TableRowIDScan(Probe)

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

Summary by CodeRabbit

• Performance Improvements

  • Improved optimizer so IN predicates on indexed non-virtual columns are planned like equality, yielding more effective index-merge range scans.

• Behavior Changes

  • Index-merge plans now emit separate range entries for multi-value IN operands and may alter ordering in range scans.

• Testing

  • Added and updated integration tests and expected plans to validate IN + OR scenarios and LIMIT interactions.

[pantheon-ai]

@time-and-fate I've received your pull request and will start the review. I'll conduct a thorough review covering code quality, potential issues, and implementation details.

⏳ This process typically takes 10-30 minutes depending on the complexity of the changes.

_{ℹ️ Learn more details on Pantheon AI.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Recognize scalar IN with constant operands on non-MV index columns as an EQ-like access filter (eqOrInOnNonMVColTp), propagate that filter type through MV-index filter collection and unfinished-path initialization, and update integration tests/expected plans to reflect resulting index-merge ranges.

Changes

Index Merge IN-Predicate Support

Layer / File(s)	Summary
Access-filter type definition and EQ mapping pkg/planner/core/indexmerge_path.go	Introduces eqOrInOnNonMVColTp and maps the EQ branch of checkAccessFilter4IdxCol to return this unified non-MV access-filter type.
IN predicate recognition and constant validation pkg/planner/core/indexmerge_path.go	Expands checkAccessFilter4IdxCol to recognize scalar IN predicates, require the first arg to be the index column, check there are >=2 args, and ensure all IN operands are constants before returning eqOrInOnNonMVColTp.
Filter propagation through planning pipeline pkg/planner/core/indexmerge_path.go, pkg/planner/core/indexmerge_unfinished_path.go	Updates collectFilters4MVIndex to allow eqOrInOnNonMVColTp to overwrite/combine previous access types and updates initUnfinishedPathsFromExpr to accept eqOrInOnNonMVColTp when collecting usable CNF filters.
Integration test coverage and expected plans tests/integrationtest/t/planner/core/indexmerge_path.test, tests/integrationtest/r/planner/core/indexmerge_path.result, tests/integrationtest/r/planner/core/casetest/physicalplantest/physical_plan.result	Adds TestIndexMergeINInORList and updates expected plan_tree outputs to show IndexMerge builds enumerating IN-value ranges and adjusted keep order:false flags where applicable.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

size/L, ok-to-test, approved, lgtm

Suggested reviewers

• qw4990
• winoros
• terry1purcell

Poem

A rabbit hopped through filter trees,
Found IN and EQ were friends at ease,
Constants counted, ranges aligned,
Index merges now better designed. 🐰✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description includes the required issue reference, problem summary, detailed explanation of changes, test selection (integration test added), side effects checked, and release note; all major template sections are addressed.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title accurately reflects the main change: extending the planner to support IN expressions (in addition to EQ) when building IndexMerge paths, which is the core objective of this PR.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.0967%. Comparing base (032f5ac) to head (954a7ac).
⚠️ Report is 9 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #68962        +/-   ##
================================================
- Coverage   76.3186%   75.0967%   -1.2220%     
================================================
  Files          2041       2028        -13     
  Lines        562849     570593      +7744     
================================================
- Hits         429559     428497      -1062     
- Misses       132377     141917      +9540     
+ Partials        913        179       -734     

Flag	Coverage Δ
integration	41.4643% <66.6666%> (+1.7256%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	60.4610% <ø> (ø)
parser	∅ <ø> (∅)
br	49.3992% <ø> (-13.4004%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: qw4990, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [qw4990,winoros]
• pkg/planner/OWNERS [qw4990,winoros]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-06-05 14:22:09.473116898 +0000 UTC m=+537830.543434287: ☑️ agreed by winoros.
• 2026-06-06 12:12:58.892339805 +0000 UTC m=+616479.962657205: ☑️ agreed by qw4990.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-8.5: #69003.
But this PR has conflicts, please resolve them!