Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner: support OR list nested in AND list for mv index | tidb-test=pr/2309 #51716

Merged
merged 45 commits into from
Mar 18, 2024

Conversation

time-and-fate
Copy link
Member

@time-and-fate time-and-fate commented Mar 12, 2024

What problem does this PR solve?

Issue Number: close #51778

What changed and how does it work?

Use new methods to replace the original implementation for DNF case in generateIndexMergeOnDNF4MVIndex() and generateIndexMerge4ComposedIndex().
Please see comments for details.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Mar 12, 2024
Copy link

codecov bot commented Mar 12, 2024

Codecov Report

Merging #51716 (bbabe1e) into master (26e4988) will decrease coverage by 17.8248%.
Report is 2 commits behind head on master.
The diff coverage is 95.3068%.

Additional details and impacted files
@@                Coverage Diff                @@
##             master     #51716         +/-   ##
=================================================
- Coverage   72.4655%   54.6408%   -17.8248%     
=================================================
  Files          1476       1588        +112     
  Lines        364511     607934     +243423     
=================================================
+ Hits         264145     332180      +68035     
- Misses        80908     252695     +171787     
- Partials      19458      23059       +3601     
Flag Coverage Δ
integration 36.6463% <93.1407%> (?)
unit 70.4688% <90.2527%> (-1.8792%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.9957% <ø> (-2.3014%) ⬇️
parser ∅ <ø> (∅)
br 51.5326% <ø> (+5.1258%) ⬆️

@time-and-fate
Copy link
Member Author

/retest

@time-and-fate time-and-fate changed the title planner: [WIP] planner: [WIP] | tidb-test=pr/2309 Mar 13, 2024
@time-and-fate
Copy link
Member Author

/retest

Comment on lines -57 to +60
IndexMerge_9 0.01 root type: union
├─IndexRangeScan_5(Build) 10.00 cop[tikv] table:t2, index:idx(c, cast(`a` as signed array)) range:[1,1], keep order:false, stats:pseudo
├─IndexRangeScan_6(Build) 0.10 cop[tikv] table:t2, index:idx2(cast(`b` as signed array), c) range:[2 3,2 3], keep order:false, stats:pseudo
└─Selection_8(Probe) 0.01 cop[tikv] or(and(json_contains(planner__core__casetest__index__index.t2.a, cast("[1, 2, 3]", json BINARY)), and(eq(planner__core__casetest__index__index.t2.c, 1), eq(planner__core__casetest__index__index.t2.d, 2))), and(json_memberof(cast(2, json BINARY), planner__core__casetest__index__index.t2.b), and(eq(planner__core__casetest__index__index.t2.c, 3), eq(planner__core__casetest__index__index.t2.d, 2))))
└─TableRowIDScan_7 10.10 cop[tikv] table:t2 keep order:false, stats:pseudo
IndexLookUp_11 8.00 root
├─IndexRangeScan_8(Build) 10.00 cop[tikv] table:t2, index:idx3(c, d) range:[1 2,1 2], [3 2,3 2], keep order:false, stats:pseudo
└─Selection_10(Probe) 8.00 cop[tikv] or(and(json_contains(planner__core__casetest__index__index.t2.a, cast("[1, 2, 3]", json BINARY)), and(eq(planner__core__casetest__index__index.t2.c, 1), eq(planner__core__casetest__index__index.t2.d, 2))), and(json_memberof(cast(2, json BINARY), planner__core__casetest__index__index.t2.b), and(eq(planner__core__casetest__index__index.t2.c, 3), eq(planner__core__casetest__index__index.t2.d, 2))))
└─TableRowIDScan_9 10.00 cop[tikv] table:t2 keep order:false, stats:pseudo
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to #50183, we should not generate a scan on a mv index if the access filter doesn't cover the mv column.

verifyPlanCacheForMVIndex(t, tk, true, false,
verifyPlanCacheForMVIndex(t, tk, false, false,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.
Actually this test case uses exactly the same query.

@time-and-fate
Copy link
Member Author

/hold
wait for tidb-test pr

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 15, 2024
Copy link
Contributor

@AilinKid AilinKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

pkg/planner/core/indexmerge_unfinished_path.go Outdated Show resolved Hide resolved
pkg/planner/core/indexmerge_unfinished_path.go Outdated Show resolved Hide resolved
pkg/planner/core/indexmerge_unfinished_path.go Outdated Show resolved Hide resolved
Comment on lines +397 to +399
if len(bestPaths) == 0 {
return nil
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add one more test case for this branch, for example:

create table t (a int, b int, c int, d int, j json, key kb(b, (cast(j as unsigned array))), key(d, c));
explain select * from t where (c=1 or b=1) and (1 member of (j));
explain select * from t where (c=1 or b=1) and (1 member of (j)) and d=1;

Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REST LGTM

@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Mar 18, 2024
@time-and-fate
Copy link
Member Author

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 18, 2024
Copy link

ti-chi-bot bot commented Mar 18, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AilinKid, qw4990

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Mar 18, 2024
Copy link

ti-chi-bot bot commented Mar 18, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-03-18 04:16:19.562052158 +0000 UTC m=+1263806.584298546: ☑️ agreed by AilinKid.
  • 2024-03-18 05:49:51.856645438 +0000 UTC m=+1269418.878891826: ☑️ agreed by qw4990.

@ti-chi-bot ti-chi-bot bot merged commit af76c2f into pingcap:master Mar 18, 2024
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable optimizer to generate index merge path for mv index + OR list nested in AND list
3 participants