New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
*: fix union scan for partitioned table #8871
Conversation
1. dirty table is table level, so AddRecord should use tableID instead of physicalTableID 2. UnionScan should not be removed for the Union generated by table partition
PTAL @lamxTyler @zz-jason |
PTAL @zz-jason @lamxTyler |
@tiancaiamao any update? |
Codecov Report
@@ Coverage Diff @@
## master #8871 +/- ##
==========================================
+ Coverage 67.4% 67.42% +0.01%
==========================================
Files 373 373
Lines 78544 78593 +49
==========================================
+ Hits 52946 52990 +44
- Misses 20838 20844 +6
+ Partials 4760 4759 -1
Continue to review full report at Codecov.
|
I'll do a tiny refactor to remove the physicalID from three reader after this PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/run-all-tests |
/run-all-tests |
/run-integration-compatibility-test |
PTAL @lamxTyler |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@tiancaiamao Need cherry pick to v2.1. |
What problem does this PR solve?
What is changed and how it works?
How does union scan works for partitioned table:
rule_partition_processor.go
, the rewrite rule changesto
Note that UnionScan is now the parent of every Partition, rather than the previous
UnionScan->Partition1 transformed to UnionScanExec+three reader, the reader read data from partition, and UnionExec get the incremental buffer changes on partition, not table
To get the incremental buffer changes on partition:
DirtyDB = map(physicalID -> changes)
In the three kinds of reader(TableReader,IndexReader,IndexLookup) struct,
table
is also changed to partition (PhysicalTable
)The function signature
StmtAddDirtyTableOP
should beinstead of
This change is