Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds BatchTableScanExecutor #4351

Merged
merged 20 commits into from Mar 21, 2019

Conversation

@breeswish
Copy link
Member

commented Mar 11, 2019

What have you changed? (mandatory)

This PR adds BatchTableScanExecutor, which depends on:

  • RangesIterator: An abstracted module for handling iterating unknown number of rows over a given range.
  • ScanExecutor: A shared (for table scan and index scan) scan executor, which utilizes RangesIterator and scan over specified ranges. It accepts an Implementation which handles row and outputs it into the output batch vector.
  • BatchTableScanExecutor: Provides the table scan implementation, based on ScanExecutor.

This is extracted from #3989.

What are the type of the changes? (mandatory)

  • New feature (change which adds functionality)

How has this PR been tested? (mandatory)

Unit test.

  • TODO: Unit test default value.

Benchmark

Integration benchmarks

Schema:

CREATE TABLE `sbtest1` (
  `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  `k` int(10) UNSIGNED NOT NULL DEFAULT '0',
  `c` char(120) NOT NULL DEFAULT '',
  `pad` char(60) NOT NULL DEFAULT '',
  PRIMARY KEY (`id`)
);

Result:

Case SQL batch version master version
1 select id from sbtest1; 0.17 0.23
2 select k from sbtest1; 0.24 0.30
3 select * from sbtest1; 0.60 0.65

breeswish added some commits Mar 11, 2019

Introduce context for AsMySQLBool and remove BatchExecutorContext
Signed-off-by: Breezewish <breezewish@pingcap.com>
Extract table scanner
Signed-off-by: Breezewish <breezewish@pingcap.com>

@breeswish breeswish added the C: Copr label Mar 11, 2019

breeswish added some commits Mar 11, 2019

Merge remote-tracking branch 'origin/master' into ___batch_extract/ta…
…ble_scan

Signed-off-by: Breezewish <breezewish@pingcap.com>
Add new test cases from parent PR
Signed-off-by: Breezewish <breezewish@pingcap.com>
@breeswish

This comment has been minimized.

Copy link
Member Author

commented Mar 13, 2019

/run-integration-tests

Fix typo
Signed-off-by: Breezewish <breezewish@pingcap.com>

@breeswish breeswish force-pushed the breeswish:___batch_extract/table_scan branch from 00ccb72 to 363bd23 Mar 18, 2019

breeswish added some commits Mar 18, 2019

Merge master and upstream change
Signed-off-by: Breezewish <breezewish@pingcap.com>

@breeswish breeswish force-pushed the breeswish:___batch_extract/table_scan branch from 363bd23 to 7321cdd Mar 18, 2019

@breeswish

This comment has been minimized.

Copy link
Member Author

commented Mar 18, 2019

/run-integration-tests

breeswish added some commits Mar 19, 2019

Address comments to remove some unnecessary code
Signed-off-by: Breezewish <breezewish@pingcap.com>

breeswish added some commits Mar 19, 2019

Improve performance
Signed-off-by: Breezewish <breezewish@pingcap.com>
Add test cases for lock scenarios
Signed-off-by: Breezewish <breezewish@pingcap.com>

breeswish added some commits Mar 20, 2019

Place truncate function inside LazyColumnVec
Signed-off-by: Breezewish <breezewish@pingcap.com>

breeswish added some commits Mar 20, 2019

Merge two cases together
Signed-off-by: Breezewish <breezewish@pingcap.com>

match &is_drained {
// Note: `self.is_ended` is only used for assertion purpose.
Err(_) | Ok(true) => self.is_ended = true,

This comment has been minimized.

Copy link
@rleungx

rleungx Mar 21, 2019

Member

Can we directly use _?

This comment has been minimized.

Copy link
@breeswish

breeswish Mar 21, 2019

Author Member

Yes but I prefer to be more explicit to reduce thinking cost for readers (thinking what _ indicates)

breeswish added some commits Mar 21, 2019

Make truncate column more strict
Signed-off-by: Breezewish <breezewish@pingcap.com>
Add comments for warnings
Signed-off-by: Breezewish <breezewish@pingcap.com>
@AndreMouche
Copy link
Member

left a comment

LGTM

@breeswish

This comment has been minimized.

Copy link
Member Author

commented Mar 21, 2019

/rebuild

@AndreMouche AndreMouche merged commit c226478 into tikv:master Mar 21, 2019

2 checks passed

DCO All commits are signed off!
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details

@breeswish breeswish deleted the breeswish:___batch_extract/table_scan branch Mar 21, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.