Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support table region split #881

Merged
merged 20 commits into from Jul 2, 2019

Conversation

Projects
None yet
3 participants
@zhexuany
Copy link
Member

commented Jul 1, 2019

In this PR, we start using tidb's api to do region presplit.

We remove original region split and scatter logic and some parameters.

A test case is added to ensure code works as expected.

@zhexuany zhexuany force-pushed the zhexuany:refine_region_presplit branch from 9fc6113 to 403336e Jul 1, 2019

@zhexuany zhexuany requested review from birdstorm and marsishandsome and removed request for birdstorm Jul 1, 2019

zhexuany added some commits Jul 2, 2019

zhexuany added some commits Jul 2, 2019

@@ -39,7 +39,6 @@ object TiConfigConst {
val DB_PREFIX: String = "spark.tispark.db_prefix"
val WRITE_ALLOW_SPARK_SQL: String = "spark.tispark.write.allow_spark_sql"
val WRITE_ENABLE: String = "spark.tispark.write.enable"

This comment has been minimized.

Copy link
@marsishandsome

marsishandsome Jul 2, 2019

Collaborator

do not need to delete this line, may conflict if cherry-pick latter

zhexuany added some commits Jul 2, 2019

@marsishandsome marsishandsome changed the title support table region presplit support table region split Jul 2, 2019

@zhexuany

This comment has been minimized.

Copy link
Member Author

commented Jul 2, 2019

/run-all-tests

val maxHandle = wrappedRowRdd.max().handle
// region split
if (regionSplitNum > 1) {
logger.info("region split is enabled.")

This comment has been minimized.

Copy link
@marsishandsome

marsishandsome Jul 2, 2019

Collaborator

suggest move 699-700 in if

)
)

test("region pre split test") {

This comment has been minimized.

Copy link
@marsishandsome

marsishandsome Jul 2, 2019

Collaborator

remove pre

@marsishandsome
Copy link
Collaborator

left a comment

others LGTM
pls make sure ci test success in both cases:

  1. tidb support region split
  2. tidb do not support region split
@zhexuany

This comment has been minimized.

Copy link
Member Author

commented Jul 2, 2019

/run-all-tests

@birdstorm
Copy link
Member

left a comment

LGTM

@zhexuany

This comment has been minimized.

Copy link
Member Author

commented Jul 2, 2019

/run-all-tests tikv=v2.1.9 tidb=v2.1.9 pd=v2.1.9

@zhexuany zhexuany merged commit a0916ae into pingcap:master Jul 2, 2019

@zhexuany zhexuany deleted the zhexuany:refine_region_presplit branch Jul 2, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.