Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cherry-pick 2.1] import,etc: implement a speed limit for tikv-importer (#4412) #4607

Merged
merged 4 commits into from May 5, 2019

Conversation

@kennytm
Copy link
Contributor

commented Apr 29, 2019

What have you changed? (mandatory)

Cherry-picks #4412 to release-2.1 (needed for users with low network bandwidth).

What are the type of the changes? (mandatory)

  • New feature (change which adds functionality)

How has this PR been tested? (mandatory)

Cherry-picked from master

Does this PR affect documentation (docs) or release note? (mandatory)

No? The release note would be part of Importer.

Does this PR affect tidb-ansible update? (mandatory)

Yes, will be filed later.

Refer to a related PR or issue link (optional)

#4412.

Benchmark result if necessary (optional)

Add a few positive/negative examples (optional)

* import,etc: implement a speed limit for tikv-importer

Implemented a maximum speed limit for uploading from Importer to TiKV,
using Token Bucket algorithm. The speed limit is needed to avoid
saturating the network bandwidth which causes PD to assume TiKV nodes
went down due to heartbeat not going through.

Signed-off-by: kennytm <kennytm@gmail.com>

* import: moved SpeedLimiter into its own module, and added unit tests

Signed-off-by: kennytm <kennytm@gmail.com>

* import: fix compile error

Signed-off-by: kennytm <kennytm@gmail.com>

* import: Duration::from_float_secs has been renamed to from_secs_f64

Signed-off-by: kennytm <kennytm@gmail.com>
Signed-off-by: kennytm <kennytm@gmail.com>
@kennytm kennytm force-pushed the kennytm:cherry-pick-4412 branch from 10a1082 to da4085f May 1, 2019
Signed-off-by: kennytm <kennytm@gmail.com>
@kennytm kennytm force-pushed the kennytm:cherry-pick-4412 branch from da4085f to 6622dcd May 1, 2019
@kennytm

This comment was marked as outdated.

Copy link
Contributor Author

commented May 2, 2019

/test

thread 'raftstore_cases::test_snap::test_server_concurrent_snap' panicked at 'called `Result::unwrap()` on an `Err` value: Other(StringError("[components/test_raftstore/cluster.rs:263]: can\'t get leader of region 1"))', libcore/result.rs:945:5
Signed-off-by: kennytm <kennytm@gmail.com>
Copy link
Member

left a comment

LGTM

@kennytm kennytm added the S: LGT1 label May 5, 2019
@kennytm

This comment has been minimized.

Copy link
Contributor Author

commented May 5, 2019

@hicqu

This comment has been minimized.

Copy link
Contributor

commented May 5, 2019

LGTM

@hicqu
hicqu approved these changes May 5, 2019
@hicqu hicqu merged commit 8789cf7 into tikv:release-2.1 May 5, 2019
2 checks passed
2 checks passed
DCO All commits are signed off!
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details
@kennytm kennytm deleted the kennytm:cherry-pick-4412 branch May 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.