Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[cherry-pick 2.1] import,etc: implement a speed limit for tikv-importer (#4412) #4607
What have you changed? (mandatory)
Cherry-picks #4412 to release-2.1 (needed for users with low network bandwidth).
What are the type of the changes? (mandatory)
How has this PR been tested? (mandatory)
Cherry-picked from master
Does this PR affect documentation (docs) or release note? (mandatory)
No? The release note would be part of Importer.
Does this PR affect tidb-ansible update? (mandatory)
Yes, will be filed later.
Refer to a related PR or issue link (optional)
Benchmark result if necessary (optional)
Add a few positive/negative examples (optional)
* import,etc: implement a speed limit for tikv-importer Implemented a maximum speed limit for uploading from Importer to TiKV, using Token Bucket algorithm. The speed limit is needed to avoid saturating the network bandwidth which causes PD to assume TiKV nodes went down due to heartbeat not going through. Signed-off-by: kennytm <firstname.lastname@example.org> * import: moved SpeedLimiter into its own module, and added unit tests Signed-off-by: kennytm <email@example.com> * import: fix compile error Signed-off-by: kennytm <firstname.lastname@example.org> * import: Duration::from_float_secs has been renamed to from_secs_f64 Signed-off-by: kennytm <email@example.com>