From 3bd7f2c25278ec8dbef2a39ecc879069ef884d2d Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 2 Jul 2020 14:44:29 +0800 Subject: [PATCH 1/3] cherry pick #2947 to release-3.1 Signed-off-by: ti-srebot --- TOC.md | 123 +++++++++++++++++++++++++++++++++++++++ configure-store-limit.md | 69 ++++++++++++++++++++++ 2 files changed, 192 insertions(+) create mode 100644 configure-store-limit.md diff --git a/TOC.md b/TOC.md index ff7cf41eaeda9..3318eca7e90ab 100644 --- a/TOC.md +++ b/TOC.md @@ -91,6 +91,7 @@ - [Identify Slow Queries](/identify-slow-queries.md) - [Identify Expensive Queries](/identify-expensive-queries.md) + Scale +<<<<<<< HEAD - [Scale using Ansible](/scale-tidb-using-ansible.md) + Upgrade - [Upgrade to TiDB 3.1](/upgrade-tidb-using-ansible.md) @@ -98,6 +99,128 @@ - [TiDB Troubleshooting Map](/tidb-troubleshooting-map.md) - [Troubleshoot Cluster Setup](/troubleshoot-tidb-cluster.md) - [Troubleshoot TiDB Lightning](/troubleshoot-tidb-lightning.md) +======= + + [Use TiUP (Recommended)](/scale-tidb-using-tiup.md) + + [Use TiDB Ansible](/scale-tidb-using-ansible.md) + + [Use TiDB Operator](https://docs.pingcap.com/tidb-in-kubernetes/v1.1/scale-a-tidb-cluster) + + Backup and Restore + + [Use Mydumper and TiDB Lightning](/backup-and-restore-using-mydumper-lightning.md) + + [Use Dumpling for Export or Backup](/export-or-backup-using-dumpling.md) + + Use BR Tool + + [Use BR Tool](/br/backup-and-restore-tool.md) + + [BR Use Cases](/br/backup-and-restore-use-cases.md) + + [BR storages](/br/backup-and-restore-storages.md) + + [Configure Time Zone](/configure-time-zone.md) + + [Daily Checklist](/daily-check.md) + + [Manage TiCDC Cluster and Replication Tasks](/ticdc/manage-ticdc.md) + + [Maintain TiFlash](/tiflash/maintain-tiflash.md) + + [Maintain TiDB Using TiUP](/maintain-tidb-using-tiup.md) + + [Maintain TiDB Using Ansible](/maintain-tidb-using-ansible.md) ++ Monitor and Alert + + [Monitoring Framework Overview](/tidb-monitoring-framework.md) + + [Monitoring API](/tidb-monitoring-api.md) + + [Deploy Monitoring Services](/deploy-monitoring-services.md) + + [TiDB Cluster Alert Rules](/alert-rules.md) + + [TiFlash Alert Rules](/tiflash/tiflash-alert-rules.md) ++ Troubleshoot + + [Identify Slow Queries](/identify-slow-queries.md) + + [SQL Diagnostics](/system-tables/system-table-sql-diagnostics.md) + + [Identify Expensive Queries](/identify-expensive-queries.md) + + [Statement Summary Tables](/statement-summary-tables.md) + + [Troubleshoot Cluster Setup](/troubleshoot-tidb-cluster.md) + + [TiDB Troubleshooting Map](/tidb-troubleshooting-map.md) + + [Troubleshoot TiCDC](/ticdc/troubleshoot-ticdc.md) + + [Troubleshoot TiFlash](/tiflash/troubleshoot-tiflash.md) ++ Performance Tuning + + Software Tuning + + Configuration + + [Tune TiDB Memory](/configure-memory-usage.md) + + [Tune TiKV Threads](/tune-tikv-thread-performance.md) + + [Tune TiKV Memory](/tune-tikv-memory-performance.md) + + [TiKV Follower Read](/follower-read.md) + + [TiFlash Tuning](/tiflash/tune-tiflash-performance.md) + + [Coprocessor Cache](/coprocessor-cache.md) + + SQL Tuning + + [SQL Tuning with `EXPLAIN`](/query-execution-plan.md) + + SQL Optimization + + [SQL Optimization Process](/sql-optimization-concepts.md) + + Logic Optimization + + [Join Reorder](/join-reorder.md) + + Physical Optimization + + [Statistics](/statistics.md) + + Control Execution Plan + + [Optimizer Hints](/optimizer-hints.md) + + [SQL Plan Management](/sql-plan-management.md) + + [Access Tables Using `IndexMerge`](/index-merge.md) ++ Tutorials + + [Multiple Data Centers in One City Deployment](/multi-data-centers-in-one-city-deployment.md) + + [Three Data Centers in Two Cities Deployment](/three-data-centers-in-two-cities-deployment.md) + + Best Practices + + [Use TiDB](/tidb-best-practices.md) + + [Java Application Development](/best-practices/java-app-best-practices.md) + + [Use HAProxy](/best-practices/haproxy-best-practices.md) + + [Highly Concurrent Write](/best-practices/high-concurrency-best-practices.md) + + [Grafana Monitoring](/best-practices/grafana-monitor-best-practices.md) + + [PD Scheduling](/best-practices/pd-scheduling-best-practices.md) + + [TiKV Performance Tuning with Massive Regions](/best-practices/massive-regions-best-practices.md) + + [Use Placement Rules](/configure-placement-rules.md) + + [Use Load Base Split](/configure-load-base-split.md) + + [Use Store Limit](/configure-store-limit.md) ++ TiDB Ecosystem Tools + + [Overview](/ecosystem-tool-user-guide.md) + + [Use Cases](/ecosystem-tool-user-case.md) + + [Download](/download-ecosystem-tools.md) + + Backup & Restore (BR) + + [BR FAQ](/br/backup-and-restore-faq.md) + + [Use BR Tool](/br/backup-and-restore-tool.md) + + [BR Use Cases](/br/backup-and-restore-use-cases.md) + + TiDB Binlog + + [Overview](/tidb-binlog/tidb-binlog-overview.md) + + [Deploy](/tidb-binlog/deploy-tidb-binlog.md) + + [Maintain](/tidb-binlog/maintain-tidb-binlog-cluster.md) + + [Configure](/tidb-binlog/tidb-binlog-configuration-file.md) + + [Pump](/tidb-binlog/tidb-binlog-configuration-file.md#pump) + + [Drainer](/tidb-binlog/tidb-binlog-configuration-file.md#drainer) + + [Upgrade](/tidb-binlog/upgrade-tidb-binlog.md) + + [Monitor](/tidb-binlog/monitor-tidb-binlog-cluster.md) + + [Reparo](/tidb-binlog/tidb-binlog-reparo.md) + + [binlogctl](/tidb-binlog/binlog-control.md) + + [Binlog Slave Client](/tidb-binlog/binlog-slave-client.md) + + [TiDB Binlog Relay Log](/tidb-binlog/tidb-binlog-relay-log.md) + + [Bidirectional Replication Between TiDB Clusters](/tidb-binlog/bidirectional-replication-between-tidb-clusters.md) + + [Glossary](/tidb-binlog/tidb-binlog-glossary.md) + + Troubleshoot + + [Troubleshoot](/tidb-binlog/troubleshoot-tidb-binlog.md) + + [Handle Errors](/tidb-binlog/handle-tidb-binlog-errors.md) + + [FAQ](/tidb-binlog/tidb-binlog-faq.md) + + TiDB Lightning + + [Overview](/tidb-lightning/tidb-lightning-overview.md) + + [Tutorial](/get-started-with-tidb-lightning.md) + + [Deploy](/tidb-lightning/deploy-tidb-lightning.md) + + [Configure](/tidb-lightning/tidb-lightning-configuration.md) + + Key Features + + [Checkpoints](/tidb-lightning/tidb-lightning-checkpoints.md) + + [Table Filter](/tidb-lightning/tidb-lightning-table-filter.md) + + [CSV Support](/tidb-lightning/migrate-from-csv-using-tidb-lightning.md) + + [TiDB-backend](/tidb-lightning/tidb-lightning-tidb-backend.md) + + [Web Interface](/tidb-lightning/tidb-lightning-web-interface.md) + + [Monitor](/tidb-lightning/monitor-tidb-lightning.md) + + [Troubleshoot](/troubleshoot-tidb-lightning.md) + + [FAQ](/tidb-lightning/tidb-lightning-faq.md) + + [Glossary](/tidb-lightning/tidb-lightning-glossary.md) + + [TiCDC](/ticdc/ticdc-overview.md) + + sync-diff-inspector + + [Overview](/sync-diff-inspector/sync-diff-inspector-overview.md) + + [Data Check for Tables with Different Schema/Table Names](/sync-diff-inspector/route-diff.md) + + [Data Check in Sharding Scenarios](/sync-diff-inspector/shard-diff.md) + + [Data Check for TiDB Upstream/Downstream Clusters](/sync-diff-inspector/upstream-downstream-diff.md) + + [Loader](/loader-overview.md) + + [Mydumper](/mydumper-overview.md) + + [Syncer](/syncer-overview.md) + + TiSpark + + [Quick Start](/get-started-with-tispark.md) + + [User Guide](/tispark-overview.md) +>>>>>>> 23e53ef... tutorial: add doc for store limit (#2947) + Reference + SQL - [MySQL Compatibility](/mysql-compatibility.md) diff --git a/configure-store-limit.md b/configure-store-limit.md new file mode 100644 index 0000000000000..12facdd4d1fd0 --- /dev/null +++ b/configure-store-limit.md @@ -0,0 +1,69 @@ +--- +title: Store Limit +summary: Learn the feature of Store Limit. +category: tutorials +--- + +# Store Limit + +Store Limit is a feature of PD, introduced in TiDB 3.0. It is designed to control the scheduling speed in a finer manner for better performance in different scenarios. + +## Implementation principles + +PD performs scheduling at the unit of operator. An operator might contain several scheduling operations. For example: + +``` +"replace-down-replica {mv peer: store [2] to [3]} (kind:region,replica, region:10(4,5), createAt:2020-05-18 06:40:25.775636418 +0000 UTC m=+2168762.679540369, startAt:2020-05-18 06:40:25.775684648 +0000 UTC m=+2168762.679588599, currentStep:0, steps:[add learner peer 20 on store 3, promote learner peer 20 on store 3 to voter, remove peer on store 2])" +``` + +In this above example, the `replace-down-replica` operator contains the following specific operations: + +1. Add a learner peer with the ID `20` to `store 3`. +2. Promote the learner peer with the ID `20` on `store 3` to a voter. +3. Delete the peer on `store 2`. + +Store Limit achieves the store-level speed limit by maintaining a mapping from store IDs to token buckets in memory. The different operations here correspond to different token buckets. Currently, Store Limit only supports limiting the speed of two operations: adding learners/peers and deleting peers. That is, each store has two types of token buckets. + +Every time an operator is generated, it checks whether enough tokens exist in the token buckets for its operations. If yes, the operator is added to the scheduling queue, and the corresponding token is taken from the token bucket. Otherwise, the operator is abandoned. Because the token bucket replenishes tokens at a fixed rate, the speed limit is thus achieved. + +Store Limit is different from other limit-related parameters in PD (such as `region-schedule-limit` and `leader-schedule-limit`) in that it mainly limits the consuming speed of operators, while other parameters limits the generating speed of operators. Before introducing the Store Limit feature, the speed limit of scheduling is mostly at the global scope. Therefore, even if the global speed is limited, it is still possible that the scheduling operations are concentrated on some stores, affecting the performance of the cluster. By limiting the speed at a finer level, Store Limit can better control the scheduling behavior. + +## Usage + +The parameters of Store Limit can be configured using `pd-ctl`. + +### View setting of the current store + +To view the limit setting of the current store, run the following commands: + +{{< copyable "shell-regular" >}} + +```bash +store limit // Shows the speed limit of adding and deleting peers in all stores. +store limit add-peer // Shows the speed limit of adding peers in all stores. +store limit remove-peer // Shows the speed limit of deleting peers in all stores. +``` + +### Set limit for all stores + +To set the speed limit for all stores, run the following commands: + +{{< copyable "shell-regular" >}} + +```bash +store limit all 5 // All stores can at most add and delete 5 peers per minute. +store limit all 5 add-peer // All stores can at most add 5 peers per minute. +store limit all 5 remove-peer // All stores can at most delete 5 peers per minute. +``` + +### Set limit for a single store + +To set the speed limit for a single store, run the following commands: + +{{< copyable "shell-regular" >}} + +```bash +store limit 1 5 // store 1 can at most add and delete 5 peers per minute. +store limit 1 5 add-peer // store 1 can at most add 5 peers per minute. +store limit 1 5 remove-peer // store 1 can at most delete 5 peers per minute. +``` From c736181a624056e65f56e33ad6603300040486f0 Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 2 Jul 2020 16:56:58 +0800 Subject: [PATCH 2/3] resolve conflict --- TOC.md | 123 --------------------------------------------------------- 1 file changed, 123 deletions(-) diff --git a/TOC.md b/TOC.md index 3318eca7e90ab..ff7cf41eaeda9 100644 --- a/TOC.md +++ b/TOC.md @@ -91,7 +91,6 @@ - [Identify Slow Queries](/identify-slow-queries.md) - [Identify Expensive Queries](/identify-expensive-queries.md) + Scale -<<<<<<< HEAD - [Scale using Ansible](/scale-tidb-using-ansible.md) + Upgrade - [Upgrade to TiDB 3.1](/upgrade-tidb-using-ansible.md) @@ -99,128 +98,6 @@ - [TiDB Troubleshooting Map](/tidb-troubleshooting-map.md) - [Troubleshoot Cluster Setup](/troubleshoot-tidb-cluster.md) - [Troubleshoot TiDB Lightning](/troubleshoot-tidb-lightning.md) -======= - + [Use TiUP (Recommended)](/scale-tidb-using-tiup.md) - + [Use TiDB Ansible](/scale-tidb-using-ansible.md) - + [Use TiDB Operator](https://docs.pingcap.com/tidb-in-kubernetes/v1.1/scale-a-tidb-cluster) - + Backup and Restore - + [Use Mydumper and TiDB Lightning](/backup-and-restore-using-mydumper-lightning.md) - + [Use Dumpling for Export or Backup](/export-or-backup-using-dumpling.md) - + Use BR Tool - + [Use BR Tool](/br/backup-and-restore-tool.md) - + [BR Use Cases](/br/backup-and-restore-use-cases.md) - + [BR storages](/br/backup-and-restore-storages.md) - + [Configure Time Zone](/configure-time-zone.md) - + [Daily Checklist](/daily-check.md) - + [Manage TiCDC Cluster and Replication Tasks](/ticdc/manage-ticdc.md) - + [Maintain TiFlash](/tiflash/maintain-tiflash.md) - + [Maintain TiDB Using TiUP](/maintain-tidb-using-tiup.md) - + [Maintain TiDB Using Ansible](/maintain-tidb-using-ansible.md) -+ Monitor and Alert - + [Monitoring Framework Overview](/tidb-monitoring-framework.md) - + [Monitoring API](/tidb-monitoring-api.md) - + [Deploy Monitoring Services](/deploy-monitoring-services.md) - + [TiDB Cluster Alert Rules](/alert-rules.md) - + [TiFlash Alert Rules](/tiflash/tiflash-alert-rules.md) -+ Troubleshoot - + [Identify Slow Queries](/identify-slow-queries.md) - + [SQL Diagnostics](/system-tables/system-table-sql-diagnostics.md) - + [Identify Expensive Queries](/identify-expensive-queries.md) - + [Statement Summary Tables](/statement-summary-tables.md) - + [Troubleshoot Cluster Setup](/troubleshoot-tidb-cluster.md) - + [TiDB Troubleshooting Map](/tidb-troubleshooting-map.md) - + [Troubleshoot TiCDC](/ticdc/troubleshoot-ticdc.md) - + [Troubleshoot TiFlash](/tiflash/troubleshoot-tiflash.md) -+ Performance Tuning - + Software Tuning - + Configuration - + [Tune TiDB Memory](/configure-memory-usage.md) - + [Tune TiKV Threads](/tune-tikv-thread-performance.md) - + [Tune TiKV Memory](/tune-tikv-memory-performance.md) - + [TiKV Follower Read](/follower-read.md) - + [TiFlash Tuning](/tiflash/tune-tiflash-performance.md) - + [Coprocessor Cache](/coprocessor-cache.md) - + SQL Tuning - + [SQL Tuning with `EXPLAIN`](/query-execution-plan.md) - + SQL Optimization - + [SQL Optimization Process](/sql-optimization-concepts.md) - + Logic Optimization - + [Join Reorder](/join-reorder.md) - + Physical Optimization - + [Statistics](/statistics.md) - + Control Execution Plan - + [Optimizer Hints](/optimizer-hints.md) - + [SQL Plan Management](/sql-plan-management.md) - + [Access Tables Using `IndexMerge`](/index-merge.md) -+ Tutorials - + [Multiple Data Centers in One City Deployment](/multi-data-centers-in-one-city-deployment.md) - + [Three Data Centers in Two Cities Deployment](/three-data-centers-in-two-cities-deployment.md) - + Best Practices - + [Use TiDB](/tidb-best-practices.md) - + [Java Application Development](/best-practices/java-app-best-practices.md) - + [Use HAProxy](/best-practices/haproxy-best-practices.md) - + [Highly Concurrent Write](/best-practices/high-concurrency-best-practices.md) - + [Grafana Monitoring](/best-practices/grafana-monitor-best-practices.md) - + [PD Scheduling](/best-practices/pd-scheduling-best-practices.md) - + [TiKV Performance Tuning with Massive Regions](/best-practices/massive-regions-best-practices.md) - + [Use Placement Rules](/configure-placement-rules.md) - + [Use Load Base Split](/configure-load-base-split.md) - + [Use Store Limit](/configure-store-limit.md) -+ TiDB Ecosystem Tools - + [Overview](/ecosystem-tool-user-guide.md) - + [Use Cases](/ecosystem-tool-user-case.md) - + [Download](/download-ecosystem-tools.md) - + Backup & Restore (BR) - + [BR FAQ](/br/backup-and-restore-faq.md) - + [Use BR Tool](/br/backup-and-restore-tool.md) - + [BR Use Cases](/br/backup-and-restore-use-cases.md) - + TiDB Binlog - + [Overview](/tidb-binlog/tidb-binlog-overview.md) - + [Deploy](/tidb-binlog/deploy-tidb-binlog.md) - + [Maintain](/tidb-binlog/maintain-tidb-binlog-cluster.md) - + [Configure](/tidb-binlog/tidb-binlog-configuration-file.md) - + [Pump](/tidb-binlog/tidb-binlog-configuration-file.md#pump) - + [Drainer](/tidb-binlog/tidb-binlog-configuration-file.md#drainer) - + [Upgrade](/tidb-binlog/upgrade-tidb-binlog.md) - + [Monitor](/tidb-binlog/monitor-tidb-binlog-cluster.md) - + [Reparo](/tidb-binlog/tidb-binlog-reparo.md) - + [binlogctl](/tidb-binlog/binlog-control.md) - + [Binlog Slave Client](/tidb-binlog/binlog-slave-client.md) - + [TiDB Binlog Relay Log](/tidb-binlog/tidb-binlog-relay-log.md) - + [Bidirectional Replication Between TiDB Clusters](/tidb-binlog/bidirectional-replication-between-tidb-clusters.md) - + [Glossary](/tidb-binlog/tidb-binlog-glossary.md) - + Troubleshoot - + [Troubleshoot](/tidb-binlog/troubleshoot-tidb-binlog.md) - + [Handle Errors](/tidb-binlog/handle-tidb-binlog-errors.md) - + [FAQ](/tidb-binlog/tidb-binlog-faq.md) - + TiDB Lightning - + [Overview](/tidb-lightning/tidb-lightning-overview.md) - + [Tutorial](/get-started-with-tidb-lightning.md) - + [Deploy](/tidb-lightning/deploy-tidb-lightning.md) - + [Configure](/tidb-lightning/tidb-lightning-configuration.md) - + Key Features - + [Checkpoints](/tidb-lightning/tidb-lightning-checkpoints.md) - + [Table Filter](/tidb-lightning/tidb-lightning-table-filter.md) - + [CSV Support](/tidb-lightning/migrate-from-csv-using-tidb-lightning.md) - + [TiDB-backend](/tidb-lightning/tidb-lightning-tidb-backend.md) - + [Web Interface](/tidb-lightning/tidb-lightning-web-interface.md) - + [Monitor](/tidb-lightning/monitor-tidb-lightning.md) - + [Troubleshoot](/troubleshoot-tidb-lightning.md) - + [FAQ](/tidb-lightning/tidb-lightning-faq.md) - + [Glossary](/tidb-lightning/tidb-lightning-glossary.md) - + [TiCDC](/ticdc/ticdc-overview.md) - + sync-diff-inspector - + [Overview](/sync-diff-inspector/sync-diff-inspector-overview.md) - + [Data Check for Tables with Different Schema/Table Names](/sync-diff-inspector/route-diff.md) - + [Data Check in Sharding Scenarios](/sync-diff-inspector/shard-diff.md) - + [Data Check for TiDB Upstream/Downstream Clusters](/sync-diff-inspector/upstream-downstream-diff.md) - + [Loader](/loader-overview.md) - + [Mydumper](/mydumper-overview.md) - + [Syncer](/syncer-overview.md) - + TiSpark - + [Quick Start](/get-started-with-tispark.md) - + [User Guide](/tispark-overview.md) ->>>>>>> 23e53ef... tutorial: add doc for store limit (#2947) + Reference + SQL - [MySQL Compatibility](/mysql-compatibility.md) From 65c3f95cb4a7f4319d9c780e481488940e78f109 Mon Sep 17 00:00:00 2001 From: Ran Date: Fri, 3 Jul 2020 12:06:13 +0800 Subject: [PATCH 3/3] update 3.1 specific changes --- TOC.md | 1 + configure-store-limit.md | 30 ++++++++++++++++++++---------- 2 files changed, 21 insertions(+), 10 deletions(-) diff --git a/TOC.md b/TOC.md index ff7cf41eaeda9..52bcca4b1b050 100644 --- a/TOC.md +++ b/TOC.md @@ -64,6 +64,7 @@ - [Time Zone](/configure-time-zone.md) - [Memory Control](/configure-memory-usage.md) - [Placement Rules](/configure-placement-rules.md) + - [Store Limit](/configure-store-limit.md) + Secure + Transport Layer Security (TLS) - [Enable TLS For MySQL Clients](/encrypted-connections-with-tls-protocols.md) diff --git a/configure-store-limit.md b/configure-store-limit.md index 12facdd4d1fd0..731cafef923cd 100644 --- a/configure-store-limit.md +++ b/configure-store-limit.md @@ -1,7 +1,7 @@ --- title: Store Limit summary: Learn the feature of Store Limit. -category: tutorials +category: how-to --- # Store Limit @@ -39,9 +39,9 @@ To view the limit setting of the current store, run the following commands: {{< copyable "shell-regular" >}} ```bash -store limit // Shows the speed limit of adding and deleting peers in all stores. -store limit add-peer // Shows the speed limit of adding peers in all stores. -store limit remove-peer // Shows the speed limit of deleting peers in all stores. +store limit // Shows the speed limit of adding learners/peers in all stores (if a specific type is not set, this command shows the speed of adding learners/peers). +store limit region-add // Shows the speed limit of adding learners/peers in all stores. +store limit region-remove // Shows the speed limit of deleting learners/peers in all stores. ``` ### Set limit for all stores @@ -51,9 +51,9 @@ To set the speed limit for all stores, run the following commands: {{< copyable "shell-regular" >}} ```bash -store limit all 5 // All stores can at most add and delete 5 peers per minute. -store limit all 5 add-peer // All stores can at most add 5 peers per minute. -store limit all 5 remove-peer // All stores can at most delete 5 peers per minute. +store limit all 5 // All stores can at most add 5 learns/peers per minute (if a specific type is not set, this command sets the speed of adding learners/peers). +store limit all 5 region-add // All stores can at most add 5 learns/peers per minute. +store limit all 5 region-remove // All stores can at most delete 5 learns/peers per minute. ``` ### Set limit for a single store @@ -63,7 +63,17 @@ To set the speed limit for a single store, run the following commands: {{< copyable "shell-regular" >}} ```bash -store limit 1 5 // store 1 can at most add and delete 5 peers per minute. -store limit 1 5 add-peer // store 1 can at most add 5 peers per minute. -store limit 1 5 remove-peer // store 1 can at most delete 5 peers per minute. +store limit 1 5 // store 1 can at most add 5 learners/peers per minute (if a specific type is not set, this command sets the speed of adding learners/peers). +store limit 1 5 region-add // store 1 can at most add 5 learners/peers per minute. +store limit 1 5 region-remove // store 1 can at most delete 5 learners/peers per minute. +``` + +### Persist store limit modification + +Because the store limit is a mapping in the memory, the above modification is reset after the leader is switched or PD is restarted. If you want to persist the modification, run the following command: + +{{< copyable "shell-regular" >}} + +```bash +config set store-balance-rate 20 // All stores can at most add 20 learners/peers or delete 20 peers per minute. ```