how-to: add placement rules guide #2246

rleungx · 2020-02-18T04:48:23Z

What is changed, added or deleted?

This PR adds the Placement Rules related document.

What is the related PR or file link(s)?

N/A

Which TiDB version(s) does your changes apply to?

3.1/dev

disksing · 2020-02-18T05:59:36Z

dev/how-to/configure/placement-rules.md

+{
+  "group_id": "tiflash",
+  "id": "learner-replica-table-03",
+  "start_key": "748000000000000003", // start key of table 3


这里的key要encode下不

我从之前那个文章里 copy 过来的

之前那个简单化过，主要是为了好理解。。

全部改成 6E 和 6D 了

不行呀，都改成 6E 6D 的话，跟文字说明的“某个table"不匹配

yikeke · 2020-02-23T04:50:04Z

After this PR merges, sre-bot will automatically cherry-pick this PR to the release-3.1 branch (the needs-cherry-pick-xxx label will trigger the behavior). FYI. @rleungx @disksing

disksing · 2020-02-24T02:58:37Z

how-to/configure/placement-rules.md

+  "count": 2,
+  "label_constraints": [
+    {"key": "zone", "op": "in", "values": ["bj1", "bj2"]},
+    {"key", "disk", "op": "notIn", "values": ["hdd"]}


Suggested change

{"key", "disk", "op": "notIn", "values": ["hdd"]}

{"key": "disk", "op": "notIn", "values": ["hdd"]}

how-to/configure/placement-rules.md

TomShawn · 2020-02-24T03:39:04Z

how-to/configure/placement-rules.md

+>> config set enable-placement-rules true
+```
+
+开启后会根据之前的副本配置默认生成一条规则，下面的生成的规则对应于之前版本的 `max-replicas=3,location-labels=[zone,rack,host]` 配置：


之前版本具体是哪个版本之前的呢？

Suggested change

开启后会根据之前的副本配置默认生成一条规则，下面的生成的规则对应于之前版本的 `max-replicas=3,location-labels=[zone,rack,host]` 配置：

开启后会根据之前的副本配置默认生成一条规则。下面的生成的规则对应之前版本的 `max-replicas=3,location-labels=[zone,rack,host]` 配置：

@rleungx 请明确一下之前版本。

how-to/configure/placement-rules.md

TomShawn · 2020-02-24T03:42:53Z

how-to/configure/placement-rules.md

+
+### group_id
+
+标识规则的创建者，不同创建者之间互不影响


这部分改为列表形式是否更好？每一个字段为一个列表项目。+ group_id：标识规则的创建者。不同创建者之间互不影响。

Suggested change

标识规则的创建者，不同创建者之间互不影响

标识规则的创建者。不同创建者之间互不影响。

TomShawn · 2020-02-24T06:14:57Z

how-to/configure/placement-rules.md

+
+*注意* 由于不同的 Rule 调度时单独计算，只能保证 default 规则的 3 副本按照 `location-labels` 隔离，meta 规则的 2 副本也隔离，而不能防止 default 规则的副本和 meta 规则的副本被调度在一起。
+
+可以通过 default 规则的范围拆分成 2 个，key range 分别为 `["", "6D")` 和 `["6E", "")`，然后把 meta 规则的 `count` 调整为 5，让 meta 数据只适配一条 5 副本的规则，实现 `location-label` 的隔离。


Suggested change

可以通过 default 规则的范围拆分成 2 个，key range 分别为 `["", "6D")` 和 `["6E", "")`，然后把 meta 规则的 `count` 调整为 5，让 meta 数据只适配一条 5 副本的规则，实现 `location-label` 的隔离。

可以通过 default 规则的范围拆分成 2 个 key range，分别为 `["", "6D")` 和 `["6E", "")`，然后把 meta 规则的 `count` 调整为 `5`，让 meta 数据只适配一条 5 副本的规则，实现 `location-label` 的隔离。

TomShawn · 2020-02-24T06:18:01Z

how-to/configure/placement-rules.md

+}
+```
+
+### 场景二：5 副本按 2-2-1 的比例放置在 3 个数据中心，且第 3 个中心不调度 Leader


Suggested change

### 场景二：5 副本按 2-2-1 的比例放置在 3 个数据中心，且第 3 个中心不调度 Leader

### 场景二

场景描述：5 副本按 2:2:1 的比例放置在 3 个数据中心，且第 3 个中心不调度 Leader。

TomShawn · 2020-02-24T06:24:41Z

how-to/configure/placement-rules.md

+
+每个数据中心对应一条规则，通过 `count` 设置副本数，通过 `role` 标识副本角色（是否需要 `leader`）。由于每个规则对应的副本都使用 `label_constraints` 约束在单个数据中心内了，`location_labels` 只需配置 `rack`，`host` 两级。
+
+### 场景三：为某个 table 添加 2 个 TiFlash 副本


Suggested change

### 场景三：为某个 table 添加 2 个 TiFlash 副本

### 场景三

场景描述：为某个 table 添加 2 个 TiFlash 副本。

TomShawn · 2020-02-24T06:25:39Z

how-to/configure/placement-rules.md

+}
+```
+
+该规则指定添加的角色为 `learner`，并且通过 `label_constraints` 被用于筛选 `engine` 的类型，指定 `engine` 类型为 `tiflash`。


Suggested change

该规则指定添加的角色为 `learner`，并且通过 `label_constraints` 被用于筛选 `engine` 的类型，指定 `engine` 类型为 `tiflash`。

该规则指定添加的角色为 `learner`，并且通过 `label_constraints` 筛选 `engine` 的类型，指定 `engine` 类型为 `tiflash`。

TomShawn · 2020-02-24T06:26:21Z

how-to/configure/placement-rules.md

+
+该规则指定添加的角色为 `learner`，并且通过 `label_constraints` 被用于筛选 `engine` 的类型，指定 `engine` 类型为 `tiflash`。
+
+### 场景四：为某个 table 在有高性能磁盘的北京节点添加 2 个 Follower 副本


Suggested change

### 场景四：为某个 table 在有高性能磁盘的北京节点添加 2 个 Follower 副本

### 场景四

场景描述：为某张表在有高性能磁盘的北京节点添加 2 个 Follower 副本。

rleungx · 2020-02-25T11:11:24Z

PTAL @disksing @TomShawn

TomShawn · 2020-03-05T03:12:41Z

LGTM
@disksing PTAL again, thanks!

lilin90 · 2020-03-06T08:01:17Z

how-to/configure/placement-rules.md

+{{< copyable "shell-regular" >}}
+
+```bash
+>> config set enable-placement-rules true


Suggested change

>> config set enable-placement-rules true

config set enable-placement-rules true

IANTHEREAL · 2020-03-11T14:54:23Z

how-to/configure/placement-rules.md

+
+## 开启 Placement Rules
+
+可通过 pd-ctl 设置 `enable-placement-rules` 为 `true` 开启 Placement Rules。


pd-ctl 使用文档 link 一下？

IANTHEREAL · 2020-03-11T15:51:34Z

how-to/configure/placement-rules.md

+
+{{< copyable "" >}}
+
+```json


我运行后我没看到这条规则，怎么看？

使用 pd-ctl 的 config show 没有看到相关配置

使用 curl 通过 GET 命令才看到

IANTHEREAL · 2020-03-11T16:01:34Z

how-to/configure/placement-rules.md

+
+以下部分介绍添加 Placement Rules 的典型场景。
+
+### 场景一


典型场景能不能把之行的命令直接贴一下，要不我还要自己转换 json 。。。

disksing · 2020-03-12T06:52:17Z

这个comment还没有处理

IANTHEREAL · 2020-03-12T07:06:15Z

how-to/configure/placement-rules.md

+  "start_key": "6D",
+  "end_key": "6E",
+  "role": "voter",
+  "count": "2",


[root@4 bin]# curl -X POST -H "Content-Type: application/json" --data @data.json http://172.16.5.204:2379/pd/api/v1/config/rule { "code": "input", "msg": "json: cannot unmarshal string into Go struct field Rule.count of type int", "data": { "Value": "string", "Type": {}, "Offset": 101, "Struct": "Rule", "Field": "count" } }

IANTHEREAL · 2020-03-12T07:13:00Z

how-to/configure/placement-rules.md

+{
+  "group_id": "pd",
+  "id": "meta",
+  "start_key": "6D",


"start key should be encoded in table mode: insufficient bytes to decode value"

Signed-off-by: Ryan Leung <rleungx@gmail.com>

This reverts commit b2208b9, reversing changes made to e50d22e.

This reverts commit 922e368.

Signed-off-by: Ryan Leung <rleungx@gmail.com>

lilin90 · 2020-03-12T08:51:33Z

@rleungx It seems that some comments haven't been resolved yet.

TomShawn · 2020-03-17T01:55:42Z

@GregoryIan @disksing PTAL again, thanks!

IANTHEREAL · 2020-03-17T08:53:01Z

还有两个建议

提供方法怎么便捷的查询到一些用户常用的 start key & end key，比如某 table
案例太单薄，操作有风险，需要提供更详细的学习文档，或者更加惊醒的提示。可能会因为误操作 placement rule 而导致的数据丢失

rleungx · 2020-03-19T11:46:37Z

@disksing PTAL

disksing · 2020-03-23T04:44:56Z

No body is satisfied with this document. Closing it, I'll write it later.

rleungx added dev priority/P1 The issue has P1 priority. labels Feb 18, 2020

rleungx requested review from TomShawn, disksing and nolouch February 18, 2020 04:48

TomShawn added v4.0 This PR/issue applies to TiDB v4.0. translation/doing This PR’s assignee is translating this PR. labels Feb 18, 2020

disksing reviewed Feb 18, 2020

View reviewed changes

yikeke added the needs-cherry-pick-3.1 label Feb 23, 2020

disksing reviewed Feb 24, 2020

View reviewed changes

TomShawn reviewed Feb 24, 2020

View reviewed changes

yikeke assigned ran-huang Mar 4, 2020

lilin90 reviewed Mar 6, 2020

View reviewed changes

IANTHEREAL reviewed Mar 11, 2020

View reviewed changes

IANTHEREAL reviewed Mar 12, 2020

View reviewed changes

rleungx and others added 8 commits March 12, 2020 15:13

add placement rules guide

22c6a68

Signed-off-by: Ryan Leung <rleungx@gmail.com>

Revert "Merge branch 'master' into pr/2246"

2c1dbdc

This reverts commit b2208b9, reversing changes made to e50d22e.

Revert "Revert "Merge branch 'master' into pr/2246""

4ed25a5

This reverts commit 922e368.

make the same changes as the first commit

6d3e97a

address comments

8d83e0e

Signed-off-by: Ryan Leung <rleungx@gmail.com>

address comments

79a60ee

Signed-off-by: Ryan Leung <rleungx@gmail.com>

address comments

30d10e0

Signed-off-by: Ryan Leung <rleungx@gmail.com>

address comments

42674e4

Signed-off-by: Ryan Leung <rleungx@gmail.com>

rleungx force-pushed the placement-rules branch from 110ccaf to 42674e4 Compare March 12, 2020 08:39

disksing approved these changes Mar 23, 2020

View reviewed changes

disksing closed this Mar 23, 2020

disksing mentioned this pull request Mar 31, 2020

how-to: add placement rule guide #2587

Merged

4 tasks

sre-bot mentioned this pull request Apr 2, 2020

how-to: add placement rule guide (#2587) #2610

Merged

4 tasks

ran-huang removed their assignment Apr 16, 2020

yikeke removed the translation/doing This PR’s assignee is translating this PR. label May 6, 2020

	{"key", "disk", "op": "notIn", "values": ["hdd"]}
	{"key": "disk", "op": "notIn", "values": ["hdd"]}

	开启后会根据之前的副本配置默认生成一条规则，下面的生成的规则对应于之前版本的 `max-replicas=3,location-labels=[zone,rack,host]` 配置：
	开启后会根据之前的副本配置默认生成一条规则。下面的生成的规则对应之前版本的 `max-replicas=3,location-labels=[zone,rack,host]` 配置：


		### group_id

		标识规则的创建者，不同创建者之间互不影响

	标识规则的创建者，不同创建者之间互不影响
	标识规则的创建者。不同创建者之间互不影响。


		注意由于不同的 Rule 调度时单独计算，只能保证 default 规则的 3 副本按照 `location-labels` 隔离，meta 规则的 2 副本也隔离，而不能防止 default 规则的副本和 meta 规则的副本被调度在一起。

		可以通过 default 规则的范围拆分成 2 个，key range 分别为 `["", "6D")` 和 `["6E", "")`，然后把 meta 规则的 `count` 调整为 5，让 meta 数据只适配一条 5 副本的规则，实现 `location-label` 的隔离。

	可以通过 default 规则的范围拆分成 2 个，key range 分别为 `["", "6D")` 和 `["6E", "")`，然后把 meta 规则的 `count` 调整为 5，让 meta 数据只适配一条 5 副本的规则，实现 `location-label` 的隔离。
	可以通过 default 规则的范围拆分成 2 个 key range，分别为 `["", "6D")` 和 `["6E", "")`，然后把 meta 规则的 `count` 调整为 `5`，让 meta 数据只适配一条 5 副本的规则，实现 `location-label` 的隔离。


		每个数据中心对应一条规则，通过 `count` 设置副本数，通过 `role` 标识副本角色（是否需要 `leader`）。由于每个规则对应的副本都使用 `label_constraints` 约束在单个数据中心内了，`location_labels` 只需配置 `rack`，`host` 两级。

		### 场景三：为某个 table 添加 2 个 TiFlash 副本

	该规则指定添加的角色为 `learner`，并且通过 `label_constraints` 被用于筛选 `engine` 的类型，指定 `engine` 类型为 `tiflash`。
	该规则指定添加的角色为 `learner`，并且通过 `label_constraints` 筛选 `engine` 的类型，指定 `engine` 类型为 `tiflash`。


		该规则指定添加的角色为 `learner`，并且通过 `label_constraints` 被用于筛选 `engine` 的类型，指定 `engine` 类型为 `tiflash`。

		### 场景四：为某个 table 在有高性能磁盘的北京节点添加 2 个 Follower 副本

	>> config set enable-placement-rules true
	config set enable-placement-rules true


		{{< copyable "" >}}

		```json


		## 开启 Placement Rules

		可通过 pd-ctl 设置 `enable-placement-rules` 为 `true` 开启 Placement Rules。


		以下部分介绍添加 Placement Rules 的典型场景。

		### 场景一

how-to: add placement rules guide #2246

how-to: add placement rules guide #2246

Uh oh!

Conversation

rleungx commented Feb 18, 2020

What is changed, added or deleted?

What is the related PR or file link(s)?

Which TiDB version(s) does your changes apply to?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yikeke commented Feb 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rleungx commented Feb 25, 2020

Uh oh!

TomShawn commented Mar 5, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

disksing commented Mar 12, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lilin90 commented Mar 12, 2020

Uh oh!

TomShawn commented Mar 17, 2020

Uh oh!

IANTHEREAL commented Mar 17, 2020

Uh oh!

rleungx commented Mar 19, 2020

Uh oh!

disksing commented Mar 23, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

yikeke commented Feb 23, 2020 •

edited

Loading