From 8c1c525a04031bf68a383cb9bde3316e60ed7be4 Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Tue, 16 Jun 2020 15:28:58 +0200 Subject: [PATCH 01/21] add replace yb-master docs --- .../troubleshoot/cluster/replace_master.md | 60 +++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 docs/content/latest/troubleshoot/cluster/replace_master.md diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md new file mode 100644 index 000000000000..1758f810af3c --- /dev/null +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -0,0 +1,60 @@ +--- +title: Replace failed master +linkTitle: Replace failed master +description: Replace failed master +aliases: + - /troubleshoot/cluster/replace-master/ + - /latest/troubleshoot/cluster/replace-master/ +menu: + latest: + parent: troubleshoot-cluster + weight: 831 +isTocNested: true +showAsideToc: true +--- + +Suppose you have a cluster running and now a yb-master has failed and needs to be replaced. The procedure to follow in this +scenario is: + +{{< note title="Note" >}} +Assumptions: +- An initial set of masters M1, M2, M3 +- We are removing M1 +- We want to add M4 +- We'll be using the default master RPC port of 7100 +{{< /note >}} + + + +## Start new master and add to cluster +Start `M4` with `--master_addresses= (empty string)`, this will put the master in what we refer to as `ShellMode`. + +Add the new master into the quorum using: +```bash +./yb-admin -master_addresses M1:7100,M2:7100,M3:7100 change_master_config ADD_SERVER M4 7100 +``` + +## Remove old master +Remove the old master from the quorum using: +```bash +./yb-admin -master_addresses M1:7100,M2:7100,M3:7100,M4:7100 change_master_config REMOVE_SERVER M1 7100 +``` + +Note: the master addresses needs to include all 4 masters, in case the new one is suddenly the master leader!) + +## Validate cluster + +Validate that your set of masters is now `M2`, `M3` and `M4` using: +```bash +yb-admin -master_addresses M2:7100,M3:7100,M4:7100 list_all_masters +``` + +Until [#1542](https://github.com/yugabyte/yugabyte-db/issues/1542) is implemented, the TS will by default only know of +whatever masters are encoded in the `--tserver_master_addrs` flag that they are started with. + +If any one of those masters is still part of the active quorum, then they will propagate the new master quorum over via heartbeats. +If, however, none of the current masters are present in the TS flag, then the TS will not be able to join the cluster! + +So it is important to make sure to update `--tserver_master_addrs` on every TS to the new set of master addresses, `M2:7100,M3:7100,M4:7100`! + +If the master you wish to replace is already dead (eg: VM was terminated), you might want to first do the `REMOVE` step, then do the `ADD` step. From 12f8a53460d436a4af7d61e0dd4af47a00308f89 Mon Sep 17 00:00:00 2001 From: ddorian Date: Thu, 25 Jun 2020 02:26:00 -0700 Subject: [PATCH 02/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 1758f810af3c..8b0212d6ef37 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -7,6 +7,7 @@ aliases: - /latest/troubleshoot/cluster/replace-master/ menu: latest: + identifier: replace-failed-master parent: troubleshoot-cluster weight: 831 isTocNested: true From 45fe03232095228c011b635f16e0387923c5e1c0 Mon Sep 17 00:00:00 2001 From: ddorian Date: Thu, 25 Jun 2020 02:26:13 -0700 Subject: [PATCH 03/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 8b0212d6ef37..e6082685de34 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -14,8 +14,7 @@ isTocNested: true showAsideToc: true --- -Suppose you have a cluster running and now a yb-master has failed and needs to be replaced. The procedure to follow in this -scenario is: +To replace a failed YB-Master server in a YugabyteDB cluster, follow these steps: {{< note title="Note" >}} Assumptions: From 69432c5cce210fff2d27ebc11b1afcfff035b88b Mon Sep 17 00:00:00 2001 From: ddorian Date: Thu, 25 Jun 2020 02:32:27 -0700 Subject: [PATCH 04/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- .../latest/troubleshoot/cluster/replace_master.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index e6082685de34..c65b7688943d 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -16,13 +16,12 @@ showAsideToc: true To replace a failed YB-Master server in a YugabyteDB cluster, follow these steps: -{{< note title="Note" >}} -Assumptions: -- An initial set of masters M1, M2, M3 -- We are removing M1 -- We want to add M4 -- We'll be using the default master RPC port of 7100 -{{< /note >}} +For the steps below, the examples use the following scenario: + +- The cluster includes three `yb-master` servers: `M1`, `M2`, `M3`. +- YB-Master server `M1` failed and needs to be replaced. +- A new YB-Master server (`M4`) will replace `M1`. +- The default master RPC port is `7100` From 9aacb832db232fb89ff9363ba166cfcdfa8af454 Mon Sep 17 00:00:00 2001 From: ddorian Date: Thu, 25 Jun 2020 02:32:47 -0700 Subject: [PATCH 05/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index c65b7688943d..886e956a19a3 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -1,7 +1,8 @@ --- -title: Replace failed master -linkTitle: Replace failed master -description: Replace failed master +title: Replace a failed YB-Master +headerTitle: Replace a failed YB-Master +linkTitle: Replace a failed YB-Master +description: Steps to replace a failed YB-Master in a YugabyteDB cluster. aliases: - /troubleshoot/cluster/replace-master/ - /latest/troubleshoot/cluster/replace-master/ From fdffc1cc0443f1f921f56df7b433bf61a3f12f81 Mon Sep 17 00:00:00 2001 From: ddorian Date: Thu, 25 Jun 2020 02:43:39 -0700 Subject: [PATCH 06/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- .../troubleshoot/cluster/replace_master.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 886e956a19a3..b8bcb91cc2c2 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -26,12 +26,21 @@ For the steps below, the examples use the following scenario: -## Start new master and add to cluster -Start `M4` with `--master_addresses= (empty string)`, this will put the master in what we refer to as `ShellMode`. -Add the new master into the quorum using: -```bash -./yb-admin -master_addresses M1:7100,M2:7100,M3:7100 change_master_config ADD_SERVER M4 7100 + 1. Start the new (replacement) YB-Master server in standby mode. + + To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). + + ```sh + $ ./bin/yb-master --master_addresses="" + ``` + +2. Add the new YB-Master server into the existing cluster. + +To add the new YB-Master server, run the [`yb-admin change_master_config ADD_SERVER`](../../admin/cli/yb-admin/#change-master-config) command. + +```sh +./bin/yb-admin -master_addresses M1:7100,M2:7100,M3:7100 change_master_config ADD_SERVER M4 7100 ``` ## Remove old master From 9dd9a08f42f1ef62fe113da0cc21dd82e44d9e7e Mon Sep 17 00:00:00 2001 From: ddorian Date: Thu, 25 Jun 2020 02:43:46 -0700 Subject: [PATCH 07/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- .../latest/troubleshoot/cluster/replace_master.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index b8bcb91cc2c2..44dd8fec9eaa 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -43,13 +43,12 @@ To add the new YB-Master server, run the [`yb-admin change_master_config ADD_SER ./bin/yb-admin -master_addresses M1:7100,M2:7100,M3:7100 change_master_config ADD_SERVER M4 7100 ``` -## Remove old master -Remove the old master from the quorum using: -```bash -./yb-admin -master_addresses M1:7100,M2:7100,M3:7100,M4:7100 change_master_config REMOVE_SERVER M1 7100 -``` +3. Remove the old YB-Master server from the cluster. -Note: the master addresses needs to include all 4 masters, in case the new one is suddenly the master leader!) +To remove the failed YB-Master server from the cluster, use the [`yb-admin change_master_config REMOVE_SERVER`](../../admin/yb-admin/#change-master-config`) command. + +```sh +./yb-admin -master_addresses M1:7100,M2:7100,M3:7100,M4:7100 change_master_config REMOVE_SERVER M1 7100 ## Validate cluster From 54962e4f4b260fea32f6aa1a32d41771ccf01501 Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Thu, 25 Jun 2020 15:20:07 +0200 Subject: [PATCH 08/21] added missing note --- .../troubleshoot/cluster/replace_master.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 44dd8fec9eaa..54418c1bb768 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -25,15 +25,13 @@ For the steps below, the examples use the following scenario: - The default master RPC port is `7100` +1. Start the new (replacement) YB-Master server in standby mode. +To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). - 1. Start the new (replacement) YB-Master server in standby mode. - - To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). - - ```sh - $ ./bin/yb-master --master_addresses="" - ``` +```sh +./bin/yb-master --master_addresses="" +``` 2. Add the new YB-Master server into the existing cluster. @@ -49,7 +47,12 @@ To remove the failed YB-Master server from the cluster, use the [`yb-admin chang ```sh ./yb-admin -master_addresses M1:7100,M2:7100,M3:7100,M4:7100 change_master_config REMOVE_SERVER M1 7100 +``` +{{< note title="Note" >}} + +Make sure to specify all YB-Master addresses which will prevent the new YB-Master server from unexpectedly becoming the leader. +{{< /note >}} ## Validate cluster Validate that your set of masters is now `M2`, `M3` and `M4` using: From e32b0149c1775d6c1773f70be04d520bcbd54840 Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Mon, 29 Jun 2020 16:12:23 +0200 Subject: [PATCH 09/21] add note --- docs/content/latest/troubleshoot/cluster/replace_master.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 54418c1bb768..79986e515842 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -68,4 +68,6 @@ If, however, none of the current masters are present in the TS flag, then the TS So it is important to make sure to update `--tserver_master_addrs` on every TS to the new set of master addresses, `M2:7100,M3:7100,M4:7100`! +{{< note title="Note" >}} If the master you wish to replace is already dead (eg: VM was terminated), you might want to first do the `REMOVE` step, then do the `ADD` step. +{{< /note >}} From 34d4435163920dd36ea5fc14369dccc808750726 Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Tue, 29 Sep 2020 10:41:37 +0200 Subject: [PATCH 10/21] move note to the top --- .../content/latest/troubleshoot/cluster/replace_master.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 79986e515842..0b37d2ef94e5 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -24,6 +24,10 @@ For the steps below, the examples use the following scenario: - A new YB-Master server (`M4`) will replace `M1`. - The default master RPC port is `7100` +{{< note title="Note" >}} +If the master you wish to replace is already dead (eg: VM was terminated), you might want to first do the `REMOVE` step, then do the `ADD` step. +{{< /note >}} + 1. Start the new (replacement) YB-Master server in standby mode. @@ -67,7 +71,3 @@ If any one of those masters is still part of the active quorum, then they will p If, however, none of the current masters are present in the TS flag, then the TS will not be able to join the cluster! So it is important to make sure to update `--tserver_master_addrs` on every TS to the new set of master addresses, `M2:7100,M3:7100,M4:7100`! - -{{< note title="Note" >}} -If the master you wish to replace is already dead (eg: VM was terminated), you might want to first do the `REMOVE` step, then do the `ADD` step. -{{< /note >}} From ddf04ef81e4b1fd1c93ccc6e8647222c798420dc Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Tue, 29 Sep 2020 10:54:08 +0200 Subject: [PATCH 11/21] nicer headers --- docs/content/latest/troubleshoot/cluster/replace_master.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 0b37d2ef94e5..c6346a14988b 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -29,7 +29,7 @@ If the master you wish to replace is already dead (eg: VM was terminated), you m {{< /note >}} -1. Start the new (replacement) YB-Master server in standby mode. +## Start the new (replacement) YB-Master server in standby mode. To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). @@ -37,7 +37,7 @@ To start `yb-master` in standby mode, set the `--master_addresses` flag to an em ./bin/yb-master --master_addresses="" ``` -2. Add the new YB-Master server into the existing cluster. +## Add the new YB-Master server into the existing cluster. To add the new YB-Master server, run the [`yb-admin change_master_config ADD_SERVER`](../../admin/cli/yb-admin/#change-master-config) command. @@ -45,7 +45,7 @@ To add the new YB-Master server, run the [`yb-admin change_master_config ADD_SER ./bin/yb-admin -master_addresses M1:7100,M2:7100,M3:7100 change_master_config ADD_SERVER M4 7100 ``` -3. Remove the old YB-Master server from the cluster. +## Remove the old YB-Master server from the cluster. To remove the failed YB-Master server from the cluster, use the [`yb-admin change_master_config REMOVE_SERVER`](../../admin/yb-admin/#change-master-config`) command. From 67996cc5a0d936e7b8b6d6fe45aa43b30e5f4933 Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Tue, 29 Sep 2020 22:20:39 +0200 Subject: [PATCH 12/21] fix sentence for including all yb-masters --- docs/content/latest/troubleshoot/cluster/replace_master.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index c6346a14988b..845ff4405c5d 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -54,7 +54,7 @@ To remove the failed YB-Master server from the cluster, use the [`yb-admin chang ``` {{< note title="Note" >}} -Make sure to specify all YB-Master addresses which will prevent the new YB-Master server from unexpectedly becoming the leader. +Make sure to specify all YB-Master addresses including M4 to make sure that if M4 becomes the leader, then yb-admin can find it. {{< /note >}} ## Validate cluster From 96afa870d67da5dd4c45a9607d81bbfbfccc0c36 Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Tue, 29 Sep 2020 22:28:42 +0200 Subject: [PATCH 13/21] explain better adding a master later --- docs/content/latest/troubleshoot/cluster/replace_master.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 845ff4405c5d..87e44e8fb847 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -31,7 +31,9 @@ If the master you wish to replace is already dead (eg: VM was terminated), you m ## Start the new (replacement) YB-Master server in standby mode. -To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). +To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). When the + `--master_addresses` parameter is not set, this master server starts running without joining any existing master + quorum. The node will be added to the master quorum in a later step. ```sh ./bin/yb-master --master_addresses="" From 2849ae335c05d41a4e81f538b94e8c1d835c0cd6 Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Thu, 8 Oct 2020 11:20:56 +0200 Subject: [PATCH 14/21] replacement -> standby --- docs/content/latest/troubleshoot/cluster/replace_master.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 87e44e8fb847..9eaf55a18536 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -29,7 +29,7 @@ If the master you wish to replace is already dead (eg: VM was terminated), you m {{< /note >}} -## Start the new (replacement) YB-Master server in standby mode. +## Start the new (standby) YB-Master server in standby mode. To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). When the `--master_addresses` parameter is not set, this master server starts running without joining any existing master From 6568157f6fe8b537bd89469583803b620f84cc80 Mon Sep 17 00:00:00 2001 From: Dorian Hoxha Date: Thu, 8 Oct 2020 11:38:37 +0200 Subject: [PATCH 15/21] use empty string for --master_addresses --- docs/content/latest/troubleshoot/cluster/replace_master.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 9eaf55a18536..1d7bffed1a46 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -32,11 +32,12 @@ If the master you wish to replace is already dead (eg: VM was terminated), you m ## Start the new (standby) YB-Master server in standby mode. To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). When the - `--master_addresses` parameter is not set, this master server starts running without joining any existing master + `--master_addresses` is `""`, this master server starts running without joining any existing master quorum. The node will be added to the master quorum in a later step. ```sh -./bin/yb-master --master_addresses="" +./bin/yb-master --master_addresses="" --fs_data_dirs= [any other flags you would normally pass + to this master process] ``` ## Add the new YB-Master server into the existing cluster. From 7d6017b037f933c2679f6db635b91143502030b4 Mon Sep 17 00:00:00 2001 From: ddorian Date: Tue, 13 Oct 2020 00:28:07 -0700 Subject: [PATCH 16/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 1d7bffed1a46..78263918b54d 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -29,7 +29,7 @@ If the master you wish to replace is already dead (eg: VM was terminated), you m {{< /note >}} -## Start the new (standby) YB-Master server in standby mode. +## Start the replacement YB-Master server in standby mode. To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). When the `--master_addresses` is `""`, this master server starts running without joining any existing master From f752eff4386e955496190532cfca37f547806aab Mon Sep 17 00:00:00 2001 From: ddorian Date: Tue, 13 Oct 2020 00:28:32 -0700 Subject: [PATCH 17/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 78263918b54d..9a5c14b25224 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -32,8 +32,7 @@ If the master you wish to replace is already dead (eg: VM was terminated), you m ## Start the replacement YB-Master server in standby mode. To start `yb-master` in standby mode, set the `--master_addresses` flag to an empty string (`""`). When the - `--master_addresses` is `""`, this master server starts running without joining any existing master - quorum. The node will be added to the master quorum in a later step. + `--master_addresses` is `""`, this master server starts without joining any existing master quorum. The node will be added to the master quorum in a later step. ```sh ./bin/yb-master --master_addresses="" --fs_data_dirs= [any other flags you would normally pass From 80c100d627e20c80ac57e1dbd9eaf788e0778fbc Mon Sep 17 00:00:00 2001 From: ddorian Date: Tue, 13 Oct 2020 00:34:19 -0700 Subject: [PATCH 18/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 9a5c14b25224..408dedf0dc0c 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -39,7 +39,7 @@ To start `yb-master` in standby mode, set the `--master_addresses` flag to an em to this master process] ``` -## Add the new YB-Master server into the existing cluster. +## Add the replacement YB-Master server into the existing cluster. To add the new YB-Master server, run the [`yb-admin change_master_config ADD_SERVER`](../../admin/cli/yb-admin/#change-master-config) command. From ceac03438423b01b842449dd90f9d1b7485613bd Mon Sep 17 00:00:00 2001 From: ddorian Date: Tue, 13 Oct 2020 00:34:29 -0700 Subject: [PATCH 19/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 408dedf0dc0c..4b84efe968ac 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -47,7 +47,7 @@ To add the new YB-Master server, run the [`yb-admin change_master_config ADD_SER ./bin/yb-admin -master_addresses M1:7100,M2:7100,M3:7100 change_master_config ADD_SERVER M4 7100 ``` -## Remove the old YB-Master server from the cluster. +## Remove the failed YB-Master server from the cluster. To remove the failed YB-Master server from the cluster, use the [`yb-admin change_master_config REMOVE_SERVER`](../../admin/yb-admin/#change-master-config`) command. From 92959e567e89c1185e7c0ba54a8869373c2ec5a6 Mon Sep 17 00:00:00 2001 From: ddorian Date: Tue, 13 Oct 2020 00:34:45 -0700 Subject: [PATCH 20/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index 4b84efe968ac..d828360a8264 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -56,7 +56,7 @@ To remove the failed YB-Master server from the cluster, use the [`yb-admin chang ``` {{< note title="Note" >}} -Make sure to specify all YB-Master addresses including M4 to make sure that if M4 becomes the leader, then yb-admin can find it. +Make sure to specify all YB-Master addresses, including M4, to make sure that if M4 becomes the leader, then yb-admin can find it. {{< /note >}} ## Validate cluster From 2e2a24ff00fe3229f87351329539d7283eb2b0f6 Mon Sep 17 00:00:00 2001 From: ddorian Date: Tue, 13 Oct 2020 00:35:08 -0700 Subject: [PATCH 21/21] Update docs/content/latest/troubleshoot/cluster/replace_master.md Co-authored-by: Steve Bang --- docs/content/latest/troubleshoot/cluster/replace_master.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/content/latest/troubleshoot/cluster/replace_master.md b/docs/content/latest/troubleshoot/cluster/replace_master.md index d828360a8264..340069d11e74 100644 --- a/docs/content/latest/troubleshoot/cluster/replace_master.md +++ b/docs/content/latest/troubleshoot/cluster/replace_master.md @@ -25,7 +25,9 @@ For the steps below, the examples use the following scenario: - The default master RPC port is `7100` {{< note title="Note" >}} -If the master you wish to replace is already dead (eg: VM was terminated), you might want to first do the `REMOVE` step, then do the `ADD` step. + +If the master to be replaced is already dead (for example, the VM was terminated), perform the `REMOVE` step first, and then the `ADD` step. + {{< /note >}}