Update local cluster tutorials and document start-single-node #5445

jseldess · 2019-09-18T03:53:57Z

Refactor start a local cluster tutorials.
Document cockroach start-single-node, including how to scale up from a single-node cluster.
Refresh other local cluster tutorials:
- Data replication
- Fault tolerance and recovery
- JSON support

Will finish the rest of the local cluster tutorials, as well as document new cockroach start and missing cockroach start-single-node flags in a separate PR. Getting this merged so @thatnerd has updated potential source material for training scripts.

cockroach-teamcity · 2019-09-18T03:54:04Z

This change is

cockroach-teamcity · 2019-09-19T21:31:34Z

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/08c27af1428c5a6a4afa01e76d004c29018dd285/

Edited pages:

cockroach-teamcity · 2019-09-20T12:08:50Z

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/34ec3c2cc2be69594f91dc84546f509696460afb/

Edited pages:

knz

Looks good overall, just a few minor comments.
Also I did not review the secure page, as it is a bit out of my depth.

Reviewed 11 of 12 files at r1.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @jseldess)

v19.1/start-a-node.md, line 24 at r1 (raw file):

~~~ shell
$ cockroach start <flags, including --join>

If you remove the ampersand and don't include --background, you probably want to say something about running init in a separate shell.

v19.2/cockroach-start-single-node.md, line 29 at r1 (raw file):

All flags must be specified each time the node is started, as they will not be remembered.

"All flags that you wish to override must be specified..."

(We have useful defaults, so all flags are not mandatory)

I haven't checked the page for cockroach start but it may need the same tweak?

v19.2/cockroach-start-single-node.md, line 213 at r1 (raw file):

### Scale to multiple nodes

Scaling a cluster started with `cockroach start-single-node` involves restarting the first node with the `cockroach start` command instead, and then adding new nodes with that command as well, all using a `--join` flag that forms them into a single multi-node cluster. Since replication is disable in clusters started with `start-single-node`, you also need to enable replication to get CockroachDB's availability and consistency guarantees.

... replication is disabled ...

v19.2/cockroach-start-single-node.md, line 281 at r1 (raw file):

    {% include copy-clipboard.html %}
    ~~~ sql
    > ALTER RANGE default CONFIGURE ZONE USING num_replicas = 5;

I think you need to set 5 replicas for system, and 3 for default.

actually in truth the command is a bit more complex, because none of the default zones do not inherit from the top, so in truth you must set them all.

alter range default configure zone using num_replicas = 3;
alter range system configure zone using num_replicas = 5;
alter database system configure zone using num_replicas = 5;
alter table system.public.jobs configure zone using num_replicas = 5;
alter range meta configure zone using num_replicas = 5;
alter range liveness configure zone using num_replicas = 5;

(This is a UX defect now that I am coming to try it out myself. We should provide a one-stop command to do all this. Maybe file an issue.)

v19.2/start-a-node.md, line 20 at r1 (raw file):

$ cockroach start <flags, including --join>
~~~
~~~ shell

ditto previous comment on this

bdarnell

Reviewed 8 of 12 files at r1.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @jseldess)

v19.2/cockroach-start-single-node.md, line 83 at r1 (raw file):

### Logging

By default, `cockroach start` writes all messages to log files, and prints nothing to `stderr`. However, you can control the process's [logging](debug-and-error-logs.html) behavior with the following flags:

Here and in the next two sections you've got cockroach start instead of cockroach start-single-node.

jseldess

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @bdarnell and @knz)

v19.1/start-a-node.md, line 24 at r1 (raw file):

Previously, knz (kena) wrote…

If you remove the ampersand and don't include --background, you probably want to say something about running init in a separate shell.

Revised.

v19.2/cockroach-start-single-node.md, line 29 at r1 (raw file):

Previously, knz (kena) wrote…

All flags must be specified each time the node is started, as they will not be remembered.
"All flags that you wish to override must be specified..."

(We have useful defaults, so all flags are not mandatory)

I haven't checked the page for cockroach start but it may need the same tweak?

Done.

v19.2/cockroach-start-single-node.md, line 83 at r1 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

Here and in the next two sections you've got cockroach start instead of cockroach start-single-node.

Done.

v19.2/cockroach-start-single-node.md, line 213 at r1 (raw file):

Previously, knz (kena) wrote…

... replication is disabled ...

Done.

v19.2/cockroach-start-single-node.md, line 281 at r1 (raw file):

Previously, knz (kena) wrote…

I think you need to set 5 replicas for system, and 3 for default.

actually in truth the command is a bit more complex, because none of the default zones do not inherit from the top, so in truth you must set them all.
alter range default configure zone using num_replicas = 3;
alter range system configure zone using num_replicas = 5;
alter database system configure zone using num_replicas = 5;
alter table system.public.jobs configure zone using num_replicas = 5;
alter range meta configure zone using num_replicas = 5;
alter range liveness configure zone using num_replicas = 5;
(This is a UX defect now that I am coming to try it out myself. We should provide a one-stop command to do all this. Maybe file an issue.)

Thanks for catching that! Updated to encompass all of the replication zone explicitly set to rep factor 1 by cockroach start-single-node:

root@localhost:26257/defaultdb> show zone configurations;
                       target                      |                                 config_sql
+--------------------------------------------------+-----------------------------------------------------------------------------+
  RANGE default                                    | ALTER RANGE default CONFIGURE ZONE USING
                                                   |     range_min_bytes = 16777216,
                                                   |     range_max_bytes = 67108864,
                                                   |     gc.ttlseconds = 90000,
                                                   |     num_replicas = 1,
                                                   |     constraints = '[]',
                                                   |     lease_preferences = '[]'
  DATABASE system                                  | ALTER DATABASE system CONFIGURE ZONE USING
                                                   |     range_min_bytes = 16777216,
                                                   |     range_max_bytes = 67108864,
                                                   |     gc.ttlseconds = 90000,
                                                   |     num_replicas = 1,
                                                   |     constraints = '[]',
                                                   |     lease_preferences = '[]'
  TABLE system.public.jobs                         | ALTER TABLE system.public.jobs CONFIGURE ZONE USING
                                                   |     range_min_bytes = 16777216,
                                                   |     range_max_bytes = 67108864,
                                                   |     gc.ttlseconds = 600,
                                                   |     num_replicas = 1,
                                                   |     constraints = '[]',
                                                   |     lease_preferences = '[]'
  RANGE meta                                       | ALTER RANGE meta CONFIGURE ZONE USING
                                                   |     range_min_bytes = 16777216,
                                                   |     range_max_bytes = 67108864,
                                                   |     gc.ttlseconds = 3600,
                                                   |     num_replicas = 1,
                                                   |     constraints = '[]',
                                                   |     lease_preferences = '[]'
  RANGE system                                     | ALTER RANGE system CONFIGURE ZONE USING
                                                   |     range_min_bytes = 16777216,
                                                   |     range_max_bytes = 67108864,
                                                   |     gc.ttlseconds = 90000,
                                                   |     num_replicas = 1,
                                                   |     constraints = '[]',
                                                   |     lease_preferences = '[]'
  RANGE liveness                                   | ALTER RANGE liveness CONFIGURE ZONE USING
                                                   |     range_min_bytes = 16777216,
                                                   |     range_max_bytes = 67108864,
                                                   |     gc.ttlseconds = 600,
                                                   |     num_replicas = 1,
                                                   |     constraints = '[]',
                                                   |     lease_preferences = '[]'
  TABLE system.public.replication_constraint_stats | ALTER TABLE system.public.replication_constraint_stats CONFIGURE ZONE USING
                                                   |     gc.ttlseconds = 600,
                                                   |     num_replicas = 1,
                                                   |     constraints = '[]',
                                                   |     lease_preferences = '[]'
  TABLE system.public.replication_stats            | ALTER TABLE system.public.replication_stats CONFIGURE ZONE USING
                                                   |     gc.ttlseconds = 600,
                                                   |     num_replicas = 1,
                                                   |     constraints = '[]',
                                                   |     lease_preferences = '[]'
(8 rows)

v19.2/start-a-node.md, line 20 at r1 (raw file):

Previously, knz (kena) wrote…

ditto previous comment on this

Done.

cockroach-teamcity · 2019-09-26T16:50:17Z

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/5d1afd68ac9238a8339a2f5e95ef12a60f028b47/

Edited pages:

cockroach-teamcity · 2019-09-26T21:24:08Z

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/619448fb08ced9bb40ce7da9441ad1f730cbc897/

Edited pages:

cockroach-teamcity · 2019-09-27T03:57:15Z

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/c08f6366fca88df1d5d5253870ebf29962d7bd9b/

Edited pages:

cockroach-teamcity · 2019-09-27T04:14:22Z

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/ea74db3771de843fdaec758ef108f64fc7770a56/

Edited pages:

cockroach-teamcity · 2019-09-27T17:36:22Z

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/96f47831d8654df33a0b18bedf97099883c90c33/

Edited pages:

- Remove node startup without --join. - Start nodes in background for better UX. - Refresh existing content and add steps to show built-in workload, resiliency to node failure, scaling and rebalancing. Fixes #5430.

Including example for scaling from one to multiple nodes. Fixes #5301.

Refresh it to be more interestoing and comprehensive.

cockroach-teamcity · 2019-09-28T01:43:13Z

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/b44a8df6dfbc6d364c9cbe25467a01360c0d600c/

Edited pages:

jseldess force-pushed the local-cluster-refresh branch 4 times, most recently from 710adfe to 08c27af Compare September 19, 2019 21:28

cockroachdb deleted a comment from cockroach-teamcity Sep 19, 2019

jseldess force-pushed the local-cluster-refresh branch from 08c27af to 34ec3c2 Compare September 20, 2019 12:05

knz reviewed Sep 20, 2019

View reviewed changes

bdarnell approved these changes Sep 21, 2019

View reviewed changes

jseldess force-pushed the local-cluster-refresh branch from 34ec3c2 to 5d1afd6 Compare September 26, 2019 16:45

jseldess commented Sep 26, 2019

View reviewed changes

jseldess force-pushed the local-cluster-refresh branch from 5d1afd6 to 619448f Compare September 26, 2019 21:21

jseldess force-pushed the local-cluster-refresh branch from 619448f to c08f636 Compare September 27, 2019 03:54

jseldess force-pushed the local-cluster-refresh branch from c08f636 to ea74db3 Compare September 27, 2019 04:11

jseldess changed the title ~~[WIP] Refactor start a local cluster tutorials~~ Refactor start a local cluster tutorials Sep 28, 2019

jseldess added 6 commits September 27, 2019 21:37

Refactor local cluster tutorials

355abb4

- Remove node startup without --join. - Start nodes in background for better UX. - Refresh existing content and add steps to show built-in workload, resiliency to node failure, scaling and rebalancing. Fixes #5430.

Document start-single-node command

c0fa681

Including example for scaling from one to multiple nodes. Fixes #5301.

Update json tutorial to use start-single-node

fd44097

Fix misc single-node start commands

ef8a66f

Update fault tolerance tutorial

1de1ed9

Refresh it to be more interestoing and comprehensive.

Update replication tutorial

b44a8df

jseldess force-pushed the local-cluster-refresh branch from 96f4783 to b44a8df Compare September 28, 2019 01:37

jseldess changed the title ~~Refactor start a local cluster tutorials~~ Update local cluster tutorials and document start-single-node Sep 28, 2019

jseldess merged commit 170b598 into master Sep 28, 2019

jseldess deleted the local-cluster-refresh branch September 28, 2019 01:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update local cluster tutorials and document start-single-node #5445

Update local cluster tutorials and document start-single-node #5445

Uh oh!

jseldess commented Sep 18, 2019 •

edited

Loading

Uh oh!

cockroach-teamcity commented Sep 18, 2019

Uh oh!

cockroach-teamcity commented Sep 19, 2019

Uh oh!

cockroach-teamcity commented Sep 20, 2019

Uh oh!

knz left a comment

Uh oh!

bdarnell left a comment

Uh oh!

jseldess left a comment

Uh oh!

cockroach-teamcity commented Sep 26, 2019

Uh oh!

cockroach-teamcity commented Sep 26, 2019

Uh oh!

cockroach-teamcity commented Sep 27, 2019

Uh oh!

cockroach-teamcity commented Sep 27, 2019

Uh oh!

cockroach-teamcity commented Sep 27, 2019

Uh oh!

cockroach-teamcity commented Sep 28, 2019

Uh oh!

Uh oh!

Update local cluster tutorials and document start-single-node #5445

Update local cluster tutorials and document start-single-node #5445

Uh oh!

Conversation

jseldess commented Sep 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cockroach-teamcity commented Sep 18, 2019

Uh oh!

cockroach-teamcity commented Sep 19, 2019

Uh oh!

cockroach-teamcity commented Sep 20, 2019

Uh oh!

knz left a comment

Choose a reason for hiding this comment

Uh oh!

bdarnell left a comment

Choose a reason for hiding this comment

Uh oh!

jseldess left a comment

Choose a reason for hiding this comment

Uh oh!

cockroach-teamcity commented Sep 26, 2019

Uh oh!

cockroach-teamcity commented Sep 26, 2019

Uh oh!

cockroach-teamcity commented Sep 27, 2019

Uh oh!

cockroach-teamcity commented Sep 27, 2019

Uh oh!

cockroach-teamcity commented Sep 27, 2019

Uh oh!

cockroach-teamcity commented Sep 28, 2019

Uh oh!

Uh oh!

jseldess commented Sep 18, 2019 •

edited

Loading