diff --git a/_includes/v19.2/orchestration/kubernetes-prometheus-alertmanager.md b/_includes/v19.2/orchestration/kubernetes-prometheus-alertmanager.md index a8cf522d689..e2a0df4e8b9 100644 --- a/_includes/v19.2/orchestration/kubernetes-prometheus-alertmanager.md +++ b/_includes/v19.2/orchestration/kubernetes-prometheus-alertmanager.md @@ -67,6 +67,10 @@ If you're on Hosted GKE, before starting, make sure the email address associated 4. Use our [`prometheus.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/prometheus.yaml) file to create the various objects necessary to run a Prometheus instance: + {{site.data.alerts.callout_success}} + This configuration defaults to using the Kubernetes CA for authentication. + {{site.data.alerts.end}} + {% include copy-clipboard.html %} ~~~ shell $ kubectl apply \ @@ -142,7 +146,7 @@ Active monitoring helps you spot problems early, but it is also essential to sen ~~~ {{site.data.alerts.callout_danger}} - The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `altermanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen. + The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen. {{site.data.alerts.end}} 4. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts: diff --git a/_includes/v19.2/orchestration/kubernetes-scale-cluster.md b/_includes/v19.2/orchestration/kubernetes-scale-cluster.md index 64f7ca9102b..e847a6ba8d1 100644 --- a/_includes/v19.2/orchestration/kubernetes-scale-cluster.md +++ b/_includes/v19.2/orchestration/kubernetes-scale-cluster.md @@ -20,6 +20,10 @@ To do this, add a new worker node and then edit your StatefulSet configuration t ~~~ statefulset.apps/cockroachdb scaled ~~~ + + {{site.data.alerts.callout_success}} + If you aren't using the Kubernetes CA to sign certificates, you can now skip to step 6. + {{site.data.alerts.end}}
diff --git a/_includes/v19.2/orchestration/start-cockroachdb-local-insecure.md b/_includes/v19.2/orchestration/start-cockroachdb-local-insecure.md new file mode 100644 index 00000000000..bebb6eb3062 --- /dev/null +++ b/_includes/v19.2/orchestration/start-cockroachdb-local-insecure.md @@ -0,0 +1,83 @@ +1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml + ~~~ + + ~~~ + service/cockroachdb-public created + service/cockroachdb created + poddisruptionbudget.policy/cockroachdb-budget created + statefulset.apps/cockroachdb created + ~~~ + +2. Confirm that three pods are `Running` successfully. Note that they will not + be considered `Ready` until after the cluster has been initialized: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 0/1 Running 0 2m + cockroachdb-1 0/1 Running 0 2m + cockroachdb-2 0/1 Running 0 2m + ~~~ + +3. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pv + ~~~ + + ~~~ + NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE + pvc-52f51ecf-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-0 26s + pvc-52fd3a39-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-1 27s + pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s + ~~~ + +4. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create \ + -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml + ~~~ + + ~~~ + job.batch/cluster-init created + ~~~ + +5. Confirm that cluster initialization has completed successfully. The job should be considered successful and the Kubernetes pods should soon be considered `Ready`: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get job cluster-init + ~~~ + + ~~~ + NAME COMPLETIONS DURATION AGE + cluster-init 1/1 7s 27s + ~~~ + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cluster-init-cqf8l 0/1 Completed 0 56s + cockroachdb-0 1/1 Running 0 7m51s + cockroachdb-1 1/1 Running 0 7m51s + cockroachdb-2 1/1 Running 0 7m51s + ~~~ + +{{site.data.alerts.callout_success}} +The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to a pod/node's logs to troubleshoot, use `kubectl logs ` rather than checking the log on the persistent volume. +{{site.data.alerts.end}} diff --git a/_includes/v19.2/orchestration/start-cockroachdb-local-secure.md b/_includes/v19.2/orchestration/start-cockroachdb-local-secure.md new file mode 100644 index 00000000000..0558bcdf3b2 --- /dev/null +++ b/_includes/v19.2/orchestration/start-cockroachdb-local-secure.md @@ -0,0 +1,366 @@ +Download and modify our StatefulSet configuration, depending on how you want to sign your certificates. + +{{site.data.alerts.callout_danger}} +Some environments, such as Amazon EKS, do not support certificates signed by Kubernetes' built-in CA. In this case, use the second configuration below. +{{site.data.alerts.end}} + +- Using the Kubernetes CA: [`cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml). + + {% include copy-clipboard.html %} + ~~~ shell + $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml + ~~~ + +- Using a non-Kubernetes CA: [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml) + + {% include copy-clipboard.html %} + ~~~ shell + $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml + ~~~ + +{{site.data.alerts.callout_success}} +If you change the StatefulSet name from the default `cockroachdb`, be sure to start and end with an alphanumeric character and otherwise use lowercase alphanumeric characters, `-`, or `.` so as to comply with [CSR naming requirements](orchestrate-cockroachdb-with-kubernetes.html#csr-names). +{{site.data.alerts.end}} + +#### Initialize the cluster + +Choose the authentication method that corresponds to the StatefulSet configuration you downloaded and modified above. + +- [Kubernetes CA](#kubernetes-ca) +- [Non-Kubernetes CA](#non-kubernetes-ca) + +{{site.data.alerts.callout_success}} +The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to a pod/node's logs to troubleshoot, use `kubectl logs ` rather than checking the log on the persistent volume. +{{site.data.alerts.end}} + +##### Kubernetes CA + +1. Use the config file you downloaded to create the StatefulSet that automatically creates 3 pods, each running a CockroachDB node: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create -f cockroachdb-statefulset-secure.yaml + ~~~ + + ~~~ + serviceaccount/cockroachdb created + role.rbac.authorization.k8s.io/cockroachdb created + clusterrole.rbac.authorization.k8s.io/cockroachdb created + rolebinding.rbac.authorization.k8s.io/cockroachdb created + clusterrolebinding.rbac.authorization.k8s.io/cockroachdb created + service/cockroachdb-public created + service/cockroachdb created + poddisruptionbudget.policy/cockroachdb-budget created + statefulset.apps/cockroachdb created + ~~~ + +2. As each pod is created, it issues a Certificate Signing Request, or CSR, to have the node's certificate signed by the Kubernetes CA. You must manually check and approve each node's certificates, at which point the CockroachDB node is started in the pod. + + 1. Get the names of the `Pending` CSRs: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get csr + ~~~ + + ~~~ + NAME AGE REQUESTOR CONDITION + default.node.cockroachdb-0 1m system:serviceaccount:default:cockroachdb Pending + default.node.cockroachdb-1 1m system:serviceaccount:default:cockroachdb Pending + default.node.cockroachdb-2 1m system:serviceaccount:default:cockroachdb Pending + ... + ~~~ + + If you do not see a `Pending` CSR, wait a minute and try again. + + 2. Examine the CSR for the first pod: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl describe csr default.node.cockroachdb-0 + ~~~ + + ~~~ + Name: default.node.cockroachdb-0 + Labels: + Annotations: + CreationTimestamp: Thu, 09 Nov 2017 13:39:37 -0500 + Requesting User: system:serviceaccount:default:cockroachdb + Status: Pending + Subject: + Common Name: node + Serial Number: + Organization: Cockroach + Subject Alternative Names: + DNS Names: localhost + cockroachdb-0.cockroachdb.default.svc.cluster.local + cockroachdb-0.cockroachdb + cockroachdb-public + cockroachdb-public.default.svc.cluster.local + IP Addresses: 127.0.0.1 + 10.48.1.6 + Events: + ~~~ + + 3. If everything looks correct, approve the CSR for the first pod: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl certificate approve default.node.cockroachdb-0 + ~~~ + + ~~~ + certificatesigningrequest "default.node.cockroachdb-0" approved + ~~~ + + 4. Repeat steps 2 and 3 for the other 2 pods. + +3. Initialize the CockroachDB cluster: + + 1. Confirm that three pods are `Running` successfully. Note that they will not be considered `Ready` until after the cluster has been initialized: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 0/1 Running 0 2m + cockroachdb-1 0/1 Running 0 2m + cockroachdb-2 0/1 Running 0 2m + ~~~ + + 2. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pv + ~~~ + + ~~~ + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + pvc-9e435563-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-0 standard 51m + pvc-9e47d820-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-1 standard 51m + pvc-9e4f57f0-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-2 standard 51m + ~~~ + + 3. Use our [`cluster-init-secure.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init-secure.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create \ + -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init-secure.yaml + ~~~ + + ~~~ + job.batch/cluster-init-secure created + ~~~ + + 4. Approve the CSR for the one-off pod from which cluster initialization happens: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl certificate approve default.client.root + ~~~ + + ~~~ + certificatesigningrequest.certificates.k8s.io/default.client.root approved + ~~~ + + 5. Confirm that cluster initialization has completed successfully. The job should be considered successful and the Kubernetes pods should soon be considered `Ready`: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get job cluster-init-secure + ~~~ + + ~~~ + NAME COMPLETIONS DURATION AGE + cluster-init-secure 1/1 23s 35s + ~~~ + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cluster-init-secure-q8s7v 0/1 Completed 0 55s + cockroachdb-0 1/1 Running 0 3m + cockroachdb-1 1/1 Running 0 3m + cockroachdb-2 1/1 Running 0 3m + ~~~ + +##### Non-Kubernetes CA + +{{site.data.alerts.callout_info}} +The below steps use [`cockroach cert` commands](cockroach-cert.html) to quickly generate and sign the CockroachDB node and client certificates. Read our [Authentication](authentication.html#using-digital-certificates-with-cockroachdb) docs to learn about other methods of signing certificates. +{{site.data.alerts.end}} + +1. Create two directories: + + {% include copy-clipboard.html %} + ~~~ shell + $ mkdir certs + ~~~ + + {% include copy-clipboard.html %} + ~~~ shell + $ mkdir my-safe-directory + ~~~ + + Directory | Description + ----------|------------ + `certs` | You'll generate your CA certificate and all node and client certificates and keys in this directory. + `my-safe-directory` | You'll generate your CA key in this directory and then reference the key when generating node and client certificates. + +2. Create the CA certificate and key pair: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-ca \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +3. Create a client certificate and key pair for the root user: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-client \ + root \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +4. Upload the client certificate and key to the Kubernetes cluster as a secret: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.client.root \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.client.root created + ~~~ + +5. Create the certificate and key pair for your CockroachDB nodes: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-node \ + localhost 127.0.0.1 \ + cockroachdb-public \ + cockroachdb-public.default \ + cockroachdb-public.default.svc.cluster.local \ + *.cockroachdb \ + *.cockroachdb.default \ + *.cockroachdb.default.svc.cluster.local \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +6. Upload the node certificate and key to the Kubernetes cluster as a secret: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.node \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.node created + ~~~ + +7. Check that the secrets were created on the cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get secrets + ~~~ + + ~~~ + NAME TYPE DATA AGE + cockroachdb.client.root Opaque 3 41m + cockroachdb.node Opaque 5 14s + default-token-6qjdb kubernetes.io/service-account-token 3 4m + ~~~ + +8. Use the config file you downloaded to create the StatefulSet that automatically creates 3 pods, each running a CockroachDB node: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create -f cockroachdb-statefulset.yaml + ~~~ + + ~~~ + serviceaccount/cockroachdb created + role.rbac.authorization.k8s.io/cockroachdb created + rolebinding.rbac.authorization.k8s.io/cockroachdb created + service/cockroachdb-public created + service/cockroachdb created + poddisruptionbudget.policy/cockroachdb-budget created + statefulset.apps/cockroachdb created + ~~~ + +9. Initialize the CockroachDB cluster: + + 1. Confirm that three pods are `Running` successfully. Note that they will not be considered `Ready` until after the cluster has been initialized: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 0/1 Running 0 2m + cockroachdb-1 0/1 Running 0 2m + cockroachdb-2 0/1 Running 0 2m + ~~~ + + 2. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pv + ~~~ + + ~~~ + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + pvc-9e435563-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-0 standard 51m + pvc-9e47d820-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-1 standard 51m + pvc-9e4f57f0-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-2 standard 51m + ~~~ + + 3. Run `cockroach init` on one of the pods to complete the node startup process and have them join together as a cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl exec -it cockroachdb-0 \ + -- /cockroach/cockroach init \ + --certs-dir=/cockroach/cockroach-certs + ~~~ + + ~~~ + Cluster successfully initialized + ~~~ + + 4. Confirm that cluster initialization has completed successfully. The job should be considered successful and the Kubernetes pods should soon be considered `Ready`: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 1/1 Running 0 3m + cockroachdb-1 1/1 Running 0 3m + cockroachdb-2 1/1 Running 0 3m + ~~~ \ No newline at end of file diff --git a/_includes/v19.2/orchestration/start-cockroachdb-secure.md b/_includes/v19.2/orchestration/start-cockroachdb-secure.md index 3918e655b02..b918eca3c8f 100644 --- a/_includes/v19.2/orchestration/start-cockroachdb-secure.md +++ b/_includes/v19.2/orchestration/start-cockroachdb-secure.md @@ -1,24 +1,81 @@ -1. From your local workstation, use our [`cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it. +Download and modify our StatefulSet configuration, depending on how you want to sign your certificates. - Download [`cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml): +{{site.data.alerts.callout_danger}} +Some environments, such as Amazon EKS, do not support certificates signed by Kubernetes' built-in CA. In this case, use the second configuration below. +{{site.data.alerts.end}} + +- Using the Kubernetes CA: [`cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml). {% include copy-clipboard.html %} ~~~ shell $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml ~~~ - {{site.data.alerts.callout_danger}} - To avoid running out of memory when CockroachDB is not the only pod on a Kubernetes node, you must set memory limits explicitly. This is because CockroachDB does not detect the amount of memory allocated to its pod when run in Kubernetes. Specify this amount by adjusting `resources.requests.memory` and `resources.limits.memory` in `cockroachdb-statefulset-secure.yaml`. Their values should be identical. - - We recommend setting `cache` and `max-sql-memory` each to 1/4 of your memory allocation. For example, if you are allocating 8Gi of memory to each CockroachDB node, substitute the following values in [this line](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml#L247): - {{site.data.alerts.end}} +- Using a non-Kubernetes CA: [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml) {% include copy-clipboard.html %} ~~~ shell + $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml + ~~~ + +#### Set up configuration file + +Modify the values in the StatefulSet configuration. + +1. To avoid running out of memory when CockroachDB is not the only pod on a Kubernetes node, you *must* set memory limits explicitly. This is because CockroachDB does not detect the amount of memory allocated to its pod when run in Kubernetes. In the `containers` specification, set this amount in both `resources.requests.memory` and `resources.limits.memory`. + + ~~~ + resources: + requests: + memory: "8Gi" + limits: + memory: "8Gi" + ~~~ + + We recommend setting `cache` and `max-sql-memory` each to 1/4 of the memory allocation. These are defined as placeholder percentages in the StatefulSet command that creates the CockroachDB nodes: + + ~~~ + - "exec /cockroach/cockroach start --logtostderr --certs-dir /cockroach/cockroach-certs --advertise-host $(hostname -f) --http-addr 0.0.0.0 --join cockroachdb-0.cockroachdb,cockroachdb-1.cockroachdb,cockroachdb-2.cockroachdb --cache 25% --max-sql-memory 25%" + ~~~ + + {{site.data.alerts.callout_success}} + For example, if you are allocating 8Gi of `memory` to each CockroachDB node, allocate 2Gi to `cache` and 2Gi to `max-sql-memory`. + {{site.data.alerts.end}} + + ~~~ --cache 2Gi --max-sql-memory 2Gi ~~~ - Use the file to create the StatefulSet and start the cluster: +2. In the `volumeClaimTemplates` specification, you may want to modify `resources.requests.storage` for your use case. This configuration defaults to 100Gi of disk space per pod. For more details on customizing disks for performance, see [these instructions](kubernetes-performance.html#disk-type). + + ~~~ + resources: + requests: + storage: "100Gi" + ~~~ + + {{site.data.alerts.callout_info}} + If necessary, you can [expand disk size](orchestrate-cockroachdb-with-kubernetes.html#expand-disk-size) after the cluster is live. + {{site.data.alerts.end}} + +{{site.data.alerts.callout_success}} +If you change the StatefulSet name from the default `cockroachdb`, be sure to start and end with an alphanumeric character and otherwise use lowercase alphanumeric characters, `-`, or `.` so as to comply with [CSR naming requirements](orchestrate-cockroachdb-with-kubernetes.html#csr-names). +{{site.data.alerts.end}} + +#### Initialize the cluster + +Choose the authentication method that corresponds to the StatefulSet configuration you downloaded and modified above. + +- [Kubernetes CA](#kubernetes-ca) +- [Non-Kubernetes CA](#non-kubernetes-ca) + +{{site.data.alerts.callout_success}} +The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to a pod/node's logs to troubleshoot, use `kubectl logs ` rather than checking the log on the persistent volume. +{{site.data.alerts.end}} + +##### Kubernetes CA + +1. Use the config file you downloaded to create the StatefulSet that automatically creates 3 pods, each running a CockroachDB node: {% include copy-clipboard.html %} ~~~ shell @@ -37,28 +94,6 @@ statefulset.apps/cockroachdb created ~~~ - Alternatively, if you'd rather start with a configuration file that has been customized for performance: - - 1. Download our [performance version of `cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-secure.yaml): - - {% include copy-clipboard.html %} - ~~~ shell - $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-secure.yaml - ~~~ - - 2. Modify the file wherever there is a `TODO` comment. - - 3. Use the file to create the StatefulSet and start the cluster: - - {% include copy-clipboard.html %} - ~~~ shell - $ kubectl create -f cockroachdb-statefulset-secure.yaml - ~~~ - - {{site.data.alerts.callout_success}} - If you change the StatefulSet name from the default `cockroachdb`, be sure to start and end with an alphanumeric character and otherwise use lowercase alphanumeric characters, `-`, or `.` so as to comply with [CSR naming requirements](orchestrate-cockroachdb-with-kubernetes.html#csr-names). - {{site.data.alerts.end}} - 2. As each pod is created, it issues a Certificate Signing Request, or CSR, to have the node's certificate signed by the Kubernetes CA. You must manually check and approve each node's certificates, at which point the CockroachDB node is started in the pod. 1. Get the names of the `Pending` CSRs: @@ -140,7 +175,7 @@ {% include copy-clipboard.html %} ~~~ shell - $ kubectl get persistentvolumes + $ kubectl get pv ~~~ ~~~ @@ -198,6 +233,174 @@ cockroachdb-2 1/1 Running 0 3m ~~~ -{{site.data.alerts.callout_success}} -The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to a pod/node's logs to troubleshoot, use `kubectl logs ` rather than checking the log on the persistent volume. +##### Non-Kubernetes CA + +{{site.data.alerts.callout_info}} +The below steps use [`cockroach cert` commands](cockroach-cert.html) to quickly generate and sign the CockroachDB node and client certificates. Read our [Authentication](authentication.html#using-digital-certificates-with-cockroachdb) docs to learn about other methods of signing certificates. {{site.data.alerts.end}} + +1. Create two directories: + + {% include copy-clipboard.html %} + ~~~ shell + $ mkdir certs + ~~~ + + {% include copy-clipboard.html %} + ~~~ shell + $ mkdir my-safe-directory + ~~~ + + Directory | Description + ----------|------------ + `certs` | You'll generate your CA certificate and all node and client certificates and keys in this directory. + `my-safe-directory` | You'll generate your CA key in this directory and then reference the key when generating node and client certificates. + +2. Create the CA certificate and key pair: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-ca \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +3. Create a client certificate and key pair for the root user: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-client \ + root \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +4. Upload the client certificate and key to the Kubernetes cluster as a secret: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.client.root \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.client.root created + ~~~ + +5. Create the certificate and key pair for your CockroachDB nodes: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-node \ + localhost 127.0.0.1 \ + cockroachdb-public \ + cockroachdb-public.default \ + cockroachdb-public.default.svc.cluster.local \ + *.cockroachdb \ + *.cockroachdb.default \ + *.cockroachdb.default.svc.cluster.local \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +6. Upload the node certificate and key to the Kubernetes cluster as a secret: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.node \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.node created + ~~~ + +7. Check that the secrets were created on the cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get secrets + ~~~ + + ~~~ + NAME TYPE DATA AGE + cockroachdb.client.root Opaque 3 41m + cockroachdb.node Opaque 5 14s + default-token-6qjdb kubernetes.io/service-account-token 3 4m + ~~~ + +8. Use the config file you downloaded to create the StatefulSet that automatically creates 3 pods, each running a CockroachDB node: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create -f cockroachdb-statefulset.yaml + ~~~ + + ~~~ + serviceaccount/cockroachdb created + role.rbac.authorization.k8s.io/cockroachdb created + rolebinding.rbac.authorization.k8s.io/cockroachdb created + service/cockroachdb-public created + service/cockroachdb created + poddisruptionbudget.policy/cockroachdb-budget created + statefulset.apps/cockroachdb created + ~~~ + +9. Initialize the CockroachDB cluster: + + 1. Confirm that three pods are `Running` successfully. Note that they will not be considered `Ready` until after the cluster has been initialized: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 0/1 Running 0 2m + cockroachdb-1 0/1 Running 0 2m + cockroachdb-2 0/1 Running 0 2m + ~~~ + + 2. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pv + ~~~ + + ~~~ + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + pvc-9e435563-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-0 standard 51m + pvc-9e47d820-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-1 standard 51m + pvc-9e4f57f0-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-2 standard 51m + ~~~ + + 3. Run `cockroach init` on one of the pods to complete the node startup process and have them join together as a cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl exec -it cockroachdb-0 \ + -- /cockroach/cockroach init \ + --certs-dir=/cockroach/cockroach-certs + ~~~ + + ~~~ + Cluster successfully initialized + ~~~ + + 4. Confirm that cluster initialization has completed successfully. The job should be considered successful and the Kubernetes pods should soon be considered `Ready`: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 1/1 Running 0 3m + cockroachdb-1 1/1 Running 0 3m + cockroachdb-2 1/1 Running 0 3m + ~~~ \ No newline at end of file diff --git a/_includes/v19.2/orchestration/test-cluster-secure.md b/_includes/v19.2/orchestration/test-cluster-secure.md index 047dae43f8e..9cb18c8bf79 100644 --- a/_includes/v19.2/orchestration/test-cluster-secure.md +++ b/_includes/v19.2/orchestration/test-cluster-secure.md @@ -1,7 +1,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely with the `cockroach` binary inside it, get a shell into the pod, and then start the built-in SQL client.
-1. From your local workstation, use our [`client-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/client-secure.yaml) file to launch a pod and keep it running indefinitely: +- Using the Kubernetes CA: [`client-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/client-secure.yaml) {% include copy-clipboard.html %} ~~~ shell @@ -9,15 +9,23 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/client-secure.yaml ~~~ - ~~~ - pod/cockroachdb-client-secure created +- Using a non-Kubernetes CA: [`client.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/client.yaml) + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create \ + -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml ~~~ {{site.data.alerts.callout_info}} The pod uses the `root` client certificate created earlier to initialize the cluster, so there's no CSR approval required. If you issue client certificates for other users, however, be sure your SQL usernames contain only lowercase alphanumeric characters, `-`, or `.` so as to comply with [CSR naming requirements](orchestrate-cockroachdb-with-kubernetes.html#csr-names). {{site.data.alerts.end}} -2. Get a shell into the pod and start the CockroachDB [built-in SQL client](cockroach-sql.html): + ~~~ + pod/cockroachdb-client-secure created + ~~~ + +1. Get a shell into the pod and start the CockroachDB [built-in SQL client](cockroach-sql.html): {% include copy-clipboard.html %} ~~~ shell @@ -42,7 +50,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely root@cockroachdb-public:26257/defaultdb> ~~~ -3. Run some basic [CockroachDB SQL statements](learn-cockroachdb-sql.html): +2. Run some basic [CockroachDB SQL statements](learn-cockroachdb-sql.html): {% include copy-clipboard.html %} ~~~ sql @@ -71,7 +79,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely (1 row) ~~~ -4. [Create a user with a password](create-user.html#create-a-user-with-a-password): +3. [Create a user with a password](create-user.html#create-a-user-with-a-password): {% include copy-clipboard.html %} ~~~ sql @@ -80,7 +88,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely You will need this username and password to access the Admin UI later. -5. Exit the SQL shell and pod: +4. Exit the SQL shell and pod: {% include copy-clipboard.html %} ~~~ sql diff --git a/_includes/v20.1/orchestration/kubernetes-prometheus-alertmanager.md b/_includes/v20.1/orchestration/kubernetes-prometheus-alertmanager.md index bc701168cf6..6579b3220cc 100644 --- a/_includes/v20.1/orchestration/kubernetes-prometheus-alertmanager.md +++ b/_includes/v20.1/orchestration/kubernetes-prometheus-alertmanager.md @@ -52,9 +52,9 @@ If you're on Hosted GKE, before starting, make sure the email address associated serviceaccount/prometheus-operator created deployment.apps/prometheus-operator created ~~~ - 3. Confirm that the `prometheus-operator` has started: + {% include copy-clipboard.html %} ~~~ shell $ kubectl get deploy prometheus-operator @@ -67,6 +67,10 @@ If you're on Hosted GKE, before starting, make sure the email address associated 4. Use our [`prometheus.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/prometheus.yaml) file to create the various objects necessary to run a Prometheus instance: + {{site.data.alerts.callout_success}} + This configuration defaults to using the Kubernetes CA for authentication. + {{site.data.alerts.end}} + {% include copy-clipboard.html %} ~~~ shell $ kubectl apply \ @@ -142,7 +146,7 @@ Active monitoring helps you spot problems early, but it is also essential to sen ~~~ {{site.data.alerts.callout_danger}} - The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `altermanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen. + The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen. {{site.data.alerts.end}} 4. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts: diff --git a/_includes/v20.1/orchestration/kubernetes-scale-cluster.md b/_includes/v20.1/orchestration/kubernetes-scale-cluster.md index 64f7ca9102b..e847a6ba8d1 100644 --- a/_includes/v20.1/orchestration/kubernetes-scale-cluster.md +++ b/_includes/v20.1/orchestration/kubernetes-scale-cluster.md @@ -20,6 +20,10 @@ To do this, add a new worker node and then edit your StatefulSet configuration t ~~~ statefulset.apps/cockroachdb scaled ~~~ + + {{site.data.alerts.callout_success}} + If you aren't using the Kubernetes CA to sign certificates, you can now skip to step 6. + {{site.data.alerts.end}}
diff --git a/_includes/v20.1/orchestration/kubernetes-upgrade-cluster.md b/_includes/v20.1/orchestration/kubernetes-upgrade-cluster.md index 59ab5fb759c..f878108047a 100644 --- a/_includes/v20.1/orchestration/kubernetes-upgrade-cluster.md +++ b/_includes/v20.1/orchestration/kubernetes-upgrade-cluster.md @@ -72,7 +72,7 @@ Kubernetes knows how to carry out a safe rolling upgrade process of the Cockroac {% include copy-clipboard.html %} ~~~ sql - > SET CLUSTER SETTING cluster.preserve_downgrade_option = '19.1'; + > SET CLUSTER SETTING cluster.preserve_downgrade_option = '19.2'; ~~~ 3. Exit the SQL shell and delete the temporary pod: diff --git a/_includes/v20.1/orchestration/start-cockroachdb-local-insecure.md b/_includes/v20.1/orchestration/start-cockroachdb-local-insecure.md new file mode 100644 index 00000000000..bebb6eb3062 --- /dev/null +++ b/_includes/v20.1/orchestration/start-cockroachdb-local-insecure.md @@ -0,0 +1,83 @@ +1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml + ~~~ + + ~~~ + service/cockroachdb-public created + service/cockroachdb created + poddisruptionbudget.policy/cockroachdb-budget created + statefulset.apps/cockroachdb created + ~~~ + +2. Confirm that three pods are `Running` successfully. Note that they will not + be considered `Ready` until after the cluster has been initialized: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 0/1 Running 0 2m + cockroachdb-1 0/1 Running 0 2m + cockroachdb-2 0/1 Running 0 2m + ~~~ + +3. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pv + ~~~ + + ~~~ + NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE + pvc-52f51ecf-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-0 26s + pvc-52fd3a39-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-1 27s + pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s + ~~~ + +4. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create \ + -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml + ~~~ + + ~~~ + job.batch/cluster-init created + ~~~ + +5. Confirm that cluster initialization has completed successfully. The job should be considered successful and the Kubernetes pods should soon be considered `Ready`: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get job cluster-init + ~~~ + + ~~~ + NAME COMPLETIONS DURATION AGE + cluster-init 1/1 7s 27s + ~~~ + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cluster-init-cqf8l 0/1 Completed 0 56s + cockroachdb-0 1/1 Running 0 7m51s + cockroachdb-1 1/1 Running 0 7m51s + cockroachdb-2 1/1 Running 0 7m51s + ~~~ + +{{site.data.alerts.callout_success}} +The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to a pod/node's logs to troubleshoot, use `kubectl logs ` rather than checking the log on the persistent volume. +{{site.data.alerts.end}} diff --git a/_includes/v20.1/orchestration/start-cockroachdb-local-secure.md b/_includes/v20.1/orchestration/start-cockroachdb-local-secure.md new file mode 100644 index 00000000000..0558bcdf3b2 --- /dev/null +++ b/_includes/v20.1/orchestration/start-cockroachdb-local-secure.md @@ -0,0 +1,366 @@ +Download and modify our StatefulSet configuration, depending on how you want to sign your certificates. + +{{site.data.alerts.callout_danger}} +Some environments, such as Amazon EKS, do not support certificates signed by Kubernetes' built-in CA. In this case, use the second configuration below. +{{site.data.alerts.end}} + +- Using the Kubernetes CA: [`cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml). + + {% include copy-clipboard.html %} + ~~~ shell + $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml + ~~~ + +- Using a non-Kubernetes CA: [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml) + + {% include copy-clipboard.html %} + ~~~ shell + $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml + ~~~ + +{{site.data.alerts.callout_success}} +If you change the StatefulSet name from the default `cockroachdb`, be sure to start and end with an alphanumeric character and otherwise use lowercase alphanumeric characters, `-`, or `.` so as to comply with [CSR naming requirements](orchestrate-cockroachdb-with-kubernetes.html#csr-names). +{{site.data.alerts.end}} + +#### Initialize the cluster + +Choose the authentication method that corresponds to the StatefulSet configuration you downloaded and modified above. + +- [Kubernetes CA](#kubernetes-ca) +- [Non-Kubernetes CA](#non-kubernetes-ca) + +{{site.data.alerts.callout_success}} +The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to a pod/node's logs to troubleshoot, use `kubectl logs ` rather than checking the log on the persistent volume. +{{site.data.alerts.end}} + +##### Kubernetes CA + +1. Use the config file you downloaded to create the StatefulSet that automatically creates 3 pods, each running a CockroachDB node: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create -f cockroachdb-statefulset-secure.yaml + ~~~ + + ~~~ + serviceaccount/cockroachdb created + role.rbac.authorization.k8s.io/cockroachdb created + clusterrole.rbac.authorization.k8s.io/cockroachdb created + rolebinding.rbac.authorization.k8s.io/cockroachdb created + clusterrolebinding.rbac.authorization.k8s.io/cockroachdb created + service/cockroachdb-public created + service/cockroachdb created + poddisruptionbudget.policy/cockroachdb-budget created + statefulset.apps/cockroachdb created + ~~~ + +2. As each pod is created, it issues a Certificate Signing Request, or CSR, to have the node's certificate signed by the Kubernetes CA. You must manually check and approve each node's certificates, at which point the CockroachDB node is started in the pod. + + 1. Get the names of the `Pending` CSRs: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get csr + ~~~ + + ~~~ + NAME AGE REQUESTOR CONDITION + default.node.cockroachdb-0 1m system:serviceaccount:default:cockroachdb Pending + default.node.cockroachdb-1 1m system:serviceaccount:default:cockroachdb Pending + default.node.cockroachdb-2 1m system:serviceaccount:default:cockroachdb Pending + ... + ~~~ + + If you do not see a `Pending` CSR, wait a minute and try again. + + 2. Examine the CSR for the first pod: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl describe csr default.node.cockroachdb-0 + ~~~ + + ~~~ + Name: default.node.cockroachdb-0 + Labels: + Annotations: + CreationTimestamp: Thu, 09 Nov 2017 13:39:37 -0500 + Requesting User: system:serviceaccount:default:cockroachdb + Status: Pending + Subject: + Common Name: node + Serial Number: + Organization: Cockroach + Subject Alternative Names: + DNS Names: localhost + cockroachdb-0.cockroachdb.default.svc.cluster.local + cockroachdb-0.cockroachdb + cockroachdb-public + cockroachdb-public.default.svc.cluster.local + IP Addresses: 127.0.0.1 + 10.48.1.6 + Events: + ~~~ + + 3. If everything looks correct, approve the CSR for the first pod: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl certificate approve default.node.cockroachdb-0 + ~~~ + + ~~~ + certificatesigningrequest "default.node.cockroachdb-0" approved + ~~~ + + 4. Repeat steps 2 and 3 for the other 2 pods. + +3. Initialize the CockroachDB cluster: + + 1. Confirm that three pods are `Running` successfully. Note that they will not be considered `Ready` until after the cluster has been initialized: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 0/1 Running 0 2m + cockroachdb-1 0/1 Running 0 2m + cockroachdb-2 0/1 Running 0 2m + ~~~ + + 2. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pv + ~~~ + + ~~~ + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + pvc-9e435563-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-0 standard 51m + pvc-9e47d820-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-1 standard 51m + pvc-9e4f57f0-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-2 standard 51m + ~~~ + + 3. Use our [`cluster-init-secure.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init-secure.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create \ + -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init-secure.yaml + ~~~ + + ~~~ + job.batch/cluster-init-secure created + ~~~ + + 4. Approve the CSR for the one-off pod from which cluster initialization happens: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl certificate approve default.client.root + ~~~ + + ~~~ + certificatesigningrequest.certificates.k8s.io/default.client.root approved + ~~~ + + 5. Confirm that cluster initialization has completed successfully. The job should be considered successful and the Kubernetes pods should soon be considered `Ready`: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get job cluster-init-secure + ~~~ + + ~~~ + NAME COMPLETIONS DURATION AGE + cluster-init-secure 1/1 23s 35s + ~~~ + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cluster-init-secure-q8s7v 0/1 Completed 0 55s + cockroachdb-0 1/1 Running 0 3m + cockroachdb-1 1/1 Running 0 3m + cockroachdb-2 1/1 Running 0 3m + ~~~ + +##### Non-Kubernetes CA + +{{site.data.alerts.callout_info}} +The below steps use [`cockroach cert` commands](cockroach-cert.html) to quickly generate and sign the CockroachDB node and client certificates. Read our [Authentication](authentication.html#using-digital-certificates-with-cockroachdb) docs to learn about other methods of signing certificates. +{{site.data.alerts.end}} + +1. Create two directories: + + {% include copy-clipboard.html %} + ~~~ shell + $ mkdir certs + ~~~ + + {% include copy-clipboard.html %} + ~~~ shell + $ mkdir my-safe-directory + ~~~ + + Directory | Description + ----------|------------ + `certs` | You'll generate your CA certificate and all node and client certificates and keys in this directory. + `my-safe-directory` | You'll generate your CA key in this directory and then reference the key when generating node and client certificates. + +2. Create the CA certificate and key pair: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-ca \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +3. Create a client certificate and key pair for the root user: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-client \ + root \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +4. Upload the client certificate and key to the Kubernetes cluster as a secret: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.client.root \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.client.root created + ~~~ + +5. Create the certificate and key pair for your CockroachDB nodes: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-node \ + localhost 127.0.0.1 \ + cockroachdb-public \ + cockroachdb-public.default \ + cockroachdb-public.default.svc.cluster.local \ + *.cockroachdb \ + *.cockroachdb.default \ + *.cockroachdb.default.svc.cluster.local \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +6. Upload the node certificate and key to the Kubernetes cluster as a secret: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.node \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.node created + ~~~ + +7. Check that the secrets were created on the cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get secrets + ~~~ + + ~~~ + NAME TYPE DATA AGE + cockroachdb.client.root Opaque 3 41m + cockroachdb.node Opaque 5 14s + default-token-6qjdb kubernetes.io/service-account-token 3 4m + ~~~ + +8. Use the config file you downloaded to create the StatefulSet that automatically creates 3 pods, each running a CockroachDB node: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create -f cockroachdb-statefulset.yaml + ~~~ + + ~~~ + serviceaccount/cockroachdb created + role.rbac.authorization.k8s.io/cockroachdb created + rolebinding.rbac.authorization.k8s.io/cockroachdb created + service/cockroachdb-public created + service/cockroachdb created + poddisruptionbudget.policy/cockroachdb-budget created + statefulset.apps/cockroachdb created + ~~~ + +9. Initialize the CockroachDB cluster: + + 1. Confirm that three pods are `Running` successfully. Note that they will not be considered `Ready` until after the cluster has been initialized: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 0/1 Running 0 2m + cockroachdb-1 0/1 Running 0 2m + cockroachdb-2 0/1 Running 0 2m + ~~~ + + 2. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pv + ~~~ + + ~~~ + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + pvc-9e435563-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-0 standard 51m + pvc-9e47d820-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-1 standard 51m + pvc-9e4f57f0-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-2 standard 51m + ~~~ + + 3. Run `cockroach init` on one of the pods to complete the node startup process and have them join together as a cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl exec -it cockroachdb-0 \ + -- /cockroach/cockroach init \ + --certs-dir=/cockroach/cockroach-certs + ~~~ + + ~~~ + Cluster successfully initialized + ~~~ + + 4. Confirm that cluster initialization has completed successfully. The job should be considered successful and the Kubernetes pods should soon be considered `Ready`: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 1/1 Running 0 3m + cockroachdb-1 1/1 Running 0 3m + cockroachdb-2 1/1 Running 0 3m + ~~~ \ No newline at end of file diff --git a/_includes/v20.1/orchestration/start-cockroachdb-secure.md b/_includes/v20.1/orchestration/start-cockroachdb-secure.md index 3918e655b02..b918eca3c8f 100644 --- a/_includes/v20.1/orchestration/start-cockroachdb-secure.md +++ b/_includes/v20.1/orchestration/start-cockroachdb-secure.md @@ -1,24 +1,81 @@ -1. From your local workstation, use our [`cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it. +Download and modify our StatefulSet configuration, depending on how you want to sign your certificates. - Download [`cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml): +{{site.data.alerts.callout_danger}} +Some environments, such as Amazon EKS, do not support certificates signed by Kubernetes' built-in CA. In this case, use the second configuration below. +{{site.data.alerts.end}} + +- Using the Kubernetes CA: [`cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml). {% include copy-clipboard.html %} ~~~ shell $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml ~~~ - {{site.data.alerts.callout_danger}} - To avoid running out of memory when CockroachDB is not the only pod on a Kubernetes node, you must set memory limits explicitly. This is because CockroachDB does not detect the amount of memory allocated to its pod when run in Kubernetes. Specify this amount by adjusting `resources.requests.memory` and `resources.limits.memory` in `cockroachdb-statefulset-secure.yaml`. Their values should be identical. - - We recommend setting `cache` and `max-sql-memory` each to 1/4 of your memory allocation. For example, if you are allocating 8Gi of memory to each CockroachDB node, substitute the following values in [this line](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset-secure.yaml#L247): - {{site.data.alerts.end}} +- Using a non-Kubernetes CA: [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml) {% include copy-clipboard.html %} ~~~ shell + $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml + ~~~ + +#### Set up configuration file + +Modify the values in the StatefulSet configuration. + +1. To avoid running out of memory when CockroachDB is not the only pod on a Kubernetes node, you *must* set memory limits explicitly. This is because CockroachDB does not detect the amount of memory allocated to its pod when run in Kubernetes. In the `containers` specification, set this amount in both `resources.requests.memory` and `resources.limits.memory`. + + ~~~ + resources: + requests: + memory: "8Gi" + limits: + memory: "8Gi" + ~~~ + + We recommend setting `cache` and `max-sql-memory` each to 1/4 of the memory allocation. These are defined as placeholder percentages in the StatefulSet command that creates the CockroachDB nodes: + + ~~~ + - "exec /cockroach/cockroach start --logtostderr --certs-dir /cockroach/cockroach-certs --advertise-host $(hostname -f) --http-addr 0.0.0.0 --join cockroachdb-0.cockroachdb,cockroachdb-1.cockroachdb,cockroachdb-2.cockroachdb --cache 25% --max-sql-memory 25%" + ~~~ + + {{site.data.alerts.callout_success}} + For example, if you are allocating 8Gi of `memory` to each CockroachDB node, allocate 2Gi to `cache` and 2Gi to `max-sql-memory`. + {{site.data.alerts.end}} + + ~~~ --cache 2Gi --max-sql-memory 2Gi ~~~ - Use the file to create the StatefulSet and start the cluster: +2. In the `volumeClaimTemplates` specification, you may want to modify `resources.requests.storage` for your use case. This configuration defaults to 100Gi of disk space per pod. For more details on customizing disks for performance, see [these instructions](kubernetes-performance.html#disk-type). + + ~~~ + resources: + requests: + storage: "100Gi" + ~~~ + + {{site.data.alerts.callout_info}} + If necessary, you can [expand disk size](orchestrate-cockroachdb-with-kubernetes.html#expand-disk-size) after the cluster is live. + {{site.data.alerts.end}} + +{{site.data.alerts.callout_success}} +If you change the StatefulSet name from the default `cockroachdb`, be sure to start and end with an alphanumeric character and otherwise use lowercase alphanumeric characters, `-`, or `.` so as to comply with [CSR naming requirements](orchestrate-cockroachdb-with-kubernetes.html#csr-names). +{{site.data.alerts.end}} + +#### Initialize the cluster + +Choose the authentication method that corresponds to the StatefulSet configuration you downloaded and modified above. + +- [Kubernetes CA](#kubernetes-ca) +- [Non-Kubernetes CA](#non-kubernetes-ca) + +{{site.data.alerts.callout_success}} +The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to a pod/node's logs to troubleshoot, use `kubectl logs ` rather than checking the log on the persistent volume. +{{site.data.alerts.end}} + +##### Kubernetes CA + +1. Use the config file you downloaded to create the StatefulSet that automatically creates 3 pods, each running a CockroachDB node: {% include copy-clipboard.html %} ~~~ shell @@ -37,28 +94,6 @@ statefulset.apps/cockroachdb created ~~~ - Alternatively, if you'd rather start with a configuration file that has been customized for performance: - - 1. Download our [performance version of `cockroachdb-statefulset-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-secure.yaml): - - {% include copy-clipboard.html %} - ~~~ shell - $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-secure.yaml - ~~~ - - 2. Modify the file wherever there is a `TODO` comment. - - 3. Use the file to create the StatefulSet and start the cluster: - - {% include copy-clipboard.html %} - ~~~ shell - $ kubectl create -f cockroachdb-statefulset-secure.yaml - ~~~ - - {{site.data.alerts.callout_success}} - If you change the StatefulSet name from the default `cockroachdb`, be sure to start and end with an alphanumeric character and otherwise use lowercase alphanumeric characters, `-`, or `.` so as to comply with [CSR naming requirements](orchestrate-cockroachdb-with-kubernetes.html#csr-names). - {{site.data.alerts.end}} - 2. As each pod is created, it issues a Certificate Signing Request, or CSR, to have the node's certificate signed by the Kubernetes CA. You must manually check and approve each node's certificates, at which point the CockroachDB node is started in the pod. 1. Get the names of the `Pending` CSRs: @@ -140,7 +175,7 @@ {% include copy-clipboard.html %} ~~~ shell - $ kubectl get persistentvolumes + $ kubectl get pv ~~~ ~~~ @@ -198,6 +233,174 @@ cockroachdb-2 1/1 Running 0 3m ~~~ -{{site.data.alerts.callout_success}} -The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to a pod/node's logs to troubleshoot, use `kubectl logs ` rather than checking the log on the persistent volume. +##### Non-Kubernetes CA + +{{site.data.alerts.callout_info}} +The below steps use [`cockroach cert` commands](cockroach-cert.html) to quickly generate and sign the CockroachDB node and client certificates. Read our [Authentication](authentication.html#using-digital-certificates-with-cockroachdb) docs to learn about other methods of signing certificates. {{site.data.alerts.end}} + +1. Create two directories: + + {% include copy-clipboard.html %} + ~~~ shell + $ mkdir certs + ~~~ + + {% include copy-clipboard.html %} + ~~~ shell + $ mkdir my-safe-directory + ~~~ + + Directory | Description + ----------|------------ + `certs` | You'll generate your CA certificate and all node and client certificates and keys in this directory. + `my-safe-directory` | You'll generate your CA key in this directory and then reference the key when generating node and client certificates. + +2. Create the CA certificate and key pair: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-ca \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +3. Create a client certificate and key pair for the root user: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-client \ + root \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +4. Upload the client certificate and key to the Kubernetes cluster as a secret: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.client.root \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.client.root created + ~~~ + +5. Create the certificate and key pair for your CockroachDB nodes: + + {% include copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-node \ + localhost 127.0.0.1 \ + cockroachdb-public \ + cockroachdb-public.default \ + cockroachdb-public.default.svc.cluster.local \ + *.cockroachdb \ + *.cockroachdb.default \ + *.cockroachdb.default.svc.cluster.local \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +6. Upload the node certificate and key to the Kubernetes cluster as a secret: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.node \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.node created + ~~~ + +7. Check that the secrets were created on the cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get secrets + ~~~ + + ~~~ + NAME TYPE DATA AGE + cockroachdb.client.root Opaque 3 41m + cockroachdb.node Opaque 5 14s + default-token-6qjdb kubernetes.io/service-account-token 3 4m + ~~~ + +8. Use the config file you downloaded to create the StatefulSet that automatically creates 3 pods, each running a CockroachDB node: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create -f cockroachdb-statefulset.yaml + ~~~ + + ~~~ + serviceaccount/cockroachdb created + role.rbac.authorization.k8s.io/cockroachdb created + rolebinding.rbac.authorization.k8s.io/cockroachdb created + service/cockroachdb-public created + service/cockroachdb created + poddisruptionbudget.policy/cockroachdb-budget created + statefulset.apps/cockroachdb created + ~~~ + +9. Initialize the CockroachDB cluster: + + 1. Confirm that three pods are `Running` successfully. Note that they will not be considered `Ready` until after the cluster has been initialized: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 0/1 Running 0 2m + cockroachdb-1 0/1 Running 0 2m + cockroachdb-2 0/1 Running 0 2m + ~~~ + + 2. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pv + ~~~ + + ~~~ + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + pvc-9e435563-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-0 standard 51m + pvc-9e47d820-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-1 standard 51m + pvc-9e4f57f0-fb2e-11e9-a65c-42010a8e0fca 100Gi RWO Delete Bound default/datadir-cockroachdb-2 standard 51m + ~~~ + + 3. Run `cockroach init` on one of the pods to complete the node startup process and have them join together as a cluster: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl exec -it cockroachdb-0 \ + -- /cockroach/cockroach init \ + --certs-dir=/cockroach/cockroach-certs + ~~~ + + ~~~ + Cluster successfully initialized + ~~~ + + 4. Confirm that cluster initialization has completed successfully. The job should be considered successful and the Kubernetes pods should soon be considered `Ready`: + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl get pods + ~~~ + + ~~~ + NAME READY STATUS RESTARTS AGE + cockroachdb-0 1/1 Running 0 3m + cockroachdb-1 1/1 Running 0 3m + cockroachdb-2 1/1 Running 0 3m + ~~~ \ No newline at end of file diff --git a/_includes/v20.1/orchestration/test-cluster-secure.md b/_includes/v20.1/orchestration/test-cluster-secure.md index 047dae43f8e..9cb18c8bf79 100644 --- a/_includes/v20.1/orchestration/test-cluster-secure.md +++ b/_includes/v20.1/orchestration/test-cluster-secure.md @@ -1,7 +1,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely with the `cockroach` binary inside it, get a shell into the pod, and then start the built-in SQL client.
-1. From your local workstation, use our [`client-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/client-secure.yaml) file to launch a pod and keep it running indefinitely: +- Using the Kubernetes CA: [`client-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/client-secure.yaml) {% include copy-clipboard.html %} ~~~ shell @@ -9,15 +9,23 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/client-secure.yaml ~~~ - ~~~ - pod/cockroachdb-client-secure created +- Using a non-Kubernetes CA: [`client.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/client.yaml) + + {% include copy-clipboard.html %} + ~~~ shell + $ kubectl create \ + -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml ~~~ {{site.data.alerts.callout_info}} The pod uses the `root` client certificate created earlier to initialize the cluster, so there's no CSR approval required. If you issue client certificates for other users, however, be sure your SQL usernames contain only lowercase alphanumeric characters, `-`, or `.` so as to comply with [CSR naming requirements](orchestrate-cockroachdb-with-kubernetes.html#csr-names). {{site.data.alerts.end}} -2. Get a shell into the pod and start the CockroachDB [built-in SQL client](cockroach-sql.html): + ~~~ + pod/cockroachdb-client-secure created + ~~~ + +1. Get a shell into the pod and start the CockroachDB [built-in SQL client](cockroach-sql.html): {% include copy-clipboard.html %} ~~~ shell @@ -42,7 +50,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely root@cockroachdb-public:26257/defaultdb> ~~~ -3. Run some basic [CockroachDB SQL statements](learn-cockroachdb-sql.html): +2. Run some basic [CockroachDB SQL statements](learn-cockroachdb-sql.html): {% include copy-clipboard.html %} ~~~ sql @@ -71,7 +79,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely (1 row) ~~~ -4. [Create a user with a password](create-user.html#create-a-user-with-a-password): +3. [Create a user with a password](create-user.html#create-a-user-with-a-password): {% include copy-clipboard.html %} ~~~ sql @@ -80,7 +88,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely You will need this username and password to access the Admin UI later. -5. Exit the SQL shell and pod: +4. Exit the SQL shell and pod: {% include copy-clipboard.html %} ~~~ sql diff --git a/v19.2/orchestrate-a-local-cluster-with-kubernetes-insecure.md b/v19.2/orchestrate-a-local-cluster-with-kubernetes-insecure.md index 31cadc5036f..24ffa16285c 100644 --- a/v19.2/orchestrate-a-local-cluster-with-kubernetes-insecure.md +++ b/v19.2/orchestrate-a-local-cluster-with-kubernetes-insecure.md @@ -29,7 +29,7 @@ To start your CockroachDB cluster, you can either use our StatefulSet configurat
-{% include {{ page.version.version }}/orchestration/start-cockroachdb-insecure.md %} +{% include {{ page.version.version }}/orchestration/start-cockroachdb-local-insecure.md %}
diff --git a/v19.2/orchestrate-a-local-cluster-with-kubernetes.md b/v19.2/orchestrate-a-local-cluster-with-kubernetes.md index bd86795ee06..383e09ff283 100644 --- a/v19.2/orchestrate-a-local-cluster-with-kubernetes.md +++ b/v19.2/orchestrate-a-local-cluster-with-kubernetes.md @@ -31,7 +31,7 @@ To start your CockroachDB cluster, you can either use our StatefulSet configurat
-{% include {{ page.version.version }}/orchestration/start-cockroachdb-secure.md %} +{% include {{ page.version.version }}/orchestration/start-cockroachdb-local-secure.md %}
diff --git a/v19.2/orchestrate-cockroachdb-with-kubernetes.md b/v19.2/orchestrate-cockroachdb-with-kubernetes.md index 9dc6796ed26..0f226eda720 100644 --- a/v19.2/orchestrate-cockroachdb-with-kubernetes.md +++ b/v19.2/orchestrate-cockroachdb-with-kubernetes.md @@ -30,7 +30,7 @@ Feature | Description [pod](http://kubernetes.io/docs/user-guide/pods/) | A pod is a group of one of more Docker containers. In this tutorial, each pod will run on a separate Kubernetes node and include one Docker container running a single CockroachDB node. You'll start with 3 pods and grow to 4. [StatefulSet](http://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/) | A StatefulSet is a group of pods treated as stateful units, where each pod has distinguishable network identity and always binds back to the same persistent storage on restart. StatefulSets are considered stable as of Kubernetes version 1.9 after reaching beta in version 1.5. [persistent volume](http://kubernetes.io/docs/user-guide/persistent-volumes/) | A persistent volume is a piece of networked storage (Persistent Disk on GCE, Elastic Block Store on AWS) mounted into a pod. The lifetime of a persistent volume is decoupled from the lifetime of the pod that's using it, ensuring that each CockroachDB node binds back to the same storage on restart.

This tutorial assumes that dynamic volume provisioning is available. When that is not the case, [persistent volume claims](http://kubernetes.io/docs/user-guide/persistent-volumes/#persistentvolumeclaims) need to be created manually. -[CSR](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/) | A CSR, or Certificate Signing Request, is a request to have a TLS certificate signed by the Kubernetes cluster's built-in CA. As each pod is created, it issues a CSR for the CockroachDB node running in the pod, which must be manually checked and approved. The same is true for clients as they connect to the cluster. Note that Kubernetes environments that don't support CSRs, such as Amazon EKS, can [use a different certificate authority](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/bring-your-own-certs) than the one Kubernetes uses. +[CSR](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/) | A CSR, or certificate signing request, is a request to have a TLS certificate signed by the Kubernetes cluster's built-in CA. As each pod is created, it issues a CSR for the CockroachDB node running in the pod, which must be manually checked and approved. The same is true for clients as they connect to the cluster. Note that Kubernetes environments that don't support CSRs, such as Amazon EKS, can use a different certificate authority than the one Kubernetes uses. [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) | RBAC, or Role-Based Access Control, is the system Kubernetes uses to manage permissions within the cluster. In order to take an action (e.g., `get` or `create`) on an API resource (e.g., a `pod` or `CSR`), the client must have a `Role` that allows it to do so. This tutorial creates the RBAC resources necessary for CockroachDB to create and access certificates. ### Limitations @@ -51,10 +51,8 @@ CSR names contain the StatefulSet or Helm chart `name`, so if you customize that To start your CockroachDB cluster, you can either use our StatefulSet configuration and related files directly, or you can use the [Helm](https://helm.sh/) package manager for Kubernetes to simplify the process. -{{site.data.alerts.callout_danger}} +{{site.data.alerts.callout_info}} Secure CockroachDB deployments on Amazon EKS via Helm are [not yet supported](https://github.com/cockroachdb/cockroach/issues/38847). In the meantime, use a StatefulSet configuration to deploy on EKS. - -Note that when running on Amazon EKS, certificates signed by Kubernetes' built-in CA are not supported, so use [this configuration file](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml) instead of `cockroachdb-statefulset-secure.yaml` referenced below. Follow the steps at the top of the file to load externally created certificates into Kubernetes and initialize the CockroachDB cluster. {{site.data.alerts.end}}
@@ -316,7 +314,7 @@ To shut down the CockroachDB cluster: pod "cockroachdb-client-secure" deleted ~~~ -3. Get the names of the CSRs for the cluster: +3. Get the names of any CSRs for the cluster: {% include copy-clipboard.html %} ~~~ shell @@ -352,7 +350,7 @@ To shut down the CockroachDB cluster: ~~~
-4. Delete the CSRs that you created: +4. Delete any CSRs that you created:
{% include copy-clipboard.html %} diff --git a/v20.1/orchestrate-a-local-cluster-with-kubernetes-insecure.md b/v20.1/orchestrate-a-local-cluster-with-kubernetes-insecure.md index 31cadc5036f..24ffa16285c 100644 --- a/v20.1/orchestrate-a-local-cluster-with-kubernetes-insecure.md +++ b/v20.1/orchestrate-a-local-cluster-with-kubernetes-insecure.md @@ -29,7 +29,7 @@ To start your CockroachDB cluster, you can either use our StatefulSet configurat
-{% include {{ page.version.version }}/orchestration/start-cockroachdb-insecure.md %} +{% include {{ page.version.version }}/orchestration/start-cockroachdb-local-insecure.md %}
diff --git a/v20.1/orchestrate-a-local-cluster-with-kubernetes.md b/v20.1/orchestrate-a-local-cluster-with-kubernetes.md index bd86795ee06..383e09ff283 100644 --- a/v20.1/orchestrate-a-local-cluster-with-kubernetes.md +++ b/v20.1/orchestrate-a-local-cluster-with-kubernetes.md @@ -31,7 +31,7 @@ To start your CockroachDB cluster, you can either use our StatefulSet configurat
-{% include {{ page.version.version }}/orchestration/start-cockroachdb-secure.md %} +{% include {{ page.version.version }}/orchestration/start-cockroachdb-local-secure.md %}
diff --git a/v20.1/orchestrate-cockroachdb-with-kubernetes.md b/v20.1/orchestrate-cockroachdb-with-kubernetes.md index 9dc6796ed26..0f226eda720 100644 --- a/v20.1/orchestrate-cockroachdb-with-kubernetes.md +++ b/v20.1/orchestrate-cockroachdb-with-kubernetes.md @@ -30,7 +30,7 @@ Feature | Description [pod](http://kubernetes.io/docs/user-guide/pods/) | A pod is a group of one of more Docker containers. In this tutorial, each pod will run on a separate Kubernetes node and include one Docker container running a single CockroachDB node. You'll start with 3 pods and grow to 4. [StatefulSet](http://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/) | A StatefulSet is a group of pods treated as stateful units, where each pod has distinguishable network identity and always binds back to the same persistent storage on restart. StatefulSets are considered stable as of Kubernetes version 1.9 after reaching beta in version 1.5. [persistent volume](http://kubernetes.io/docs/user-guide/persistent-volumes/) | A persistent volume is a piece of networked storage (Persistent Disk on GCE, Elastic Block Store on AWS) mounted into a pod. The lifetime of a persistent volume is decoupled from the lifetime of the pod that's using it, ensuring that each CockroachDB node binds back to the same storage on restart.

This tutorial assumes that dynamic volume provisioning is available. When that is not the case, [persistent volume claims](http://kubernetes.io/docs/user-guide/persistent-volumes/#persistentvolumeclaims) need to be created manually. -[CSR](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/) | A CSR, or Certificate Signing Request, is a request to have a TLS certificate signed by the Kubernetes cluster's built-in CA. As each pod is created, it issues a CSR for the CockroachDB node running in the pod, which must be manually checked and approved. The same is true for clients as they connect to the cluster. Note that Kubernetes environments that don't support CSRs, such as Amazon EKS, can [use a different certificate authority](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/bring-your-own-certs) than the one Kubernetes uses. +[CSR](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/) | A CSR, or certificate signing request, is a request to have a TLS certificate signed by the Kubernetes cluster's built-in CA. As each pod is created, it issues a CSR for the CockroachDB node running in the pod, which must be manually checked and approved. The same is true for clients as they connect to the cluster. Note that Kubernetes environments that don't support CSRs, such as Amazon EKS, can use a different certificate authority than the one Kubernetes uses. [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) | RBAC, or Role-Based Access Control, is the system Kubernetes uses to manage permissions within the cluster. In order to take an action (e.g., `get` or `create`) on an API resource (e.g., a `pod` or `CSR`), the client must have a `Role` that allows it to do so. This tutorial creates the RBAC resources necessary for CockroachDB to create and access certificates. ### Limitations @@ -51,10 +51,8 @@ CSR names contain the StatefulSet or Helm chart `name`, so if you customize that To start your CockroachDB cluster, you can either use our StatefulSet configuration and related files directly, or you can use the [Helm](https://helm.sh/) package manager for Kubernetes to simplify the process. -{{site.data.alerts.callout_danger}} +{{site.data.alerts.callout_info}} Secure CockroachDB deployments on Amazon EKS via Helm are [not yet supported](https://github.com/cockroachdb/cockroach/issues/38847). In the meantime, use a StatefulSet configuration to deploy on EKS. - -Note that when running on Amazon EKS, certificates signed by Kubernetes' built-in CA are not supported, so use [this configuration file](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml) instead of `cockroachdb-statefulset-secure.yaml` referenced below. Follow the steps at the top of the file to load externally created certificates into Kubernetes and initialize the CockroachDB cluster. {{site.data.alerts.end}}
@@ -316,7 +314,7 @@ To shut down the CockroachDB cluster: pod "cockroachdb-client-secure" deleted ~~~ -3. Get the names of the CSRs for the cluster: +3. Get the names of any CSRs for the cluster: {% include copy-clipboard.html %} ~~~ shell @@ -352,7 +350,7 @@ To shut down the CockroachDB cluster: ~~~
-4. Delete the CSRs that you created: +4. Delete any CSRs that you created:
{% include copy-clipboard.html %}