[Rancher2] Documentation for RKE2 provisioning

[sowmyav27]

Documentation for RKE2 provisioning is incomplete / needs to be reviewed for RKE2 being GA in 2.6.5.

General Documentation

• "Launching Kubernetes with Rancher" section
  • Associated PR Cluster Provisioning: added RKE2 section docs#4075
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters
  • Section for the most part is written for RKE1 with RKE2 being limited to the "Changes in Rancher v2.6" section (https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/#changes-in-rancher-v2-6).
    • @btat & @jtravee, what are your thoughts on making RKE1 vs. RKE2 more prominent in this section? For example, section on Terraform there needs to link different resource for RKE2 (https://registry.terraform.io/providers/rancher/rancher2/latest/docs/resources/cluster_v2)
• "Launching Kubernetes on New Nodes in an Infrastructure Provider" section
  • Associated PR Cluster Provisioning: added RKE2 section docs#4075
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/node-pools/
  • Mostly the same as above, this section is mostly RKE1-specific with some RKE2 notes scattered around it.
    • @btat & @jtravee, same as above: what are your thoughts on making RKE1 vs. RKE2 more prominent in this section?
• Specific infrastructure providers
  • Amazon EC2 (the same approach as Azure, see details there)
    • Current PR for reference: Launch RKE2 + EC2 docs#4084
  • DigitalOcean (the same approach as Azure, see details there)
    • Current PR for reference: Launch RKE2 + DigitalOcean docs#4085
  • Azure
    • Current docs links:
      • https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/node-pools/azure/
    • Current PR for reference: Launch RKE2 + Azure docs#4066
    • For RKE1 node template configuration is not too verbose. Is it due to RKE1 showing limited options or more due to the documentation approach not being to document all options? For RKE2 per the linked PR the approach seems to be a bit more in-depth: https://github.com/rancher/docs/pull/4066/files#diff-ea7f24f91b5e6cb6d4fca419a9bb769b1ddc9fa124bb0b9751c49d44022312a1. @btat
  • Nutanix (no custom RKE2 UI, handled the same way as any other node driver - autogenerated UI based on available fields)
  • vSphere (the same approach as Azure, see details there)
    • Current PR for reference: Launch RKE2 + vSphere docs#4083
• RKE2 Configuration Reference
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-admin/editing-clusters/rke2-config-reference/
  • "(Tech Preview)" note needs to be removed (stating it requires 2.6.5)
    • Associated PR: Remove RKE2 tech preview verbiage docs#4067
  • Cluster Config File (explaining "Edit as YAML") section probably needs to be expanded as it's rather vague
    • Associated PR: Update RKE2 config ref YAML options docs#4086
  • Contents to be reviewed for correctness/completeness in general
• Behavior Differences Between RKE1 and RKE2
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/behavior-differences-between-rke1-and-rke2/
  • @thedadams & @Oats87, please review it and determine if the information there is still applicable for 2.6.5 and if there are any corrections/additions needed

Added as of 5/9/22

• Upgrading and Rolling Back Kubernetes
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-admin/upgrading-kubernetes/#enabling-draining-nodes-during-upgrades-from-the-rancher-ui
  • Link above is RKE1-specific, with some section generally applicable to RKE2. Suggested creating a separate page for RKE2 clusters.
• Nodes and Node Pools
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-admin/nodes/
  • Link above covers different types of clusters, with the assumption that custom and node-provisioned clusters are RKE1. Probably need to add RKE2 specifics.
• Cluster Templates
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/admin-settings/rke-templates/
  • Very much RKE1-specific, many child pages too.
  • This needs to be created for RKE2 from scratch.
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/admin-settings/cluster-templates/
  • The "Cluster Templates" page covers RKE2 Cluster Templates well enough, though not to the same level of detail as RKE Templates section referenced above (strikethrough).
    • @btat @jtravee , please log enhancement to add more details to RKE2 Cluster Templates section to be at the similar level of details as RKE Templates. This is not critical for 2.6.5 release though.
      • Billy: Enhance RKE2 Cluster Templates docs #52 created. Marking this task done in the context of the 2.6.5 release.

Major Features

• etcd snapshot support (with options) - RKE2 Provisioning: Day 2 Ops - On Demand/Recurring Etcd Snapshot and Restore with Options rancher#34417
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-admin/backing-up-etcd/
    • https://rancher.com/docs/rancher/v2.6/en/cluster-admin/restoring-etcd/
  • Links above are RKE1-specific.
    • @btat & @jtravee, looking for guidance on how to best incorporate RKE2-specific information there. There is a fair amount of difference as far as I know, so maybe it's best to have separate pages for RKE2-provisioned clusters snapshot/restore.
  • Engineering expert: @Oats87
    • @Oats87, please review the current documentation linked above and determine how much is the same/different vs. what it should be for RKE2.
  • Associated PR [WIP] Add RKE2 backup and restore #502
• On-demand certificate rotation - Support for RKE2 provisioned cluster certificate rotation rancher#35343
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-admin/certificate-rotation/
  • Link above is RKE1-specific, but at a glance seems to be fully applicable to RKE2.
  • Associated PR: Update cert rotation for RKE2 docs#4087
  • Engineering expert: @paynejacob
    • @paynejacob, please review and confirm no RKE2-specific updates are needed
• Rancher-managed encryption keys (with rotation) - Support for RKE2 secrets encryption keys rotation rancher#35436
  • Current docs links:
    • No Rancher documentation found; RKE1 documentation exists (https://rancher.com/docs/rke/latest/en/config-options/secrets-encryption/), but doesn't cover Rancher functionality around this.
      • @btat & @jtravee, please help with locating Rancher-specific documentation for this feature
        • Does not exist, will need to be created.
  • Overall, need to add a note that as of 2.6.5 encryption keys rotation is not supported (however note encryption key rotation is always enabled in RKE2 - Can't turn off secrets encryption rke2#850)
  • Associated PR: Adding page for enabling RKE1/RKE2 encryption key rotation docs#4230
• SSH to RKE2 nodes from Rancher CLI - Support SSH for V2 (RKE2) provisioned nodes rancher#35203
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cli/
  • This functionality is the same as for RKE1, so no need to update any documentation, marked as done.

Added as of 5/9/22

• Drain nodes on scale down- "Drain Before Delete" support for RKE2 machine pools rancher#35274
  • Current docs links:
    • Could not find docs specifically mentioning this feature for RKE1 or RKE2
  • This should probably be added to "Nodes and Node Pools" () section mentioned in "General Documentation" section.
  • Engineering expert: @jakefhyde
• Self-healing node pools- Support for automatic replacement of unreachable nodes in RKE2 machine pool rancher#35275
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/node-pools/#about-node-auto-replace
  • This is for RKE1 and the whole docs page is listed in "General Documentation" to be updated for RKE2.
  • Engineering expert: @paynejacob
• vSphere version pinning- vSphere version pinning for RKE2 clusters rancher#35474
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere/out-of-tree/
  • This documentation seems to be RKE1-specific. For RKE2, vSphere Cloud Provider must be enabled and configured at cluster creation and vSphere charts are installed automatically (shouldn't follow the steps to install them from Apps & Marketplace). CPI/CSI charts are tied to selected k8s version and are automatically upgraded as needed with k8s version is upgraded for the cluster.
    • @slickwarren, since you were testing the linked issue, could you please confirm this is the actual behavior (in terms for setting up vSphere differences for RKE2 vs. the documentation linked)
• Terraform provider updates- rancher2_cluster_v2 resource updates for RKE2 <-> RKE1 parity enhancements terraform-provider-rancher2#799
  • Terraform docs are separate and were updated as part of implementing these features in Terraform.
• IPv6 dual-stack (front-end addition) - Improved IPv6 dual-stack support on the UI for RKE2 provisioning dashboard#4550
  • Current docs links:
    • No Rancher documentation found; RKE1 documentation exists (https://rancher.com/docs/rke/latest/en/config-options/dual-stack/), but doesn't cover Rancher functionality around this
  • IPv6 should be mentioned on https://rancher.com/docs/rancher/v2.6/en/cluster-admin/editing-clusters/rke2-config-reference/#container-network-provider, specifics as per the description on Improved IPv6 dual-stack support on the UI for RKE2 provisioning dashboard#4550. Note that IPv6 dual-stack is supported for CNIs other than Cilium and Multus,Cilium; it's just Cilium requires additional parameter to be set for which UI provides a checkbox.
  • Associated PR: Add dual-stack to RKE2 config docs#4089
• Scale down individual node - Ability to scale down individual node(s) for RKE2-provisioned clusters dashboard#4446
  • Current docs links:
    • https://rancher.com/docs/rancher/v2.6/en/cluster-admin/nodes/#deleting-a-node
  • There shouldn't be anything to do for this specifically as "Nodes and Node Pools" section to be created for RKE2 per "General Documentation" section should contain "Deleting a Node" section explaining this feature.

BELOW ARE TO BE COMPLETED

Windows:

• Windows - CIS Scans - https://github.com/rancher/windows/issues/107
• Windows - Istio - https://github.com/rancher/windows/issues/108
• Windows - CNI - Calico - PNI Support - PNI for Calico on Windows on RKE2 windows#106
• Windows - RKE2 upgrade - Upgrade k8s on Windows Nodes windows#76
• Windows - Logging / Alerting / Monitoring

[btat]

Reopening as this got automatically closed through a keyword in rancher/docs#4075

[slickwarren]

vSphere version pinning- https://github.com/rancher/rancher/issues/35474
Current docs links:
https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere/out-of-tree/
This documentation seems to be RKE1-specific. For RKE2, vSphere Cloud Provider must be enabled and configured at cluster creation and vSphere charts are installed automatically (shouldn't follow the steps to install them from Apps & Marketplace). CPI/CSI charts are tied to selected k8s version and are automatically upgraded as needed with k8s version is upgraded for the cluster.
@slickwarren, since you were testing the linked issue, could you please confirm this is the actual behavior (in terms for setting up vSphere differences for RKE2 vs. the documentation linked)

yes I agree, this would need to be updated for rke2 and Sowmya's post is correct. for rke2, you would need to follow the instructions wen selecting the vsphere cloud provider option when provisioning the cluster, which will install the CPI/CSI charts during provisioning. Manually installing the charts after the cluster is in an active state is not supported for rke2.

I also noticed the docs have 6.7u3 listed as supported, but that has also been deprecated.