Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom resource requests/limits, extra binds, and extra env for control plane static pod components #1463

Merged
merged 26 commits into from
Aug 3, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
0b84f14
Initial attempt to add customization of cpu/memory limits/requests an…
Oats87 Jul 27, 2021
cc39afe
goimports
Oats87 Jul 27, 2021
58f5b6d
fix import cycle
Oats87 Jul 27, 2021
80bb702
Don't parse if the string is empty
Oats87 Jul 27, 2021
cd42aed
Only set request/limits if defined
Oats87 Jul 27, 2021
445991b
fix formatting
Oats87 Jul 27, 2021
e2f744a
fix formatting
Oats87 Jul 27, 2021
a691a0a
change to mount from bind
Oats87 Jul 27, 2021
1d3e485
Merge branch 'issues/rke2/1046' of github.com:Oats87/rke2 into issues…
Oats87 Jul 27, 2021
0f6e9e2
add rudimentary input validation for env and volumes
Oats87 Jul 27, 2021
6e1eed4
optimization corrections to formatting
Oats87 Jul 27, 2021
0b850dd
better parsing of requests/limits
Oats87 Jul 30, 2021
454c257
minor logic correction
Oats87 Jul 30, 2021
08bccf0
more optimization
Oats87 Jul 30, 2021
fe6b8d4
gofmt
Oats87 Jul 30, 2021
0587227
more optimization
Oats87 Jul 30, 2021
eed8dd5
import cycle correction
Oats87 Jul 30, 2021
c1d8b47
move default
Oats87 Jul 30, 2021
9c90fd5
fix import cycle
Oats87 Jul 30, 2021
467cc38
remove unnecessary import
Oats87 Jul 30, 2021
ad5af17
missed import issue
Oats87 Jul 30, 2021
4be20c1
Add RO/RW options to extraMount
Oats87 Aug 3, 2021
5b40c1d
Add basic documentation for extra volumes, extra env, and resource re…
Oats87 Aug 3, 2021
680f7d6
Optimize volume mount ro/rw detection and use SplitN instead of Split…
Oats87 Aug 3, 2021
bc172b6
Fix imports
Oats87 Aug 3, 2021
2e1d275
Address PR comments
Oats87 Aug 3, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
64 changes: 64 additions & 0 deletions docs/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,3 +156,67 @@ cloud-provider-config: "/etc/rancher/rke2/cloud.conf"
4. [Install](install/methods.md) RKE2 normally (most likely in an [airgapped](install/airgap.md) capacity)

5. Validate successful installation by confirming the existence of AWS metadata on cluster node labels with `kubectl get nodes --show-labels`

## Control Plane Component Resource Requests/Limits

The following options are available under the `server` sub-command for RKE2. The options allow for specifying CPU requests and limits for the control plane components within RKE2.

```
--control-plane-resource-requests value (components) Control Plane resource requests [$RKE2_CONTROL_PLANE_RESOURCE_REQUESTS]
--control-plane-resource-limits value (components) Control Plane resource limits [$RKE2_CONTROL_PLANE_RESOURCE_LIMITS]
```

Values are a comma-delimited list of `[controlplane-component]-(cpu|memory)=[desired-value]`. The possible values for `controlplane-component` are:
```
kube-apiserver
kube-scheduler
kube-controller-manager
kube-proxy
etcd
cloud-controller-manager
```

Thus, an example `--control-plane-resource-requests` or `--control-plane-resource-limits` value may look like:

```
kube-apiserver-cpu=500m,kube-apiserver-memory=512MiB,kube-scheduler-cpu=250m,kube-scheduler-memory=512MiB,etcd-cpu=1000m
```

The unit values for CPU/memory are identical to Kubernetes resource units (See: [Resource Limits in Kubernetes](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#resource-units-in-kubernetes))

## Extra Control Plane Component Volume Mounts

The following options are available under the `server` sub-command for RKE2. These options specify host-path mounting of directories from the node filesystem into the static pod component that corresponds to the prefixed name.

```
--kube-apiserver-extra-mount value (components) kube-apiserver extra volume mounts [$RKE2_KUBE_APISERVER_EXTRA_MOUNT]
--kube-scheduler-extra-mount value (components) kube-scheduler extra volume mounts [$RKE2_KUBE_SCHEDULER_EXTRA_MOUNT]
--kube-controller-manager-extra-mount value (components) kube-controller-manager extra volume mounts [$RKE2_KUBE_CONTROLLER_MANAGER_EXTRA_MOUNT]
--kube-proxy-extra-mount value (components) kube-proxy extra volume mounts [$RKE2_KUBE_PROXY_EXTRA_MOUNT]
--etcd-extra-mount value (components) etcd extra volume mounts [$RKE2_ETCD_EXTRA_MOUNT]
--cloud-controller-manager-extra-mount value (components) cloud-controller-manager extra volume mounts [$RKE2_CLOUD_CONTROLLER_MANAGER_EXTRA_MOUNT]
```

### RW Host Path Volume Mount
`/source/volume/path/on/host:/destination/volume/path/in/staticpod`

### RO Host Path Volume Mount
In order to mount a volume as read only, append `:ro` to the end of the volume mount.
`/source/volume/path/on/host:/destination/volume/path/in/staticpod:ro`

In order to specify multiple volume mounts for the same component, specify the flag multiple times.

## Extra Control Plane Component Environment Variables

The following options are available under the `server` sub-command for RKE2. These options specify additional environmennt variables in standard format i.e. `KEY=VALUE` for the static pod component that corresponds to the prefixed name.

```
--kube-apiserver-extra-env value (components) kube-apiserver extra environment variables [$RKE2_KUBE_APISERVER_EXTRA_ENV]
--kube-scheduler-extra-env value (components) kube-scheduler extra environment variables [$RKE2_KUBE_SCHEDULER_EXTRA_ENV]
--kube-controller-manager-extra-env value (components) kube-controller-manager extra environment variables [$RKE2_KUBE_CONTROLLER_MANAGER_EXTRA_ENV]
--kube-proxy-extra-env value (components) kube-proxy extra environment variables [$RKE2_KUBE_PROXY_EXTRA_ENV]
--etcd-extra-env value (components) etcd extra environment variables [$RKE2_ETCD_EXTRA_ENV]
--cloud-controller-manager-extra-env value (components) cloud-controller-manager extra environment variables [$RKE2_CLOUD_CONTROLLER_MANAGER_EXTRA_ENV]
```

In order to specify multiple environment variables for the same component, specify the flag multiple times.