diff --git a/_content/operatingsystemmanager/master/img/operating-system-manager-logo.png b/_content/operatingsystemmanager/master/img/operating-system-manager-logo.png index fd33d0862..730a04c19 100644 Binary files a/_content/operatingsystemmanager/master/img/operating-system-manager-logo.png and b/_content/operatingsystemmanager/master/img/operating-system-manager-logo.png differ diff --git a/_content/operatingsystemmanager/v1.0/img/operating-system-manager-logo.png b/_content/operatingsystemmanager/v1.0/img/operating-system-manager-logo.png index fd33d0862..730a04c19 100644 Binary files a/_content/operatingsystemmanager/v1.0/img/operating-system-manager-logo.png and b/_content/operatingsystemmanager/v1.0/img/operating-system-manager-logo.png differ diff --git a/content/operatingsystemmanager/_index.en.md b/content/operatingsystemmanager/_index.en.md new file mode 100644 index 000000000..397b3ce0a --- /dev/null +++ b/content/operatingsystemmanager/_index.en.md @@ -0,0 +1,36 @@ ++++ +title = "OperatingSystemManager" +date = 2022-08-20T12:00:00+02:00 +weight = 6 +description = "Learn how you can use Kubermatic OperatingSystemManager to manage and customize worker node configurations for more granular control over your kubernetes clusters." ++++ + +![OperatingSystemManager logo](/img/operatingsystemmanager/common/operating-system-manager-logo.png) + +## What is OperatingSystemManager? + +[Operating System Manager (OSM)][operating-system-manager] is an open source project by Kubermatic, it is responsible for creating and managing the required configurations for worker nodes in a Kubernetes cluster. It decouples operating system configurations into dedicated and isolable resources for better modularity and maintainability. + +These isolated and extensible resources allow a high degree of customization which allows users to modify the worker node configurations to suit their use cases. This is useful for hybrid, edge, and air-gapped environments. + +Configurations for worker nodes comprise of set of scripts used to prepare the node, install packages, configure networking, storage etc. These configurations prepare the nodes for running `kubelet`. + +## Problem Statement + +[Machine-Controller][machine-controller] is used to manage the worker nodes in KubeOne clusters. It depends on user-data plugins to generate the required configurations for worker nodes. Each operating system requires its own user-data plugin. These configs are then injected into the worker nodes using provisioning utilities such as [cloud-init](https://cloud-init.io) or [ignition](https://coreos.github.io/ignition). Eventually the nodes are bootstrapped to become a part of a kubernetes cluster. + +This has been the norm till [machine-controller v1.54.0]() and it works as expected. Although over time, it has been observed that this workflow has certain limitations. + +### Machine Controller Limitations + +- Machine Controller expects **ALL** the supported user-data plugins to exist and be ready. User might only be interested in a subset of the available operating systems. For example, user might only want to work with `ubuntu`. +- The user-data plugins have templates defined [in-code](https://github.com/kubermatic/machine-controller/blob/v1.54.0/pkg/userdata/ubuntu/provider.go#L136). Which is not ideal since code changes are required to update those templates. Then those code changes need to become a part of the subsequent releases for machine-controller and KubeOne. So we need a complete release cycle to ship those changes to customers. +- Managing configs for multiple cloud providers, OS flavors and OS versions, adds a lot of complexity and redundancy in machine-controller. +- Since the templates are defined in-code, there is no way for an end user to customize them to suit their use-cases. +- Each cloud provider sets some sort of limits for the size of `user-data`, machine won't be created in case of non-compliance. For example, at the time of writing this, AWS has set a [hard limit of 16KB](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-add-user-data.html). +- Better support for air-gapped environments is required. + +Operating System Manager was created to overcome these limitations. + +[machine-controller]: https://github.com/kubermatic/machine-controller +[operating-system-manager]: https://github.com/kubermatic/operating-system-manager diff --git a/content/operatingsystemmanager/architecture/_index.en.md b/content/operatingsystemmanager/architecture/_index.en.md new file mode 100644 index 000000000..e9bc8c5de --- /dev/null +++ b/content/operatingsystemmanager/architecture/_index.en.md @@ -0,0 +1,49 @@ ++++ +title = "Architecture" +description = "Learn about the architecture of Kubermatic OperatingSystemManager" +date = 2022-08-20T12:00:00+02:00 +weight = 6 ++++ + +## Architecture + +With an aim to make machine configurations extendable, OSM introduces the following new resources as [Kubernetes Custom Resource Definitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/): + +### OperatingSystemProfile + +A resource that contains scripts for bootstrapping and provisioning the worker nodes, along with information about what operating systems and versions are supported for given scripts. Additionally, OSPs support templating so you can include some information from MachineDeployment or the OSM deployment itself. + +OSM comes with a set of default OSPs for various supported operating systems and cloud provider combinations. These OSPs are installed by OSM itself. End users can create custom OSPs as well to fit their own use-cases. + +OSPs are immutable by design and any modifications to an existing OSP requires a version bump in `.spec.version`. + +### OperatingSystemConfig + +Immutable resource that contains the **actual configurations** that are going to be used to bootstrap and provision the worker nodes. It is a subset of OperatingSystemProfile. OperatingSystemProfile is a template while OperatingSystemConfig is an instance rendered with data from OperatingSystemProfile, MachineDeployment, and flags provided at OSM command-line level. + +OperatingSystemConfigs have a 1-to-1 relation with the MachineDeployment. A dedicated controller watches the MachineDeployments and generates the OSCs in `kube-system`(can be overridden). + +### Kubernetes Secrets + +The generated OperatingSystemConfigs are then transformed into configurations accepted by cloud-init or ignition provisioning utilities. They are persisted as [kubernetes secrets](https://kubernetes.io/docs/concepts/configuration/secret/) in `cloud-init-settings` namespace in the cluster. + +The following secrets are generated by OSM: + +1. **Bootstrap**: Configuration used for initially setting up the machine and fetching the provisioning configuration. +2. **Provisioning**: Configuration with the actual `cloud-config` that is used to provision the worker machine. + +The significance behind using a dedicated namespace i.e. `cloud-init-settings` is that the secret that holds the `provisioning` configuration is dynamically fetched during the `bootstrap` phase. Using a dedicated namespace isolates these secrets and limits the access area. + +![Architecture](/img/operatingsystemmanager/master/osm-kubernetes.png?classes=shadow,border "Integration with Machine Controller") + +## Single vs management/worker cluster mode + +Conventionally OSM operates within a single cluster and expects all of the required resources like machine controller, MachineDeployments etc. to exist within the same cluster. + +Along with that, OSM also supports environments where workloads are divided into management and worker clusters. This is useful since it helps with completely abstracting away OSM from the users of worker cluster; OSM will be running in the management cluster. + +To use management/worker cluster mode, simply pass on the kubeconfig for management cluster using `kubeconfig` and worker cluster using the `worker-cluster-kubeconfig` flags at OSM level. With this topology the OSP and OSC exist within the management cluster while only the bootstrap and provisioning secrets are created in the worker clusters. + +## Integration with machine-controller + +When a MachineDeployment is created, OSM reconciles it and generates the corresponding OSC and secrets based on the selected OSP. The OSP to be used for a MachineDeployment is specified using `k8c.io/operating-system-profile` annotation on the MachineDeployment. Machine controller waits for the bootstrapping and provisioning secrets to become available. Once they are made available by OSM, machine controller will extract the configurations from those secrets and propagate them as `user-data` for the machines that are to be provisioned. diff --git a/content/operatingsystemmanager/cheat-sheets/_index.en.md b/content/operatingsystemmanager/cheat-sheets/_index.en.md new file mode 100644 index 000000000..f4524ceda --- /dev/null +++ b/content/operatingsystemmanager/cheat-sheets/_index.en.md @@ -0,0 +1,8 @@ ++++ +title = "Cheat Sheets" +date = 2022-08-20T12:00:00+02:00 +weight = 7 +chapter = true ++++ + +# Cheat Sheets diff --git a/content/operatingsystemmanager/cheat-sheets/rollout-machinedeployment/_index.en.md b/content/operatingsystemmanager/cheat-sheets/rollout-machinedeployment/_index.en.md new file mode 100644 index 000000000..a5f363588 --- /dev/null +++ b/content/operatingsystemmanager/cheat-sheets/rollout-machinedeployment/_index.en.md @@ -0,0 +1,22 @@ ++++ +title = "Rolling Restart MachineDeploments" +date = 2022-08-20T12:00:00+02:00 +description = "Rolling restart machine deployments: Learn about the concept and keep it handy" +weight = 25 ++++ + +## To rolling restart a single machineDeployment + +```shell +forceRestartAnnotations="{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"forceRestart\":\"$(date +%s)\"}}}}}" +kubectl patch machinedeployment -n kube-system --type=merge -p $forceRestartAnnotations +``` + +## To rolling restart all machineDeployments + +```shell +forceRestartAnnotations="{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"forceRestart\":\"$(date +%s)\"}}}}}" +for md in $(kubectl get machinedeployments -n kube-system --no-headers | awk '{print $1}'); do + kubectl patch machinedeployment -n kube-system $md --type=merge -p $forceRestartAnnotations +done +``` diff --git a/content/operatingsystemmanager/compatibility/_index.en.md b/content/operatingsystemmanager/compatibility/_index.en.md new file mode 100644 index 000000000..3d744b757 --- /dev/null +++ b/content/operatingsystemmanager/compatibility/_index.en.md @@ -0,0 +1,31 @@ ++++ +title = "Compatibility Matrix" +date = 2022-08-20T12:00:00+02:00 +weight = 6 ++++ + +The page provides an overview for the supported operating systems on various cloud providers. These are the combinations that are covered by the "default" OperatingSystemProfiles that OSM will install in your cluster. Users can create custom OperatingSystemProfiles that work with a provider/OS combination that are not listed here. + +## Operating System + +| | Ubuntu | CentOS | Flatcar | Amazon Linux 2 | RHEL | Rocky Linux | +|---|---|---|---|---|---|---| +| AWS | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Azure | ✓ | ✓ | ✓ | x | ✓ | ✓ | +| DigitalOcean | ✓ | ✓ | x | x | x | ✓ | +| Equinix Metal | ✓ | ✓ | ✓ | x | x | ✓ | +| Google Cloud Platform | ✓ | x | x | x | x | x | +| Hetzner | ✓ | ✓ | x | x | x | ✓ | +| KubeVirt | ✓ | ✓ | ✓ | x | ✓ | ✓ | +| Nutanix | ✓ | ✓ | x | x | x | x | +| Openstack | ✓ | ✓ | ✓ | x | ✓ | ✓ | +| VMware Cloud Director | ✓ | x | x | x | x | x | +| VSphere | ✓ | ✓ | ✓ | x | ✓ | ✓ | + +## Kubernetes Versions + +Currently supported K8S versions are: + +- 1.24 +- 1.23 +- 1.22 diff --git a/content/operatingsystemmanager/data/osp-ubuntu-swap-enabled.yaml b/content/operatingsystemmanager/data/osp-ubuntu-swap-enabled.yaml new file mode 100644 index 000000000..58841badd --- /dev/null +++ b/content/operatingsystemmanager/data/osp-ubuntu-swap-enabled.yaml @@ -0,0 +1,829 @@ +apiVersion: operatingsystemmanager.k8c.io/v1alpha1 +kind: OperatingSystemProfile +metadata: + name: osp-ubuntu-swap-enabled + namespace: kube-system +spec: + osName: "ubuntu" + osVersion: "20.04" + version: "v1.0.0" + supportedCloudProviders: + - name: "aws" + + bootstrapConfig: + files: + - path: /opt/bin/bootstrap + permissions: 755 + content: + inline: + encoding: b64 + data: | + #!/bin/bash + set -xeuo pipefail + + export DEBIAN_FRONTEND=noninteractive + apt update && apt install -y curl jq + curl -s -k -v --header 'Authorization: Bearer {{ .Token }}' {{ .ServerURL }}/api/v1/namespaces/cloud-init-settings/secrets/{{ .SecretName }} | jq '.data["cloud-config"]' -r| base64 -d > /etc/cloud/cloud.cfg.d/{{ .SecretName }}.cfg + cloud-init clean + + {{- /* The default cloud-init configurations files have a bug on Digital Ocean that causes the machine to be in-accessible on the 2nd cloud-init and in case of Hetzner, ipv6 addresses are missing. Hence we disable network configuration. */}} + {{- if (or (eq .CloudProviderName "digitalocean") (eq .CloudProviderName "hetzner")) }} + rm /etc/netplan/50-cloud-init.yaml + echo "network: {config: disabled}" > /etc/cloud/cloud.cfg.d/99-custom-networking.cfg + {{- end }} + + cloud-init --file /etc/cloud/cloud.cfg.d/{{ .SecretName }}.cfg init + systemctl daemon-reload + + {{- if eq .CloudProviderName "digitalocean" }} + netplan generate + netplan apply + {{- end }} + + systemctl daemon-reload + systemctl restart setup.service + + - path: /etc/systemd/system/bootstrap.service + permissions: 644 + content: + inline: + encoding: b64 + data: | + [Install] + WantedBy=multi-user.target + + [Unit] + Requires=network-online.target + After=network-online.target + [Service] + Type=oneshot + RemainAfterExit=true + ExecStart=/opt/bin/bootstrap + + modules: + runcmd: + - systemctl restart bootstrap.service + - systemctl daemon-reload + + provisioningConfig: + supportedContainerRuntimes: + - name: containerd + files: + - path: /etc/systemd/system/containerd.service.d/environment.conf + content: + inline: + data: | + [Service] + Restart=always + EnvironmentFile=-/etc/environment + + - path: /etc/crictl.yaml + content: + inline: + data: | + runtime-endpoint: unix:///run/containerd/containerd.sock + + - path: /etc/containerd/config.toml + permissions: 600 + content: + inline: + encoding: b64 + data: | + {{ .ContainerRuntimeConfig }} + templates: + containerRuntimeInstallation: |- + apt-get update + apt-get install -y apt-transport-https ca-certificates curl software-properties-common lsb-release + curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - + add-apt-repository "deb https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" + + apt-get install -y --allow-downgrades containerd.io=1.5* + apt-mark hold containerd.io + + systemctl daemon-reload + systemctl enable --now containerd + + - name: docker + files: + - path: /etc/systemd/system/containerd.service.d/environment.conf + content: + inline: + data: | + [Service] + Restart=always + EnvironmentFile=-/etc/environment + + - path: /etc/systemd/system/docker.service.d/environment.conf + content: + inline: + data: | + [Service] + Restart=always + EnvironmentFile=-/etc/environment + + - path: /etc/docker/daemon.json + permissions: 644 + content: + inline: + encoding: b64 + data: |- + {{ .ContainerRuntimeConfig }} + + - path: /root/.docker/config.json + permissions: 600 + content: + inline: + encoding: b64 + data: |- + {{ .ContainerRuntimeAuthConfig }} + + templates: + containerRuntimeInstallation: |- + apt-get update + apt-get install -y apt-transport-https ca-certificates curl software-properties-common lsb-release + curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - + add-apt-repository "deb https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" + + apt-get install --allow-downgrades -y \ + containerd.io=1.5* \ + docker-ce-cli=5:20.10* \ + docker-ce=5:20.10* + apt-mark hold docker-ce* containerd.io + + systemctl daemon-reload + systemctl enable --now docker + + templates: + safeDownloadBinariesScript: |- + {{- /* setup some common directories */}} + opt_bin=/opt/bin + usr_local_bin=/usr/local/bin + cni_bin_dir=/opt/cni/bin + + {{- /* create all the necessary dirs */}} + mkdir -p /etc/cni/net.d /etc/kubernetes/manifests "$opt_bin" "$cni_bin_dir" + {{- if semverCompare "<1.23" .KubeVersion }} + mkdir -p /etc/kubernetes/dynamic-config-dir + {{- end }} + {{- /* HOST_ARCH can be defined outside of machine-controller (in kubeone for example) */}} + arch=${HOST_ARCH-} + if [ -z "$arch" ] + then + case $(uname -m) in + x86_64) + arch="amd64" + ;; + aarch64) + arch="arm64" + ;; + *) + echo "unsupported CPU architecture, exiting" + exit 1 + ;; + esac + fi + + {{- /* # CNI variables */}} + CNI_VERSION="${CNI_VERSION:-v0.8.7}" + cni_base_url="https://github.com/containernetworking/plugins/releases/download/$CNI_VERSION" + cni_filename="cni-plugins-linux-$arch-$CNI_VERSION.tgz" + + {{- /* download CNI */}} + curl -Lfo "$cni_bin_dir/$cni_filename" "$cni_base_url/$cni_filename" + + {{- /* download CNI checksum */}} + cni_sum=$(curl -Lf "$cni_base_url/$cni_filename.sha256") + cd "$cni_bin_dir" + + {{- /* verify CNI checksum */}} + sha256sum -c <<<"$cni_sum" + + {{- /* unpack CNI */}} + tar xvf "$cni_filename" + rm -f "$cni_filename" + cd - + + {{- /* # cri-tools variables */}} + CRI_TOOLS_RELEASE="${CRI_TOOLS_RELEASE:-v1.22.0}" + cri_tools_base_url="https://github.com/kubernetes-sigs/cri-tools/releases/download/${CRI_TOOLS_RELEASE}" + cri_tools_filename="crictl-${CRI_TOOLS_RELEASE}-linux-${arch}.tar.gz" + + {{- /* download cri-tools */}} + curl -Lfo "$opt_bin/$cri_tools_filename" "$cri_tools_base_url/$cri_tools_filename" + + {{- /* download cri-tools checksum */}} + {{- /* the cri-tools checksum file has a filename prefix that breaks sha256sum so we need to drop it with sed */}} + cri_tools_sum=$(curl -Lf "$cri_tools_base_url/$cri_tools_filename.sha256" | sed 's/\*\///') + cd "$opt_bin" + + {{- /* verify cri-tools checksum */}} + sha256sum -c <<<"$cri_tools_sum" + + {{- /* unpack cri-tools and symlink to path so it's available to all users */}} + tar xvf "$cri_tools_filename" + rm -f "$cri_tools_filename" + ln -sf "$opt_bin/crictl" "$usr_local_bin"/crictl || echo "symbolic link is skipped" + cd - + + {{- /* kubelet */}} + KUBE_VERSION="${KUBE_VERSION:-{{ .KubeVersion }}}" + kube_dir="$opt_bin/kubernetes-$KUBE_VERSION" + kube_base_url="https://storage.googleapis.com/kubernetes-release/release/$KUBE_VERSION/bin/linux/$arch" + kube_sum_file="$kube_dir/sha256" + + {{- /* create versioned kube dir */}} + mkdir -p "$kube_dir" + : >"$kube_sum_file" + + for bin in kubelet kubeadm kubectl; do + {{- /* download kube binary */}} + curl -Lfo "$kube_dir/$bin" "$kube_base_url/$bin" + chmod +x "$kube_dir/$bin" + + {{- /* download kube binary checksum */}} + sum=$(curl -Lf "$kube_base_url/$bin.sha256") + + {{- /* save kube binary checksum */}} + echo "$sum $kube_dir/$bin" >>"$kube_sum_file" + done + + {{- /* check kube binaries checksum */}} + sha256sum -c "$kube_sum_file" + + for bin in kubelet kubeadm kubectl; do + {{- /* link kube binaries from verioned dir to $opt_bin */}} + ln -sf "$kube_dir/$bin" "$opt_bin"/$bin + done + + configureProxyScript: |- + {{- if .HTTPProxy }} + cat <.yaml. The env variables come from an env + # file provided by the systemd service. + + # This script is a slightly adjusted version of + # https://github.com/kubernetes/kubernetes/blob/e1a1aa211224fcd9b213420b80b2ae680669683d/cluster/gce/gci/health-monitor.sh + # Adjustments are: + # * Kubelet health port is 10248 not 10255 + # * Removal of all all references to the KUBE_ENV file + + set -o nounset + set -o pipefail + + # We simply kill the process when there is a failure. Another systemd service will + # automatically restart the process. + function container_runtime_monitoring() { + local -r max_attempts=5 + local attempt=1 + local -r container_runtime_name="${CONTAINER_RUNTIME_NAME:-docker}" + # We still need to use 'docker ps' when container runtime is "docker". This is because + # dockershim is still part of kubelet today. When kubelet is down, crictl pods + # will also fail, and docker will be killed. This is undesirable especially when + # docker live restore is disabled. + local healthcheck_command="docker ps" + if [[ "${CONTAINER_RUNTIME:-docker}" != "docker" ]]; then + healthcheck_command="crictl pods" + fi + # Container runtime startup takes time. Make initial attempts before starting + # killing the container runtime. + until timeout 60 ${healthcheck_command} > /dev/null; do + if ((attempt == max_attempts)); then + echo "Max attempt ${max_attempts} reached! Proceeding to monitor container runtime healthiness." + break + fi + echo "$attempt initial attempt \"${healthcheck_command}\"! Trying again in $attempt seconds..." + sleep "$((2 ** attempt++))" + done + while true; do + if ! timeout 60 ${healthcheck_command} > /dev/null; then + echo "Container runtime ${container_runtime_name} failed!" + if [[ "$container_runtime_name" == "docker" ]]; then + # Dump stack of docker daemon for investigation. + # Log file name looks like goroutine-stacks-TIMESTAMP and will be saved to + # the exec root directory, which is /var/run/docker/ on Ubuntu and COS. + pkill -SIGUSR1 dockerd + fi + systemctl kill --kill-who=main "${container_runtime_name}" + # Wait for a while, as we don't want to kill it again before it is really up. + sleep 120 + else + sleep "${SLEEP_SECONDS}" + fi + done + } + + function kubelet_monitoring() { + echo "Wait for 2 minutes for kubelet to be functional" + sleep 120 + local -r max_seconds=10 + local output="" + while true; do + local failed=false + + if journalctl -u kubelet -n 1 | grep -q "use of closed network connection"; then + failed=true + echo "Kubelet stopped posting node status. Restarting" + elif ! output=$(curl -m "${max_seconds}" -f -s -S http://127.0.0.1:10248/healthz 2>&1); then + failed=true + # Print the response and/or errors. + echo "$output" + fi + + if [[ "$failed" == "true" ]]; then + echo "Kubelet is unhealthy!" + systemctl kill kubelet + # Wait for a while, as we don't want to kill it again before it is really up. + sleep 60 + else + sleep "${SLEEP_SECONDS}" + fi + done + } + + ############## Main Function ################ + if [[ "$#" -ne 1 ]]; then + echo "Usage: health-monitor.sh " + exit 1 + fi + + SLEEP_SECONDS=10 + component=$1 + echo "Start kubernetes health monitoring for ${component}" + if [[ "${component}" == "container-runtime" ]]; then + container_runtime_monitoring + elif [[ "${component}" == "kubelet" ]]; then + kubelet_monitoring + else + echo "Health monitoring for component ${component} is not supported!" + fi + + - path: /etc/systemd/journald.conf.d/max_disk_use.conf + content: + inline: + encoding: b64 + data: | + [Journal] + SystemMaxUse=5G + + - path: /opt/load-kernel-modules.sh + permissions: 755 + content: + inline: + encoding: b64 + data: | + #!/usr/bin/env bash + set -euo pipefail + + modprobe ip_vs + modprobe ip_vs_rr + modprobe ip_vs_wrr + modprobe ip_vs_sh + + if modinfo nf_conntrack_ipv4 &> /dev/null; then + modprobe nf_conntrack_ipv4 + else + modprobe nf_conntrack + fi + + - path: /etc/sysctl.d/k8s.conf + content: + inline: + encoding: b64 + data: | + net.bridge.bridge-nf-call-ip6tables = 1 + net.bridge.bridge-nf-call-iptables = 1 + kernel.panic_on_oops = 1 + kernel.panic = 10 + net.ipv4.ip_forward = 1 + vm.overcommit_memory = 1 + fs.inotify.max_user_watches = 1048576 + fs.inotify.max_user_instances = 8192 + + - path: /etc/default/grub.d/60-swap-accounting.cfg + content: + inline: + encoding: b64 + data: | + # Added by kubermatic machine-controller + # Enable cgroups memory and swap accounting + GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1" + + - path: /opt/bin/setup + permissions: 755 + content: + inline: + encoding: b64 + data: | + #!/bin/bash + set -xeuo pipefail + if systemctl is-active ufw; then systemctl stop ufw; fi + systemctl mask ufw + + + {{- /* As we added some modules and don't want to reboot, restart the service */}} + systemctl restart systemd-modules-load.service + sysctl --system + + # Override hostname if /etc/machine-name exists + if [ -x "$(command -v hostnamectl)" ] && [ -s /etc/machine-name ]; then + machine_name=$(cat /etc/machine-name) + hostnamectl set-hostname ${machine_name} + fi + + apt-get update + + DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y \ + curl \ + ca-certificates \ + ceph-common \ + cifs-utils \ + conntrack \ + e2fsprogs \ + ebtables \ + ethtool \ + glusterfs-client \ + iptables \ + jq \ + kmod \ + openssh-client \ + nfs-common \ + socat \ + util-linux \ + {{- if or (eq .CloudProviderName "vsphere") (eq .CloudProviderName "vmware-cloud-director") }} + open-vm-tools \ + {{- end }} + {{- if eq .CloudProviderName "nutanix" }} + open-iscsi \ + {{- end }} + ipvsadm + + {{- /* iscsid service is required on Nutanix machines for CSI driver to attach volumes. */}} + {{- if eq .CloudProviderName "nutanix" }} + systemctl enable --now iscsid + {{ end }} + + {{- template "containerRuntimeInstallation" }} + + {{- template "safeDownloadBinariesScript" }} + + {{- template "configureProxyScript" }} + + # set kubelet nodeip environment variable + /opt/bin/setup_net_env.sh + + {{- /* fetch kubelet bootstrapping kubeconfig */}} + curl -s -k -v --header 'Authorization: Bearer {{ .Token }}' {{ .ServerURL }}/api/v1/namespaces/cloud-init-settings/secrets/{{ .BootstrapKubeconfigSecretName }} | jq '.data["kubeconfig"]' -r| base64 -d > /etc/kubernetes/bootstrap-kubelet.conf + + systemctl enable --now kubelet + systemctl enable --now --no-block kubelet-healthcheck.service + systemctl disable setup.service + + - path: /opt/bin/supervise.sh + permissions: 755 + content: + inline: + encoding: b64 + data: | + #!/bin/bash + set -xeuo pipefail + while ! "$@"; do + sleep 1 + done + + - path: /etc/systemd/system/kubelet.service + content: + inline: + encoding: b64 + data: | + [Unit] + After={{ .ContainerRuntime }}.service + Requires={{ .ContainerRuntime }}.service + + Description=kubelet: The Kubernetes Node Agent + Documentation=https://kubernetes.io/docs/home/ + + [Service] + User=root + Restart=always + StartLimitInterval=0 + RestartSec=10 + CPUAccounting=true + MemoryAccounting=true + + Environment="PATH=/opt/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin/" + EnvironmentFile=-/etc/environment + + ExecStartPre=/bin/bash /opt/load-kernel-modules.sh + ExecStartPre=/bin/bash /opt/bin/setup_net_env.sh + ExecStart=/opt/bin/kubelet $KUBELET_EXTRA_ARGS \ + --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf \ + --kubeconfig=/var/lib/kubelet/kubeconfig \ + --config=/etc/kubernetes/kubelet.conf \ + {{- if semverCompare "<1.24" .KubeVersion }} + --network-plugin=cni \ + {{- end }} + --cert-dir=/etc/kubernetes/pki \ + {{- if .ExternalCloudProvider }} + --cloud-provider=external \ + {{- else if .InTreeCCMAvailable }} + --cloud-provider={{- .CloudProviderName }} \ + --cloud-config=/etc/kubernetes/cloud-config \ + {{- end }} + {{- if ne .CloudProviderName "aws" }} + --hostname-override=${KUBELET_HOSTNAME} \ + {{- else if and (eq .CloudProviderName "aws") (.ExternalCloudProvider) }} + --hostname-override=${KUBELET_HOSTNAME} \ + {{- end }} + {{- if semverCompare "<1.23" .KubeVersion }} + --dynamic-config-dir=/etc/kubernetes/dynamic-config-dir \ + --feature-gates=DynamicKubeletConfig=true,NodeSwap=true \ + {{- end }} + --exit-on-lock-contention \ + --lock-file=/tmp/kubelet.lock \ + {{- if .PauseImage }} + --pod-infra-container-image={{ .PauseImage }} \ + {{- end }} + {{- if .InitialTaints }} + --register-with-taints={{- .InitialTaints }} \ + {{- end }} + {{- if eq .ContainerRuntime "containerd" }} + --container-runtime=remote \ + --container-runtime-endpoint=unix:///run/containerd/containerd.sock \ + {{- end }} + {{- if eq .ContainerRuntime "docker" }} + --container-runtime=docker \ + --container-runtime-endpoint=unix:///var/run/dockershim.sock \ + {{- end }} + {{- /* If external or in-tree CCM is in use we don't need to set --node-ip as the cloud provider will know what IPs to return. */}} + {{- if not (and (eq .NetworkIPFamily "IPv4+IPv6") (or (.InTreeCCMAvailable) (.ExternalCloudProvider))) }} + --node-ip ${KUBELET_NODE_IP} + {{- end }} + + [Install] + WantedBy=multi-user.target + + - path: /etc/systemd/system/kubelet.service.d/extras.conf + content: + inline: + encoding: b64 + data: | + [Service] + Environment="KUBELET_EXTRA_ARGS=--resolv-conf=/run/systemd/resolve/resolv.conf" + + - path: /etc/kubernetes/cloud-config + permissions: 600 + content: + inline: + encoding: b64 + data: | + {{ .CloudConfig }} + + - path: /opt/bin/setup_net_env.sh + permissions: 755 + content: + inline: + encoding: b64 + data: | + #!/usr/bin/env bash + echodate() { + echo "[$(date -Is)]" "$@" + } + + # get the default interface IP address + {{- if eq .NetworkIPFamily "IPv6" }} + DEFAULT_IFC_IP=$(ip -o -6 route get 1:: | grep -oP "src \K\S+") + {{- else if eq .NetworkIPFamily "IPv4+IPv6" }} + DEFAULT_IFC_IPv4=$(ip -o route get 1 | grep -oP "src \K\S+") + DEFAULT_IFC_IPv6=$(ip -o -6 route get 1:: | grep -oP "src \K\S+") + DEFAULT_IFC_IP=$DEFAULT_IFC_IPv4,$DEFAULT_IFC_IPv6 + {{- else }} + DEFAULT_IFC_IP=$(ip -o route get 1 | grep -oP "src \K\S+") + {{- end }} + + if [ -z "${DEFAULT_IFC_IP}" ] + then + echodate "Failed to get IP address for the default route interface" + exit 1 + fi + + # get the full hostname + FULL_HOSTNAME=$(hostname -f) + # if /etc/machine-name is not empty then use the hostname from there + if [ -s /etc/machine-name ]; then + FULL_HOSTNAME=$(cat /etc/machine-name) + fi + + # write the nodeip_env file + # we need the line below because flatcar has the same string "coreos" in that file + if grep -q coreos /etc/os-release + then + echo "KUBELET_NODE_IP=${DEFAULT_IFC_IP}\nKUBELET_HOSTNAME=${FULL_HOSTNAME}" > /etc/kubernetes/nodeip.conf + elif [ ! -d /etc/systemd/system/kubelet.service.d ] + then + echodate "Can't find kubelet service extras directory" + exit 1 + else + echo -e "[Service]\nEnvironment=\"KUBELET_NODE_IP=${DEFAULT_IFC_IP}\"\nEnvironment=\"KUBELET_HOSTNAME=${FULL_HOSTNAME}\"" > /etc/systemd/system/kubelet.service.d/nodeip.conf + fi + + - path: /etc/kubernetes/pki/ca.crt + content: + inline: + encoding: b64 + data: | + {{ .KubernetesCACert }} + + - path: /etc/systemd/system/setup.service + permissions: 644 + content: + inline: + encoding: b64 + data: | + [Install] + WantedBy=multi-user.target + + [Unit] + Requires=network-online.target + After=network-online.target + + [Service] + Type=oneshot + RemainAfterExit=true + EnvironmentFile=-/etc/environment + ExecStart=/opt/bin/supervise.sh /opt/bin/setup + + - path: /etc/profile.d/opt-bin-path.sh + permissions: 644 + content: + inline: + encoding: b64 + data: | + export PATH="/opt/bin:$PATH" + + - path: /etc/kubernetes/kubelet.conf + content: + inline: + encoding: b64 + data: | + apiVersion: kubelet.config.k8s.io/v1beta1 + kind: KubeletConfiguration + authentication: + anonymous: + enabled: false + webhook: + cacheTTL: 0s + enabled: true + x509: + clientCAFile: /etc/kubernetes/pki/ca.crt + authorization: + mode: Webhook + webhook: + cacheAuthorizedTTL: 0s + cacheUnauthorizedTTL: 0s + cgroupDriver: systemd + clusterDNS: + {{- range .ClusterDNSIPs }} + - "{{ . }}" + {{- end }} + clusterDomain: cluster.local + {{- /* containerLogMaxSize and containerLogMaxFiles have no effect for docker */}} + {{- if ne .ContainerRuntime "docker" }} + {{- if .ContainerLogMaxSize }} + containerLogMaxSize: {{ .ContainerLogMaxSize }} + {{- else }} + containerLogMaxSize: 100Mi + {{- end }} + {{- if .ContainerLogMaxFiles }} + containerLogMaxFiles: {{ .ContainerLogMaxFiles }} + {{- else }} + containerLogMaxFiles: 5 + {{- end }} + {{- end }} + cpuManagerReconcilePeriod: 0s + evictionPressureTransitionPeriod: 0s + featureGates: + {{- if .KubeletFeatureGates -}} + {{ range $key, $val := .KubeletFeatureGates }} + {{ $key }}: {{ $val }} + {{- end -}} + {{- end }} + fileCheckFrequency: 0s + httpCheckFrequency: 0s + imageMinimumGCAge: 0s + logging: + flushFrequency: 0 + options: + json: + infoBufferSize: "0" + verbosity: 0 + failSwapOn: false + memorySwap: + swapBehavior: LimitedSwap + nodeStatusReportFrequency: 0s + nodeStatusUpdateFrequency: 0s + protectKernelDefaults: true + readOnlyPort: 0 + rotateCertificates: true + runtimeRequestTimeout: 0s + serverTLSBootstrap: true + shutdownGracePeriod: 0s + shutdownGracePeriodCriticalPods: 0s + staticPodPath: /etc/kubernetes/manifests + streamingConnectionIdleTimeout: 0s + syncFrequency: 0s + kubeReserved: + {{- if .KubeReserved -}} + {{ range $key, $val := .KubeReserved }} + {{ $key }}: {{ $val }} + {{- end -}} + {{- else }} + cpu: 200m + ephemeral-storage: 1Gi + memory: 200Mi + {{- end }} + systemReserved: + {{- if .SystemReserved -}} + {{ range $key, $val := .SystemReserved }} + {{ $key }}: {{ $val }} + {{- end -}} + {{- else }} + cpu: 200m + ephemeral-storage: 1Gi + memory: 200Mi + {{- end }} + evictionHard: + {{- if .EvictionHard -}} + {{ range $key, $val := .EvictionHard }} + {{ $key }}: {{ $val }} + {{- end -}} + {{- else }} + imagefs.available: 15% + memory.available: 100Mi + nodefs.available: 10% + nodefs.inodesFree: 5% + {{- end }} + {{- if .MaxPods }} + maxPods: {{ .MaxPods }} + {{- end }} + tlsCipherSuites: + - TLS_AES_128_GCM_SHA256 + - TLS_AES_256_GCM_SHA384 + - TLS_CHACHA20_POLY1305_SHA256 + - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 + - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 + - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305 + - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 + - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 + - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 + volumePluginDir: /var/lib/kubelet/volumeplugins + volumeStatsAggPeriod: 0s + + - path: /etc/systemd/system/kubelet-healthcheck.service + permissions: 644 + content: + inline: + encoding: b64 + data: | + [Unit] + Requires=kubelet.service + After=kubelet.service + + [Service] + ExecStart=/opt/bin/health-monitor.sh kubelet + + [Install] + WantedBy=multi-user.target diff --git a/content/operatingsystemmanager/guides/_index.en.md b/content/operatingsystemmanager/guides/_index.en.md new file mode 100644 index 000000000..f396a4cb3 --- /dev/null +++ b/content/operatingsystemmanager/guides/_index.en.md @@ -0,0 +1,11 @@ ++++ +title = "Guides" +date = 2022-08-20T12:00:00+02:00 +description = "Get familiar with OperatingSystemManager and read step-by-step instructions to handle important scenarios" +weight = 4 +chapter = true ++++ + +# Guides + +Get familiar with OperatingSystemManager and read step-by-step instructions to handle important scenarios diff --git a/content/operatingsystemmanager/guides/customize-operating-system-profile/_index.en.md b/content/operatingsystemmanager/guides/customize-operating-system-profile/_index.en.md new file mode 100644 index 000000000..15ee8987b --- /dev/null +++ b/content/operatingsystemmanager/guides/customize-operating-system-profile/_index.en.md @@ -0,0 +1,76 @@ ++++ +title = "Customize Operating System Profiles" +date = 2022-08-20T12:00:00+02:00 ++++ + +One of the unique selling propositions for OSM is that it allows users to use custom OperatingSystemProfiles(OSPs). OSPs can be modified for various use cases; making them compatible with your infrastructure, installing and configuring custom/additional packages, customizing configurations for kubelet, container runtime, networking, storage etc. + +{{% notice note %}} +For customization, new dedicated OSPs should be created by the end users. Updating the "default" OSPs is not recommended since they are managed by OSM itself and any changes will be reverted automatically. +{{% /notice %}} + +## Create a custom OSP + +As an example, let's modify the [default OSP for ubuntu](https://github.com/kubermatic/operating-system-manager/blob/master/deploy/osps/default/osp-ubuntu.yaml) and enable `swap memory`. + +```yaml +{{< readfile "operatingsystemmanager/data/osp-ubuntu-swap-enabled.yaml" >}} +``` + +The changes we made to the OSP: + +![Git Diff](/img/operatingsystemmanager/master/osp-ubuntu-swap-enabled.png?classes=shadow,border "Custom OperatingSystemProfile") + +## Consume the custom OSP + +Create/update your MachineDeployment to consume the custom OSP. The annotation `k8c.io/operating-system-profile` is used to specify the OSP for a MachineDeployment. + +```yaml +apiVersion: cluster.k8s.io/v1alpha1 +kind: MachineDeployment +metadata: + name: << MACHINE_NAME >> + namespace: kube-system + annotations: + k8c.io/operating-system-profile: "osp-ubuntu-swap-enabled" +spec: + replicas: 1 + strategy: + type: RollingUpdate + rollingUpdate: + maxSurge: 1 + maxUnavailable: 0 + selector: + matchLabels: + name: << MACHINE_NAME >> + template: + metadata: + labels: + name: << MACHINE_NAME >> + spec: + providerSpec: + value: + cloudProvider: "aws" + cloudProviderSpec: + region: "eu-central-1" + availabilityZone: "eu-central-1a" + vpcId: "vpc-name" + subnetId: "subnet-id" + instanceType: "t2.micro" + instanceProfile: "kubernetes-v1" + isSpotInstance: false + diskSize: 50 + diskType: "gp2" + ebsVolumeEncrypted: false + ami: "my-custom-ami" + operatingSystem: flatcar + operatingSystemSpec: + # 'provisioningUtility` is only used for flatcar os, can be set to ignition or cloud-init. Defaults to ignition. + provisioningUtility: ignition + versions: + kubelet: "<< KUBERNETES_VERSION >>" +``` + +If you are updating the annotation on an existing MachineDeployment then the machines will not be rotated automatically. To perform a rotation for existing MachineDeployments please follow the guide at [Rolling Restart MachineDeploments][rolling-restart]. + +[rolling-restart]: {{< ref "../../cheat-sheets/rollout-machinedeployment" >}} diff --git a/content/operatingsystemmanager/guides/machine-rotation/_index.en.md b/content/operatingsystemmanager/guides/machine-rotation/_index.en.md new file mode 100644 index 000000000..94769ea47 --- /dev/null +++ b/content/operatingsystemmanager/guides/machine-rotation/_index.en.md @@ -0,0 +1,30 @@ ++++ +title = "Machine Rotation" +date = 2022-08-20T12:00:00+02:00 ++++ + +Machine rotation is required against the following scenarios: + +## Update in MachineDeployment + +Machine Controller will rotate machines automatically when there is a new `revision` for the MachineDeployment i.e. any change in the `.spec.template` in the MachineDeployment. + +OSM in that case will re-generate the OSC and secrets and annotate them with the `revision` number using `machinedeployment.clusters.k8s.io/revision` annotation. Machine Controller waits for the updated secrets and then provisions the machines. + +In any other case where a new `revision` is not rolled out against a change in MachineDeployment. Users will have to manually rotate the machines. To perform this rotation for existing MachineDeployments please follow the guide at [Rolling Restart MachineDeploments][rolling-restart]. + +For example, manual rotation is required if user wants to change the OSP for a MachineDeployment. + +```yaml +apiVersion: "cluster.k8s.io/v1alpha1" +kind: MachineDeployment +metadata: + annotations: + k8c.io/operating-system-profile: "osp-ubuntu" +``` + +## Update in OperatingSystemProfile + +An update to `OperatingSystemProfile` will not result in an automatic rotation of the machines. This is an intentional design decision since an `OperatingSystemProfile` can be associated with multiple MachineDeployments. It is the user's responsibility to rotate the machines when the `OperatingSystemProfile` is updated. To perform this rotation for existing MachineDeployments please follow the guide at [Rolling Restart MachineDeploments][rolling-restart]. + +[rolling-restart]: {{< ref "../../cheat-sheets/rollout-machinedeployment" >}} diff --git a/content/operatingsystemmanager/guides/osm-configuration/_index.en.md b/content/operatingsystemmanager/guides/osm-configuration/_index.en.md new file mode 100644 index 000000000..860c6c518 --- /dev/null +++ b/content/operatingsystemmanager/guides/osm-configuration/_index.en.md @@ -0,0 +1,68 @@ ++++ +title = "Configuring Operating System Manager" +date = 2022-08-20T12:00:00+02:00 +enableToc = true ++++ + +## Configuring Operating System Manager + +OSM can be configured using the following command line flags: + +| Flags | Type | Required | Default | Description | +| ------------------- | ------ | -------- | ------- | ----------------------------------- | +| `kubeconfig` | string | false | `""` | Path to a kubeconfig. Only required if out-of-cluster. | +| `worker-cluster-kubeconfig` | string | false | `""` | Path to kubeconfig of cluster where provisioning secrets are created. | +| `namespace` | string | true | `""` | The namespace where the OSC controller will run. | +| `container-runtime` | string | false | `containerd` | Container runtime to deploy. | +| `external-cloud-provider` | bool | false | `false` | Cloud-provider Kubelet flag set to external.. | +| `cluster-dns` | string | false | `10.10.10.10` | Comma-separated list of DNS server IP address. | +| `pause-image` | string | false | `""` | Pause image to use in Kubelet. | +| `initial-taints` | string | false | `""` | Taints to use when creating the node. | +| `node-kubelet-feature-gates` | string | false | `RotateKubeletServerCertificate=true` | Feature gates to set on the kubelet. If user overrides the value, `RotateKubeletServerCertificate=true` is appended by OSM in the feature gates. | +| `node-http-proxy` | string | false | `""` | If set, it configures the 'HTTP_PROXY' & 'HTTPS_PROXY' environment variable on the nodes. | +| `node-no-proxy` | string | false | `.svc,.cluster.local,localhost,127.0.0.1` | If set, it configures the 'NO_PROXY' environment variable on the nodes. | +| `node-insecure-registries` | string | false | `""` | Comma separated list of registries which should be configured as insecure on the container runtime. | +| `node-http-proxy` | string | false | `""` | If set, it configures the 'HTTP_PROXY' & 'HTTPS_PROXY' environment variable on the nodes. | +| `node-registry-mirrors` | string | false | `""` | Comma separated list of Docker image mirrors. | +| `node-containerd-registry-mirrors` | string | false | `""` | Configure registry mirrors endpoints. Can be used multiple times to specify multiple mirrors. | +| `node-registry-credentials-secret` | string | false | `""` | A Secret object reference, that contains auth info for image registry in namespace/secret-name form, example: kube-system/registry-credentials. See doc at . | +| `health-probe-address` | string | false | `127.0.0.1:8085` | The address on which the liveness check on /healthz and readiness check on /readyz will be available. | +| `metrics-address` | string | false | `127.0.0.1:8080` | The address on which Prometheus metrics will be available under /metrics. | +| `worker-health-probe-address` | string | false | `127.0.0.1:8086` | For worker manager, the address on which the liveness check on /healthz and readiness check on /readyz will be available. | +| `worker-metrics-address` | string | false | `127.0.0.1:8081` | For worker manager, the address on which Prometheus metrics will be available under /metrics. | +| `leader-elect` | bool | false | `true` | Enable leader election for controller manager. | +| `override-bootstrap-kubelet-apiserver` | string | false | `""` | Override for the API server address used in worker nodes bootstrap-kubelet.conf. | +| `bootstrap-token-service-account-name` | string | false | `""` | When set use the service account token from this SA as bootstrap token instead of creating a temporary one. Passed in `namespace/name` format. | +| `worker-count` | int | false | `10` | Number of workers which process reconciliation in parallel. | + +## Configuring Operating System Profile + +To generate bootstrapping and provisioning configurations, OSM uses the OSP(template) and values from MachineDeployment and command line flags. These values are substituted dynamically to generate the configurations. + +Following is the list of the variables accessible inside an OSP: + +| Variable | Type | Description | +| ------------------- | ------ | ----------------------------------- | +| `KubeVersion` | string | Kubernetes version to use, picked from the MachineDeployment. | +| `InTreeCCMAvailable` | bool | True if in-tree CCM is available for the cloud provider. | +| `ClusterDNSIPs` | []string | List of Cluster DNS IP, picked from flags. | +| `KubernetesCACert` | string | CA certificate for the worker machine. Set at `/etc/kubernetes/pki/ca.crt`. | +| `CloudConfig` | string | Cloud config for the machine. | +| `ContainerRuntime` | string | Name of the container runtime to use. | +| `CloudProviderName` | string | Name of the cloud provider. | +| `ExternalCloudProvider` | bool | External CCM should be used for the cloud provider. | +| `PauseImage` | string | Image for the pause container, specified using `--pod-infra-container` for the kubelet. | +| `InitialTaints` | string | Register the node with the given list of taints. | +| `HTTPProxy` | string | Configuration for HTTP_PROXY, HTTPS_PROXY | +| `NoProxy` | string | Configuration for NO_PROXY | +| `ContainerRuntimeConfig` | string | Configuration for NO_PROXY | +| `ContainerRuntimeAuthConfig` | string | Configuration for NO_PROXY | +| `KubeletFeatureGates` | string | Feature gates for kubelet | +| `NetworkIPFamily` | string | Type of Network IP family; IPv4, IPv6 or IPv4+IPv6 | +| `NetworkConfig` | string | Static networking configuration, picked from the MachineDeployment. | +| `KubeReserved` | map[string]string | Picked up from annotations on MachineDeployment. | +| `SystemReserved` | map[string]string | Picked up from annotations on MachineDeployment. | +| `EvictionHard` | map[string]string | Picked up from annotations on MachineDeployment. | +| `MaxPods` | int32 | Picked up from annotations on MachineDeployment. | +| `ContainerLogMaxSize` | string | Picked up from annotations on MachineDeployment. | +| `ContainerLogMaxFiles` | string | Picked up from annotations on MachineDeployment. | diff --git a/content/operatingsystemmanager/references/_index.en.md b/content/operatingsystemmanager/references/_index.en.md new file mode 100644 index 000000000..1703f43b7 --- /dev/null +++ b/content/operatingsystemmanager/references/_index.en.md @@ -0,0 +1,7 @@ ++++ +title = "References" +date = 2022-08-20T12:00:00+02:00 +weight = 40 ++++ + +This section contains a reference of the Kubermatic OperatingSystemManager Custom Resource Definitions. diff --git a/content/operatingsystemmanager/references/crds/_index.en.md b/content/operatingsystemmanager/references/crds/_index.en.md new file mode 100644 index 000000000..76fecf596 --- /dev/null +++ b/content/operatingsystemmanager/references/crds/_index.en.md @@ -0,0 +1,243 @@ ++++ +title = "OperatingSystemManager CRDs Reference" +date = 2022-08-20T12:00:00+02:00 +weight = 40 ++++ + +## Packages + +- [operatingsystemmanager.k8c.io/v1alpha1](#operatingsystemmanagerk8ciov1alpha1) + +## operatingsystemmanager.k8c.io/v1alpha1 + +Package v1alpha1 defines the v1alpha1 version of the OSM API + +### CloudInitModule + +CloudInitModule contains the fields of the cloud init module. + +_Appears in:_ + +- [OSCConfig](#oscconfig) +- [OSPConfig](#ospconfig) + +| Field | Description | +| --- | --- | +| `bootcmd` _string array_ | BootCMD module runs arbitrary commands very early in the boot process, only slightly after a boothook would run. | +| `rh_subscription` _object (keys:string, values:string)_ | RHSubscription registers a Red Hat system either by username and password or activation and org | +| `runcmd` _string array_ | RunCMD Run arbitrary commands at a rc.local like level with output to the console. | +| `yum_repos` _object (keys:string, values:object)_ | YumRepos adds yum repository configuration to the system. | +| `yum_repo_dir` _string_ | YumRepoDir the repo parts directory where individual yum repo config files will be written. Default: /etc/yum.repos.d | + +[Back to top](#top) + +### CloudProviderSpec + +CloudProviderSpec contains the os/image reference for a specific supported cloud provider + +_Appears in:_ + +- [OperatingSystemConfigSpec](#operatingsystemconfigspec) +- [OperatingSystemProfileSpec](#operatingsystemprofilespec) + +| Field | Description | +| --- | --- | +| `name` _CloudProvider_ | Name represents the name of the supported cloud provider | +| `spec` _[RawExtension](#rawextension)_ | Spec represents the os/image reference in the supported cloud provider | + +[Back to top](#top) + +### ContainerRuntimeSpec + +ContainerRuntimeSpec aggregates information about a specific container runtime + +_Appears in:_ + +- [OSPConfig](#ospconfig) + +| Field | Description | +| --- | --- | +| `name` _ContainerRuntime_ | Name of the Container runtime | +| `files` _[File](#file) array_ | Files to add to the main files list when the containerRuntime is selected | +| `templates` _object (keys:string, values:string)_ | Templates to add to the available templates when the containerRuntime is selected | + +[Back to top](#top) + +### DropIn + +DropIn is a drop-in configuration for a systemd unit. + +_Appears in:_ + +- [Unit](#unit) + +| Field | Description | +| --- | --- | +| `name` _string_ | Name is the name of the drop-in. | +| `content` _string_ | Content is the content of the drop-in. | + +[Back to top](#top) + +### File + +File is a file that should get written to the host's file system. The content can either be inlined or referenced from a secret in the same namespace. + +_Appears in:_ + +- [ContainerRuntimeSpec](#containerruntimespec) +- [OSCConfig](#oscconfig) +- [OSPConfig](#ospconfig) + +| Field | Description | +| --- | --- | +| `path` _string_ | Path is the path of the file system where the file should get written to. | +| `permissions` _integer_ | Permissions describes with which permissions the file should get written to the file system. Should be in decimal base and without any leading zeroes. | +| `content` _[FileContent](#filecontent)_ | Content describe the file's content. | + +[Back to top](#top) + +### FileContent + +FileContent can either reference a secret or contain inline configuration. + +_Appears in:_ + +- [File](#file) + +| Field | Description | +| --- | --- | +| `inline` _[FileContentInline](#filecontentinline)_ | Inline is a struct that contains information about the inlined data. | + +[Back to top](#top) + +### FileContentInline + +FileContentInline contains keys for inlining a file content's data and encoding. + +_Appears in:_ + +- [FileContent](#filecontent) + +| Field | Description | +| --- | --- | +| `encoding` _string_ | Encoding is the file's encoding (e.g. base64). | +| `data` _string_ | Data is the file's data. | + +[Back to top](#top) + +### OSCConfig + +_Appears in:_ + +- [OperatingSystemConfigSpec](#operatingsystemconfigspec) + +| Field | Description | +| --- | --- | +| `units` _[Unit](#unit) array_ | Units a list of the systemd unit files which will run on the instance | +| `files` _[File](#file) array_ | Files is a list of files that should exist in the instance | +| `userSSHKeys` _string array_ | UserSSHKeys is a list of attached user ssh keys | +| `modules` _[CloudInitModule](#cloudinitmodule)_ | CloudInitModules contains the supported cloud-init modules | + +[Back to top](#top) + +### OSPConfig + +_Appears in:_ + +- [OperatingSystemProfileSpec](#operatingsystemprofilespec) + +| Field | Description | +| --- | --- | +| `supportedContainerRuntimes` _[ContainerRuntimeSpec](#containerruntimespec) array_ | SupportedContainerRuntimes represents the container runtimes supported by the given OS | +| `templates` _object (keys:string, values:string)_ | Templates to be included in units and files | +| `units` _[Unit](#unit) array_ | Units a list of the systemd unit files which will run on the instance | +| `files` _[File](#file) array_ | Files is a list of files that should exist in the instance | +| `modules` _[CloudInitModule](#cloudinitmodule)_ | CloudInitModules field contains the optional cloud-init modules which are supported by OSM | + +[Back to top](#top) + +### OperatingSystemConfig + +OperatingSystemConfig is the object that represents the OperatingSystemConfig + +_Appears in:_ + +- [OperatingSystemConfigList](#operatingsystemconfiglist) + +| Field | Description | +| --- | --- | +| `metadata` _[ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#objectmeta-v1-meta)_ | Refer to Kubernetes API documentation for fields of `metadata`. | +| `spec` _[OperatingSystemConfigSpec](#operatingsystemconfigspec)_ | OperatingSystemConfigSpec represents the operating system configuration spec. | + +[Back to top](#top) + +### OperatingSystemConfigSpec + +OperatingSystemConfigSpec represents the data in the newly created OperatingSystemConfig + +_Appears in:_ + +- [OperatingSystemConfig](#operatingsystemconfig) + +| Field | Description | +| --- | --- | +| `osName` _OperatingSystem_ | OSType represent the operating system name e.g: ubuntu | +| `osVersion` _string_ | OSVersion the version of the operating system | +| `cloudProvider` _[CloudProviderSpec](#cloudproviderspec)_ | CloudProvider represent the cloud provider that support the given operating system version | +| `bootstrapConfig` _[OSCConfig](#oscconfig)_ | BootstrapConfig is used for initial configuration of machine and to fetch the kubernetes secret that contains the provisioning config. | +| `provisioningConfig` _[OSCConfig](#oscconfig)_ | ProvisioningConfig is used for provisioning the worker node. | + +[Back to top](#top) + +### OperatingSystemProfile + +OperatingSystemProfile is the object that represents the OperatingSystemProfile + +_Appears in:_ + +- [OperatingSystemProfileList](#operatingsystemprofilelist) + +| Field | Description | +| --- | --- | +| `metadata` _[ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#objectmeta-v1-meta)_ | Refer to Kubernetes API documentation for fields of `metadata`. | +| `spec` _[OperatingSystemProfileSpec](#operatingsystemprofilespec)_ | OperatingSystemProfileSpec represents the operating system configuration spec. | + +[Back to top](#top) + +### OperatingSystemProfileSpec + +OperatingSystemProfileSpec represents the data in the newly created OperatingSystemProfile + +_Appears in:_ + +- [OperatingSystemProfile](#operatingsystemprofile) + +| Field | Description | +| --- | --- | +| `osName` _OperatingSystem_ | OSType represent the operating system name e.g: ubuntu | +| `osVersion` _string_ | OSVersion the version of the operating system | +| `version` _string_ | Version is the version of the operating System Profile | +| `supportedCloudProviders` _[CloudProviderSpec](#cloudproviderspec) array_ | SupportedCloudProviders represent the cloud providers that support the given operating system version | +| `bootstrapConfig` _[OSPConfig](#ospconfig)_ | BootstrapConfig is used for initial configuration of machine and to fetch the kubernetes secret that contains the provisioning config. | +| `provisioningConfig` _[OSPConfig](#ospconfig)_ | ProvisioningConfig is used for provisioning the worker node. | + +[Back to top](#top) + +### Unit + +Unit is a systemd unit used for the operating system config. + +_Appears in:_ + +- [OSCConfig](#oscconfig) +- [OSPConfig](#ospconfig) + +| Field | Description | +| --- | --- | +| `name` _string_ | Name is the name of a unit. | +| `enable` _boolean_ | Enable describes whether the unit is enabled or not. | +| `mask` _boolean_ | Mask describes whether the unit is masked or not. | +| `content` _string_ | Content is the unit's content. | +| `dropIns` _[DropIn](#dropin) array_ | DropIns is a list of drop-ins for this unit. | + +[Back to top](#top) diff --git a/data/products.yaml b/data/products.yaml index 0c0a1420d..3bf2b46e4 100644 --- a/data/products.yaml +++ b/data/products.yaml @@ -64,3 +64,10 @@ kubecarrier: name: v0.2 - release: v0.1 name: v0.1 +operatingsystemmanager: + name: operatingsystemmanager + logo: "/img/operatingsystemmanager/common/operating-system-manager-logo.png" + title: Operating System Manager + textName: Operating System Manager + description: Operating System Manager + weight: 4 diff --git a/static/img/operatingsystemmanager/common/operating-system-manager-logo.png b/static/img/operatingsystemmanager/common/operating-system-manager-logo.png new file mode 100644 index 000000000..730a04c19 Binary files /dev/null and b/static/img/operatingsystemmanager/common/operating-system-manager-logo.png differ