-
Notifications
You must be signed in to change notification settings - Fork 14.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kubeadm cri installation instructions #10186
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -249,7 +249,7 @@ networking: | |
podSubnet: "" | ||
serviceSubnet: 10.96.0.0/12 | ||
nodeRegistration: | ||
criSocket: /var/run/dockershim.sock | ||
criSocket: /var/run/containerd/containerd.sock | ||
name: your-host-name | ||
taints: | ||
- effect: NoSchedule | ||
|
@@ -368,41 +368,22 @@ Here's a breakdown of what/why: | |
certificates from the `kube-apiserver` when the certificate expiration approaches. | ||
* `--cert-dir`the directory where the TLS certs are located. | ||
|
||
### Use kubeadm with other CRI runtimes | ||
### Use kubeadm with containerd | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd propose to leave this as it is and add setup instructions for at least containerd and CRI-O. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could add instructions for other runtimes with kubeadm to this page later on, does that sound good? We should definitely track these. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it would be better to leave links to other runtimes documentation even if we don't have our instructions for those. People at least would know that those runtimes exist and they can be also potentially used. |
||
|
||
Since v1.6.0, Kubernetes has enabled the use of CRI, Container Runtime Interface, by default. | ||
The container runtime used by default is Docker, which is enabled through the built-in | ||
`dockershim` CRI implementation inside of the `kubelet`. | ||
From v1.12.0 the suggested kubeadm CRI is containerd. For further information refer to [CRI Installation](/docs/setup/cri/cri-installation/) instructions. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you elaborate on this a bit? Why containerd is suggested kubeadm CRI? Why not CRI-O or both? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I believe this was due to the lack of tests with docker, not totally sure about CRI-O. I'll defer to @timothysc. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @neolit123 once we have CI-Signal back we need to double check the versions. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. my vote would be to not go with this statement for 1.12.0. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with @neolit123 here. Let's not promote switching to other runtimes just yet. Let's provide more information on how to set up CRI runtimes and how to use them. That should be enough for 1.12 I believe. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we are pending on decision for this. the state of tests from sig-node for containerd are yellow-ish: we do not have any tests for kubeadm and containerd yet, but our docker tests are passing at least. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
|
||
Other CRI-based runtimes include: | ||
|
||
- [cri-containerd](https://github.com/containerd/cri-containerd) | ||
- [cri-o](https://github.com/kubernetes-incubator/cri-o) | ||
- [frakti](https://github.com/kubernetes/frakti) | ||
- [rkt](https://github.com/kubernetes-incubator/rktlet) | ||
|
||
After you have successfully installed `kubeadm` and `kubelet`, execute | ||
these two additional steps: | ||
|
||
1. Install the runtime shim on every node, following the installation | ||
document in the runtime shim project listing above. | ||
|
||
1. Configure kubelet to use the remote CRI runtime. Please remember to change | ||
`RUNTIME_ENDPOINT` to your own value like `/var/run/{your_runtime}.sock`: | ||
|
||
```shell | ||
cat > /etc/systemd/system/kubelet.service.d/20-cri.conf <<EOF | ||
[Service] | ||
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --container-runtime-endpoint=$RUNTIME_ENDPOINT" | ||
EOF | ||
systemctl daemon-reload | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Instructions to update systemd config was quite useful. I'd suggest not to remove it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Given that this is now covered by kubeadm directly, I think it's safe to remove. The kubelet systemd example should live under kubelet docs. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Current documentation uses both ways of configuring kubelet - using configuration file and systemd drop-ins. In my opinion removing this can create confusion among users who prefer to use latter approach. |
||
After installing containerd, you should set `--cri-socket` in kubeadm init and kubeadm reset. Or, in alternative to command line flags, supply the containerd socket in your kubeadm configuration as shown in the example below: | ||
```yaml | ||
nodeRegistration: | ||
criSocket: /var/run/containerd/containerd.sock | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I did a fresh pull of containerd and the socket path I get is |
||
``` | ||
|
||
Now `kubelet` is ready to use the specified CRI runtime, and you can continue | ||
with the `kubeadm init` and `kubeadm join` workflow to deploy Kubernetes cluster. | ||
|
||
You may also want to set `--cri-socket` to `kubeadm init` and `kubeadm reset` when | ||
using an external CRI implementation. | ||
In addition, you should set kubectl flag `--container-runtime-endpoint` to containerd socket address. This can be done through kubeadm configuration as shown in the example below: | ||
```yaml | ||
nodeRegistration: | ||
kubeletExtraArgs: | ||
container-runtime-endpoint: unix:///var/run/containerd/containerd.sock | ||
``` | ||
|
||
### Using internal IPs in your cluster | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
title: Kubernetes CRI | ||
weight: 30 | ||
--- |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,140 @@ | ||
--- | ||
reviewers: | ||
- vincepri | ||
title: CRI installation | ||
content_template: templates/concept | ||
weight: 100 | ||
--- | ||
{{% capture overview %}} | ||
Since v1.6.0, Kubernetes has enabled the use of CRI, Container Runtime Interface, by default. | ||
This page contains installation instruction for various runtimes. | ||
|
||
{{% /capture %}} | ||
|
||
{{% capture body %}} | ||
|
||
### Containerd | ||
|
||
This section contains the necessary steps to use `containerd` as CRI with kubeadm. | ||
|
||
#### Prerequisites | ||
|
||
```shell | ||
modprobe overlay | ||
modprobe br_netfilter | ||
|
||
# Setup required sysctl params, these persist across reboots. | ||
cat > /etc/sysctl.d/99-kubernetes-cri.conf <<EOF | ||
net.bridge.bridge-nf-call-iptables = 1 | ||
net.ipv4.ip_forward = 1 | ||
net.bridge.bridge-nf-call-ip6tables = 1 | ||
EOF | ||
|
||
sysctl --system | ||
``` | ||
|
||
{{< tabs name="tab-cri-containerd-installation" >}} | ||
{{< tab name="Ubuntu 16.04+" codelang="bash" >}} | ||
apt-get install -y libseccomp2 | ||
{{< /tab >}} | ||
{{< tab name="CentOS/RHEL 7.4+" codelang="bash" >}} | ||
yum install -y libseccomp | ||
{{< /tab >}} | ||
{{< /tabs >}} | ||
|
||
#### Install containerd | ||
|
||
[Containerd releases](https://github.com/containerd/containerd/releases) are published regularly, the values below are hardcoded to the latest version available at the time of writing. Please check for newer versions and hashes [here](https://storage.googleapis.com/cri-containerd-release). | ||
|
||
```shell | ||
# Export required environment variables. | ||
export CONTAINERD_VERSION="1.1.2" | ||
export CONTAINERD_SHA256="d4ed54891e90a5d1a45e3e96464e2e8a4770cd380c21285ef5c9895c40549218" | ||
|
||
# Download containerd tar. | ||
wget https://storage.googleapis.com/cri-containerd-release/cri-containerd-${CONTAINERD_VERSION}.linux-amd64.tar.gz | ||
|
||
# Check hash. | ||
echo "${CONTAINERD_SHA256} cri-containerd-${CONTAINERD_VERSION}.linux-amd64.tar.gz" | sha256sum --check - | ||
|
||
# Unpack. | ||
tar --no-overwrite-dir -C / -xzf cri-containerd-${CONTAINERD_VERSION}.linux-amd64.tar.gz | ||
|
||
# Start containerd. | ||
systemctl start containerd | ||
``` | ||
|
||
### Docker | ||
|
||
{{< tabs name="tab-cri-docker-installation" >}} | ||
{{< tab name="Ubuntu 16.04" codelang="bash" >}} | ||
# Install prerequisites. | ||
apt-get install apt-transport-https ca-certificates curl software-properties-common | ||
|
||
# Download GPG key. | ||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - | ||
|
||
# Add docker apt repository. | ||
add-apt-repository \ | ||
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \ | ||
$(lsb_release -cs) \ | ||
stable" | ||
|
||
# Install docker. | ||
apt-get update && apt-get install docker-ce=17.03.2~ce-0~ubuntu-xenial | ||
|
||
# Setup daemon. | ||
cat > /etc/docker/daemon.json <<EOF | ||
{ | ||
"exec-opts": ["native.cgroupdriver=systemd"], | ||
"log-driver": "json-file", | ||
"log-opts": { | ||
"max-size": "100m" | ||
}, | ||
"storage-driver": "overlay2" | ||
} | ||
EOF | ||
|
||
mkdir -p /etc/systemd/system/docker.service.d | ||
|
||
# Restart docker. | ||
systemctl daemon-reload | ||
systemctl restart docker | ||
{{< /tab >}} | ||
{{< tab name="CentOS/RHEL 7.4+" codelang="bash" >}} | ||
# Install prerequisites. | ||
yum install yum-utils device-mapper-persistent-data lvm2 | ||
|
||
# Add docker repository. | ||
yum-config-manager \ | ||
--add-repo \ | ||
https://download.docker.com/linux/centos/docker-ce.repo | ||
|
||
# Install docker. | ||
apt-get update && yum install docker-ce-17.03.2.ce | ||
|
||
# Setup daemon. | ||
cat > /etc/docker/daemon.json <<EOF | ||
{ | ||
"exec-opts": ["native.cgroupdriver=systemd"], | ||
"log-driver": "json-file", | ||
"log-opts": { | ||
"max-size": "100m" | ||
}, | ||
"storage-driver": "overlay2", | ||
"storage-opts": [ | ||
"overlay2.override_kernel_check=true" | ||
] | ||
} | ||
EOF | ||
|
||
mkdir -p /etc/systemd/system/docker.service.d | ||
|
||
# Restart docker. | ||
systemctl daemon-reload | ||
systemctl restart docker | ||
{{< /tab >}} | ||
{{< /tabs >}} | ||
|
||
|
||
{{% /capture %}} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -79,57 +79,9 @@ The pod network plugin you use (see below) may also require certain ports to be | |
open. Since this differs with each pod network plugin, please see the | ||
documentation for the plugins about what port(s) those need. | ||
|
||
## Installing Docker | ||
## Installing a CRI | ||
|
||
On each of your machines, install Docker. | ||
Version 17.03 is recommended, but 1.11, 1.12 and 1.13 are known to work as well. | ||
Versions 17.06+ _might work_, but have not yet been tested and verified by the Kubernetes node team. | ||
Keep track of the latest verified Docker version in the Kubernetes release notes. | ||
|
||
Please proceed with executing the following commands based on your OS as root. You may become the root user by executing `sudo -i` after SSH-ing to each host. | ||
|
||
If you already have the required versions of the Docker installed, you can move on to next section. | ||
If not, you can use the following commands to install Docker on your system: | ||
|
||
{{< tabs name="docker_install" >}} | ||
{{% tab name="Ubuntu, Debian or HypriotOS" %}} | ||
Install Docker from Ubuntu's repositories: | ||
|
||
```bash | ||
apt-get update | ||
apt-get install -y docker.io | ||
``` | ||
|
||
or install Docker CE 17.03 from Docker's repositories for Ubuntu or Debian: | ||
|
||
```bash | ||
apt-get update | ||
apt-get install -y apt-transport-https ca-certificates curl software-properties-common | ||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - | ||
add-apt-repository "deb https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable" | ||
apt-get update && apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 17.03 | head -1 | awk '{print $3}') | ||
``` | ||
{{% /tab %}} | ||
{{% tab name="CentOS, RHEL or Fedora" %}} | ||
Install Docker using your operating system's bundled package: | ||
|
||
```bash | ||
yum install -y docker | ||
systemctl enable docker && systemctl start docker | ||
``` | ||
{{% /tab %}} | ||
{{% tab name="Container Linux" %}} | ||
Enable and start Docker: | ||
|
||
```bash | ||
systemctl enable docker && systemctl start docker | ||
``` | ||
{{% /tab %}} | ||
{{< /tabs >}} | ||
|
||
|
||
Refer to the [official Docker installation guides](https://docs.docker.com/engine/installation/) | ||
for more information. | ||
Refer to the [CRI installation](/docs/setup/cri/cri-installation/) guide for more information. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We will need to outline that Docker is still the default and if another CRI and outline the command line overrides in the kubeadm init / join below. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You might say something like: By default, kubernetes is configured to work with docker (18.06). In order to enable other CRIs please consult https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/ and https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/ instructions. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the docker version relevant? In the CRI docs instructions we're still suggesting to install 17.03, should we update it? This is especially relevant for Ubuntu 18.04. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. so that's a good question. these docs here seem out of date: i think this shows the truth: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we should include the information you just shared in the CRI document, under the Docker section. The There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it really depends of what version is tested with k8s, because we cannot recommend even a stable docker version if it's broken with k8s latest. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @timothysc thoughts/preferences on the above? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @vincepri we need to reflect this in the docs as recommended version. |
||
|
||
## Installing kubeadm, kubelet and kubectl | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd leave this as is as docker is still a default runtime.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, thanks!