crictl must work with containerd 1.5 - allow the fallback logic to CRI API v1alpha2

[SergeyKanzhelev]

This is a follow up from #878 (comment)

Containerd 1.5 only implements CRI API v1alpha2 and cri tools must support the same runtimes k8s supports. So there should be a fallback logic to v1alpha2.

[SergeyKanzhelev]

@saschagrunert what is the versioning story of CRI tools for CRI-O? How many versions back customers will expect to be supported?

[SergeyKanzhelev]

cc: @qiutongs

[saschagrunert]

@saschagrunert what is the versioning story of CRI tools for CRI-O? How many versions back customers will expect to be supported?

Since CRI-O versionizes together with Kubernetes, we would have any issue with a breaking change here. We also support CRI v1 since a couple of releases (v1.21 I guess), so we're already good to go.

[saschagrunert]

One way to approach this would be to switch to the internal runtime and image services completely:

cri-tools/cmd/crictl/main.go

Lines 135 to 137 in 8553b61

	func getRuntimeService(context *cli.Context) (internalapi.RuntimeService, error) {
	return remote.NewRemoteRuntimeService(RuntimeEndpoint, Timeout)
	}

This service already features the conversion of the types, but it would mean that we bound the kubelet code tighter to crictl.

I provided an example there: #884

If we agree that this is the right path forward, then I can take care of the other conversion as well.

[afbjorklund]

Same problem with Docker (cri-dockerd), I guess it deserves a special issue.

docker@minikube:~$ crictl --version
crictl version v1.21.0
docker@minikube:~$ crictl version
Version:  0.1.0
RuntimeName:  docker
RuntimeVersion:  20.10.12
RuntimeApiVersion:  1.41.0
docker@minikube:~$ cri-dockerd --version
cri-dockerd 0.2.0 (a4d1895)
docker@minikube:~$ crictl info
{
  "status": {
    "conditions": [
      {
        "type": "RuntimeReady",
        "status": true,
        "reason": "",
        "message": ""
      },
      {
        "type": "NetworkReady",
        "status": true,
        "reason": "",
        "message": ""
      }
    ]
  }
}

When upgrading cri-tools to latest/greatest (i.e. 1.24 pre-release), it no longer works:

docker@minikube:~$ ./crictl-1.23.0 --version
crictl version v1.23.0
docker@minikube:~$ ./crictl-1.23.0 info
{
  "status": {
    "conditions": [
      {
        "type": "RuntimeReady",
        "status": true,
        "reason": "",
        "message": ""
      },
      {
        "type": "NetworkReady",
        "status": true,
        "reason": "",
        "message": ""
      }
    ]
  }
}
docker@minikube:~$ ./crictl --version
crictl version 1.23.0-18-g0871ebbc
docker@minikube:~$ ./crictl info
FATA[0000] getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService 

https://github.com/Mirantis/cri-dockerd/tree/a4d1895a2659ea9974bd7528a706592ab8b74181/src

EDIT: The suggested PR (#883) seems to work OK.

[afbjorklund]

@saschagrunert

Since CRI-O versionizes together with Kubernetes, we would have any issue with a breaking change here. We also support CRI v1 since a couple of releases (v1.21 I guess), so we're already good to go.

I guess you mean "we would not have any" ? Normally CRI-O lags the Kubernetes releases for some time, but if it has already been implemented in a release version then I suppose you could* use cri-o v1.23.1 with kubernetes v1.24.0 ?

* I think we are currently using crio 1.22.1 with k8s 1.23.3.

docker@minikube:~$ ls /etc/apt/sources.list.d/
devel:kubic:libcontainers:stable.list  devel:kubic:libcontainers:stable:cri-o:1.22.list  docker.list

[afbjorklund]

@SergeyKanzhelev : Do you know when Kubernetes will switch to recommending containerd 1.5 ?

Currently still recommends 1.4:

https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd

Install the containerd.io package from the official Docker repositories. Instructions for setting up the Docker repository for your respective Linux distribution and installing the containerd.io package can be found at Install Docker Engine.

containerd.io | 1.4.12-1 | https://download.docker.com/linux/ubuntu focal/stable amd64 Packages

Maybe it will happen with k8s 1.24

[saschagrunert]

I guess you mean "we would not have any" ?

Yes, sorry for the confusion.

[afbjorklund]

Docker now installs containerd 1.5, fwiw

containerd.io |   1.5.10-1 | https://download.docker.com/linux/ubuntu focal/stable amd64 Packages

[saschagrunert]

Fixed with #889