-
Notifications
You must be signed in to change notification settings - Fork 39.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dockershim removal feedback & issues #106917
Comments
This comment was marked as outdated.
This comment was marked as outdated.
This information is confusing: https://kubernetes.io/docs/tasks/administer-cluster/migrating-from-dockershim/ "This section presents information you need to know when migrating from dockershim to other container runtimes." Shouldn't it be "when migrating from dockershim to other CRI implementations for Docker" ? i.e. going from dockershim to cri-dockerd "It is recommended to migrate from dockershim to alternative container runtimes." Shouldn't it be "it is recommended to migrate from dockerd to alternative container runtimes" ? i.e. going from dockerd to containerd We saw the same thing happen with CRI-O, before. As the saying goes: "now you have two problems". It is especially confusing when the docs still installs containerd from docker.com instead of containerd.io. Moving from dockershim to cri-dockerd should be mandatory, since CRI will be required in 1.24 and up. Moving from dockerd to containerd (or cri-o) could still be optional, and requires more user intervention. |
A transition to cri-dockerd first might be smoother, indeed. There is especially a concern for business with a lot of clusters, with a lot of docker nodes. If they cannot move to alternative runtimes due to bugs, these bugs will have to be reported and resolved. Requiring things like a host OS update with the CR update is not great. cri-dockerd is currently a third party project with an undefined maintenance state. Because of that it is difficult to maintain a k8s.io guide for it. The new guide for migrating to containerd seems like the best option for the time being. At the same time we don't want users to be stuck with a 1.23 kubelet. Note: the k8s skew policy allows nodes to continue to have a 1.23 kubelet against a 1.24 and 1.25 kube-apiserver, but that would only delay the required changes. |
If I understand correctly, the plan is to remove all support (for docker) from the documentation and from the supporting tools like kubeadm and leave users to make their own adventure (or to pay for Docker or Mirantis)
For minikube it seems like continuing with docker runtime is doable, even if it means having to build our own CRI binaries and do our own CRI configuration. We do have containerd available as opt-in, since a long time back. |
if cri-dockerd becomes a maintained and documented project, i don't see a problem for k8s.io to link and mention it as part of CR setup. https://kubernetes.io/docs/setup/production-environment/container-runtimes/
cross-linking from k8s.io to cri-dockerd guides still make sense.
testing is difficult if the CRI implementation is external and unmaintained. if cri-dockerd ramp up on the maintenance i can see kubeadm adding back e2e tests for docker. we switched all test jobs to containerd for the time being.
cri-dockerd need to start making releases and have a support matrix against the kubelet especially WRT the CRI API version that is in use by both.
i would not recommend for minikube and users of k8s to start using cri-dockerd until the project is actually maintained, has docs and releases. if Mirantis cannot staff the project, they'd have to grant community ownership. |
I think Mirantis had plenty of time for that, with the delay... Don't think moving to 1.25 would have helped ? Was referring to the auto-detection and the recommendations, it does work to use
cri-dockerd now sets up EDIT: path changed to Users of minikube are free to choose their kubernetes version and the container runtime, using parameters.. If they want to stick with k8s 1.23, or move to containerd, they are free to do so. Otherwise, it is inevitable. But it doesn't look to be too much worse than cri-o... Tested in another product only, scarce releases, etc. |
Linking my issue about local-up-cluster.sh not working nicely due to the dockershim removal: #106972 |
This comment has been minimized.
This comment has been minimized.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
@mrbobbytables 👋 I'm in the release 1.24 bug triage team. Since this issue is marked as a release blocker , want to find out if this is a placeholder issue? Let me know how we could track the progress as we go deeper into the release timeline. |
Currently there are no (working) upstream instructions for how to install the CRI for containerd and for docker. There are inline instructions for containerd (downloading from docker.com, not from containerd.io) and for CRI-O. It's not blocking, but it's not ideal. You would want the Kubernetes page to either have good info or link to good info ? https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ --> https://kubernetes.io/docs/setup/production-environment/container-runtimes/
EDIT: I think it is a real risk, trying to use a corporate website for project documentation. Should have "cri-dockerd.io" |
Adding a flag to
Currently we are bundling the pre-release cri-dockerd The default values for Kubernetes 1.23 and earlier: dockershim (internal) And yes, you can choose to run They always run the external (remote) socket, since there was no internal "shim" for these two container runtimes. |
Linking the Docs Dockershim Removal project board |
Looks like "kubenet" is still available in cri-dockerd, maybe spoke too soon after reading those other issues... Perhaps it wouldn't be a bad idea to start using CNI anyway, and use a "bridge" example like those other CR ? |
Hey all. As we approach the release for Kubernetes 1.24 (scheduled just under a month away on 19th April 2022), we're keeping track of this issue in the release team. Currently this is tracked as a release blocker, but after discussions with the SIG Docs team (cc @nate-double-u) we're under the impression that the docs are in a good enough ™️ state to no longer be blocking. I'd like to get community feedback on removing the
If people are happy, then I'll go ahead and remove the label. |
@justaugustus you had marked this release-blocker way back in December. Looks like we are in much better shape now. So we should remove the release-blocker tag. please revisit this when you get a chance (if you still think this is one). @JamesLaverack +1 to remove |
I would also +1 removing |
the decision seems up to release, but there are a number of open PRs (maybe some are not tracked in https://github.com/orgs/kubernetes/projects/67): kubernetes/website#30882 first one is rather important and has been in review since December. |
I will probably revert my change to stop using "kubenet", since it is still available (unlike indications to the contrary). * those other code changes are not until 1.27 though Hopefully we will have time to fix minikube before the beta release, so that it works (i.e. with docker) without them ? I think we can live with the other issues such as no release and no documentation, at least for 1.24 (maybe not 1.25) But it seems like all warnings about trying to use it were warranted, and we better hurry up with the
"Docker cri networking managed by network plugin kubernetes.io/no-op". Plan B is to stick with the current k8s releases (1.23 and 1.24) and utilizing the version skew policy to get past 1.25. So if you still require "docker" container runtime, you will get an older version of k8s when compared to "containerd". |
In addition, remove kubeadm check for
|
@vsxen are you sure these are dockershim only? Also socat and iptables forwarding are different things, AFAIK. Happy to cleanup some if these checks but we need proof. |
I only find socat in there for br_ netfilter it depends on which cni plugin you use, for example some cni plugin (cilium ovs) don't use bridge. |
removing |
We could move all dockershim dependencies to preflight warnings instead of errors in 1.24. We can also remove them, but cri-dockerd must document its dependencies first.
This one is trickier. If some CNI (which ones? are they 'popular'?) need it, it might be better to continue throwing errors. Preflight errors can be ignored with the CLI flag / config. |
It seems the developer documentation for cri-dockerd gives just enough information to get started, but not to complete... https://github.com/Mirantis/cri-dockerd#build-and-install Missing some requirements like os version and go version, and missing the version linker flags that are provided by |
/milestone clear /lifecycle frozen |
The docker runtime has now changed the default to "kubenet", which means that user needs to install CNI plugins.
https://github.com/Mirantis/cri-dockerd/releases/tag/v0.2.4 (previously it would use the Docker network, by default) |
And now it is up to "cni" by default, and "no-op" by choice. |
Now the docker removal is complete, since it was not supported in time for the (initial) 1.26 release. It will probably be done retroactively in 1.27 timeframe, like how CNI was added retroactively with 1.25. Will look to nerdctl for picking up the ball, I would rather not resort to having to use the proprietary tools. |
It's been exactly a year. I am gonna close this now. please open fresh issues as needed in appropriate repositories. /close |
@dims: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Please use this issue for feedback and issues regarding dockershim removal in the 1.24 release.
/triage accepted
/milestone v1.24
/sig docs node
/area docker
The text was updated successfully, but these errors were encountered: