[Antrea 2.0] Drop support for Docker for Antrea on Windows #5630

antoninbas · 2023-10-27T18:22:43Z

Describe what you are trying to solve

The K8s documentation lists 2 supported container runtimes on Windows:

Containerd
Mirantis Container Runtime (MCR) - a commercial product

Antrea on Windows supports containerd, and we do not claim any support for MCR.
We also have a dedicated installation method for Docker (implying Docker CE / Moby, but should work for Docker Desktop as well), leveraging rancher/wins to manage Antrea as a Pod, but should we keep it?

Describe the solution you have in mind

I would like to propose dropping support for Docker on Windows in Antrea v2.0, for the following reasons:

the industry seems to have overwhelmingly adopted containerd as the container runtime on both Linux & Windows (I imagine that some enterprises that require commercial support may be using MCR)
rancher/wins doesn't seem to be an active project any more; all commits in the last year or so relate to dependency updates
the rancher/wins method of managing system components as Pods has been rendered largely obsolete by hostProcess Pods support on Windows. The fact that Docker (AFAIK) does not support hostProcess Pods may be why it is no listed as a supported container runtime in the K8s documentation. Maybe @jayunit100 has some perspective here.

In addition, we have these Antrea-specific reasons:

we no longer run CI jobs on Windows using Docker as the container runtime, according to @XinShuYang
we have 3 / 4 supported installation methods for the Antrea Agent on Windows, which can be confusing for users, and dropping one if it's no longer actively used would be beneficial

Describe how your solution impacts user flows

Antrea would no longer officially support Docker on Windows. We would remove instructions specific to Docker from the documentation.

Additional context

The Microsoft documentation lists the following container runtimes as supported on Windows:

Docker CE / Moby
Mirantis Container Runtime (MCR)
Containerd

The script for installing Docker CE still seems to be maintained: https://github.com/microsoft/Windows-Containers/tree/Main/helpful_tools

Docker Desktop is listed as the container runtime for Windows desktop, but:

Docker Desktop is not free, except for individual and small businesses
One would expect Windows servers to be used when building K8s clusters

Anthos clusters on VMware uses Antrea as the CNI for Windows Nodes (dataplane v2). The documentation states:

Containerd and Windows Dataplane V2 are now general availability as of version 1.11.

Docker and Flannel for Windows nodes will be deprecated in a subsequent release. We recommend that you update your configuration now, if applicable, to use containerd and Windows Dataplane V2 instead. See Update Windows Server configuration.

We are currently at version 1.16 of Anthos.

Even with the removal of dockershim in K8s v1.24, Docker can still be used as the container runtime thanks to https://github.com/Mirantis/cri-dockerd.

The text was updated successfully, but these errors were encountered:

jayunit100 · 2023-10-28T16:54:40Z

Yeah this is the right path forward. Host Process containers are better in all ways. Hope we can get ovs to install in a windows pod someday also! Or at least to bootstrap windows components however is needed.

But yeah. I think you'll not find much docker windows antrea need in the windows k8s community.

antoninbas · 2023-10-30T17:14:15Z

Thanks for the feedback @jayunit100. We recently added a new installation method (for containerd of course) where OVS userspace daemons (ovs-vswitch / ovsdb-server) are run as a Pod, making the installation process simpler.

The user-facing Windows document is updated as follows: * containerd will be the onyl supported container runtime on Windows Nodes, Docker support (that includes cri-dockerd, Docker Desktop) is officially deprecated. * we split the "Installation as Pod" section into 2 new sections (one for containerd, one for Docker). * installation methods are re-ordered, starting with the recommended / most common one: Installation as a Pod for containerd, Installation as Windows services, Installation as a Pod for Docker (deprecated). The main idea is to make the installation instructions clearer and easier to follow for a "normal" user, deploying the latest Antrea version on a recent K8s cluster. In this situation, the recommended installation method is to use antrea-windows-containerd.yml or antrea-windows-containerd-with-ovs.yml, which leverage HostProcess Windows Pods. There is still a lot of room for future improvements. I suggest that for Antrea v2.0 we drop all references to Docker and kube-proxy from this document. The "Antrea Management on Windows" section is also dropped from the Windows design document, as it was stale information. For antrea-io#5624 antrea-io#5630 Signed-off-by: Antonin Bas <antonin.bas@broadcom.com>

The user-facing Windows document is updated as follows: * containerd will be the only supported container runtime on Windows Nodes, Docker support (that includes cri-dockerd, Docker Desktop) is officially deprecated. * we split the "Installation as Pod" section into 2 new sections (one for containerd, one for Docker). * installation methods are re-ordered, starting with the recommended / most common one: Installation as a Pod for containerd, Installation as Windows services, Installation as a Pod for Docker (deprecated). The main idea is to make the installation instructions clearer and easier to follow for a "normal" user, deploying the latest Antrea version on a recent K8s cluster. In this situation, the recommended installation method is to use antrea-windows-containerd.yml or antrea-windows-containerd-with-ovs.yml, which leverage HostProcess Windows Pods. There is still a lot of room for future improvements. I suggest that for Antrea v2.0 we drop all references to Docker and kube-proxy from this document. The "Antrea Management on Windows" section is also dropped from the Windows design document, as it was stale information. For antrea-io#5624 antrea-io#5630 Signed-off-by: Antonin Bas <antonin.bas@broadcom.com>

The user-facing Windows document is updated as follows: * containerd will be the only supported container runtime on Windows Nodes, Docker support (that includes cri-dockerd, Docker Desktop) is officially deprecated. * we split the "Installation as Pod" section into 2 new sections (one for containerd, one for Docker). * installation methods are re-ordered, starting with the recommended / most common one: Installation as a Pod for containerd, Installation as Windows services, Installation as a Pod for Docker (deprecated). The main idea is to make the installation instructions clearer and easier to follow for a "normal" user, deploying the latest Antrea version on a recent K8s cluster. In this situation, the recommended installation method is to use antrea-windows-containerd.yml or antrea-windows-containerd-with-ovs.yml, which leverage HostProcess Windows Pods. There is still a lot of room for future improvements. I suggest that for Antrea v2.0 we drop all references to Docker and kube-proxy from this document. The "Antrea Management on Windows" section is also dropped from the Windows design document, as it was stale information. For #5624 #5630 Signed-off-by: Antonin Bas <antonin.bas@broadcom.com>

antoninbas · 2024-04-11T20:40:32Z

Now that #6019 has been merged, we just need to update the documentation, which I will take care of.

We update the Windows documentation to remove all Docker-specific instructions, and all mentions of (userspace) kube-proxy. We also update Windows scripts to remove Docker support. The Prepare-AntreaAgent.ps1 script was still defaulting to installing kube-proxy, so we update the script to change this default behavior. For all the scripts, "omitting" kube-proxy is now the default behavior. The corresponding script parameters have not been removed yet, but they are not officially deprecated, and we can remove them in a future release, such as Antrea v2.3. Fixes antrea-io#5630 Signed-off-by: Antonin Bas <antonin.bas@broadcom.com>

) * Finish removing Docker and userspace kube-proxy support We update the Windows documentation to remove all Docker-specific instructions, and all mentions of (userspace) kube-proxy. We also update Windows scripts to remove Docker support. The Prepare-AntreaAgent.ps1 script was still defaulting to installing kube-proxy, so we update the script to change this default behavior. For all the scripts, "omitting" kube-proxy is now the default behavior. The corresponding script parameters have not been removed yet, but they are not officially deprecated, and we can remove them in a future release, such as Antrea v2.3. Fixes #5630 * Rename Antrea Windows YAML manifests Now that containerd is the only supported container runtime, we rename antrea-windows-containerd.yml to antrea-windows.yml and antrea-windows-containerd-with-ovs.yml to antrea-windows-with-ovs.yml. Signed-off-by: Antonin Bas <antonin.bas@broadcom.com>

antoninbas added area/OS/windows Issues or PRs related to the Windows operating system. kind/design Categorizes issue or PR as related to design. labels Oct 27, 2023

This was referenced Oct 27, 2023

Re-organize user-facing Windows documentation #5624

Open

Antrea v2.0 #4832

Closed

XinShuYang mentioned this issue Oct 31, 2023

Update Windows Jenkins Jobs to Remove Docker Testbed #5641

Closed

antoninbas mentioned this issue Nov 13, 2023

Update Windows Jenkins Jobs to Remove Docker Testbed #5661

Merged

antoninbas mentioned this issue Dec 12, 2023

Update Windows documentation #5789

Merged

luolanzone added this to the Antrea v1.16 release milestone Jan 17, 2024

rajnkamr mentioned this issue Feb 5, 2024

Add Jenkins Job for Windows OVS containerization #5603

Open

antoninbas self-assigned this Apr 11, 2024

antoninbas mentioned this issue Apr 22, 2024

[Windows] Finish removing Docker and userspace kube-proxy support #6255

Merged

antoninbas closed this as completed in #6255 Apr 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Antrea 2.0] Drop support for Docker for Antrea on Windows #5630

[Antrea 2.0] Drop support for Docker for Antrea on Windows #5630

antoninbas commented Oct 27, 2023 •

edited

jayunit100 commented Oct 28, 2023

antoninbas commented Oct 30, 2023

antoninbas commented Apr 11, 2024

[Antrea 2.0] Drop support for Docker for Antrea on Windows #5630

[Antrea 2.0] Drop support for Docker for Antrea on Windows #5630

Comments

antoninbas commented Oct 27, 2023 • edited

jayunit100 commented Oct 28, 2023

antoninbas commented Oct 30, 2023

antoninbas commented Apr 11, 2024

antoninbas commented Oct 27, 2023 •

edited