feat: Windows Container Support. Fixes #1507 and #1383

[lippertmarkus]

This PR adds support for running Windows Containers in Argo workflows. Fixes #1507, fixes #1383.

What I did:

• Extracted OS specific functionality into os_specific package under workflow/executor/os-specific and used*_windows.go or *_linux.go suffixes to enable conditional compilation. With these changes argoexec can be compiled for Windows and Linux. Also cross-compilation works fine. As the newly introduced functions are one-liners I don't think creating unit tests for them would make sense.

• Extended the Makefile part for argoexec and wrote a Dockerfile.windows doing the whole build of argoexec inside of Windows containers. I tried to create the Dockerfile as similar as possible to the existing one to make maintenance more simple. Running docker build -t myargoexec -f .\Dockerfile.windows --build-arg=DEV=1 --target argoexec . on a Windows Server 2019 node creates a Docker Image for Windows with argoexec.exe like shown in this build log.

• Changed all examples in docs and the manifests to include the nodeSelector for kubernetes.io/os: linux. This makes sure that the argo-server and the workflow-controller aren't scheduled on a Windows node in a hybrid cluster.

• I added documentation on how to get started with Windows containers inside of workflows as well as some example manifests. I think the documentation clearly shows the advantages of such a hybrid setup. My favourite are the hybrid workflows enabling e.g. to build an application on Windows and Linux within a single Workflow!

Notes for testing:

1. Have a look at docs/windows.md. You need to set up a Kubernetes cluster, add a Windows node like described here and set up Argo.

2. You can download the sources of the PR to the Windows Server 2019 node and build the argoexec image yourself via docker build -t myargoexec -f .\Dockerfile.windows --build-arg=DEV=1 --target argoexec . I also pushed a pre-built image to lippertmarkus/argoexec:v2.7.4 which you can use.

3. For now, on your Windows node you need to re-tag the built or pulled argoexec image to match the one used by Argo, e.g. via docker tag myargoexec argoproj/argoexec:v2.7.4. Make sure to use the correct version. This ensures that Docker uses the already existing argoexec Docker image for Windows when running the workflow instead of trying to pull the image for Linux from Docker Hub. This isn't necessary for the end user when we publish multi-platform images to Docker Hub like described below.

4. Change your workflow controller configmap to use the kubelet executor and schedule some Windows or hybrid workflows with Argo CLI like described in docs/windows.md.

TODO:

• As the automated Docker builds can't build Docker images for Windows, we need to use a CI service to build the image and push it to the Docker Hub. I had good experience with AppVeyor which is free for public projects. The build should create a multi-platform image so that the argoexec image for the correct OS will be pulled automatically when the workflow is started. I did set up an example multiarch AppVeyor build config for argoexec. You can also view an example build of the multiarch image here. This example multiarch image is pushed to my DockerHub Repo.

• A Windows node should be added to the environment for the e2e tests and those tests should also test the added example workflows including Windows containers. As k3s doesn't provide Windows support yet, we maybe need to introduce a separate e2e environment. I never used CircleCI, but as far as I understand it's possible to create multiple Orbs (VMs) for Linux and Windows. Nevertheless, I think it won't be possible for those VMs to communicate to each other to manually initialize a hybrid cluster. I also don't know another CI services where this would be possible, but we could create such an e2e testing environment manually within GCP, AWS or Azure or elsewhere.

• This currently doesn't work with docker workflow executor as the hostPath volume is set to the fixed Socket type here. For Windows the type must be set to null like shown in Correctly handle named pipe host mounts for Windows kubernetes/kubernetes#69484 (comment). Also to support real hybrid clusters then, we would need the dockerSockPath in the workflow executor configmap and the just mentioned type to be different for Windows and Linux.

[CLAassistant]

All committers have signed the CLA.

[lippertmarkus]

@alexec any chance someone could have a look into this or do I miss something in the PR? I think cross-plattform workflows would really add value 🙂

[alexec]

Thank you for this.

You can fix the build errors by running make codegen then make lint.

No one in the core team runs Windows. Would you be willing to support this going forward?

Is there anyway to build the Windows image on Linux so we can publish it?

[lippertmarkus]

Thanks for your feedback, the build seems to be successful now. I would try to support this as far as my spare time allows.

Unfortunately there is no way to build this on Linux but as mentioned we could use any CI service supporting Windows and maybe just extend your CircleCI setup with Windows orbs to build the Windows image. For the upcoming versions of the argoexec docker image we would need to do the following:

• Include the OS/arch in the tag for the Linux and Windows image, e.g. argoproj/argoexec:v2.9-linux and argoproj/argoexec:v2.9-windows
• Create a manifest argoproj/argoexec:v2.9 referencing those two tags within the build. Here is an example on how to do this.

We should also consider adding the docker build for windows as a regular CI check for this repo.

[alexec]

Ok. Thank you. Can I request you update the doc to tell people how to build the Windows executor please?

[lippertmarkus]

I added the documentation

[alexec]

Great stuff -- thank you.

[alexec]

Can you please run make docs and I'll merge.

[lippertmarkus]

done :)

[sonarcloud]

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

0.0% Coverage
0.0% Duplication

[alexec]

🚀