Docker: Switch daemon from Windows containers to Linux containers

[philippgille]

I need to use Linux containers for a build. From using Docker for Windows on Windows 10 Pro I knew you can switch the container type in the GUI. Searching for a way to do it via command line lead me to this article, mentioning the following command: & 'C:\Program Files\Docker\Docker\DockerCli.exe' -SwitchDaemon.

I tried that in a PowerShell script that's being executed in a before_build step in AppVeyor, but now the builds get stuck completely during executing that command.

The AppVeyor documentation says AppVeyor is running Docker 17.03.1-ee and uses Windows Server 2016 as OS when using Visual Studio 2017 as image, so maybe this version doesn't support switching the container type in the daemon? There's a forum post that describes how to install a Docker version on Windows Server 2016 that does support it, but it only links to the CE version and says to install the Edge (Windows Server 2016) version, but doesn't mentioned which version exactly that is. Docker EE doesn't have an edge channel, but on the other hand the post is from June, so maybe the CE edge features are in the regular Enterprise Edition already? Or maybe Docker 17.03.1-ee does support it but nested virtualization or something similar is not activated in the AppVeyor Hyper-V VMs?

Being able to build software in a Linux container on a Continuous Integration cloud service that runs on Windows would be the killer feature!

[staticfloat]

I would also like this; I have a project that actually uses a Linux cross-compiler to build binaries for Windows, it'd be great to get this continually tested on AppVeyor.

[paddie]

Any updates on this one? We'd love to use Appveyor to build docker images instead of travis etc. and this is a show-stopper for us.

[FeodorFitsner]

You are probably referring to Docker Community Edition (Docker CE) which is available on Windows 10 only, though AppVeyor platform is built on Windows Server 2016 and Docker there is Docker Enterprise Edition which is unable to run Linux containers as Docker CE.

However, we are looking into offering Linux builds pretty soon - hopefully it will allow you to build Linux containers.

As alternative solution which we can also consider is enabling nested virtualization on Windows Server 2016 VMs and then having a Linux image there, so during a build a Linux VM with can be run inside Windows VM and controlled by the same Docker client.

[paddie]

Both options are fine with us. The real issue we're having currently is that we're converting some large'ish systems to .NetCore running on linux servers. Due to the size of the systems, we're having to do it in multiple steps, meaning we're building both .Net and netcore2.0 services in the same solution and that is currently breaking due to the limitations you've described.

We're looking into ways of getting around it, but it is extremely frustrating.

[philippgille]

@FeodorFitsner

You are probably referring to Docker Community Edition (Docker CE) which is available on Windows 10 only, though AppVeyor platform is built on Windows Server 2016

From my initial issue description:

There's a forum post that describes how to install a Docker version on Windows Server 2016 that does support it, but it only links to the CE version and says to install the Edge (Windows Server 2016) version, but doesn't mentioned which version exactly that is.

So one idea is that you could offer another image, similar to the Visual Studio 2017 one, but with the Docker CE edge version that supports switching between Windows and Linux containers. You could name the image "Docker CE edge" or something similar, and mark it as beta or with limited support. But users willing to experiment could already make use of the feature.

[philippgille]

@FeodorFitsner: Actually, Windows Server and Docker moved forward since I created this issue and it now is possible to run Linux containers in Windows Server natively without Hyper-V:

https://www.thomasmaurer.ch/2017/10/how-to-run-docker-linux-container-on-windows-server-1709/

[stats-build]

Our team would very much like to be able to run Linux containers on Windows Server 2016 as well.

[FeodorFitsner]

@philippgille thanks for the tip - I'll look at that article

[IlyaFinkelshteyn]

@philippgille this article states the following in the Create Container Host Virtual Machine section:

If you run Docker on a physical server you can skip this step. If you want to run Docker Containers using Linux inside a Virtual Machine running on Hyper-V you should enable Nested Virtualization for the Container Host Virtual Machine.

So it is still requires Hyper-V or maybe I misunderstand something...

[philippgille]

@IlyaFinkelshteyn: The way I understand that paragraph is:

• If you have a physical server and it runs Windows Server 2016, then you can install Docker and switch between Windows and Linux containers
• If you have Windows Server 2016 running inside a Hyper-V VM and want to switch between Windows and Linux containers, then you need to enable Nested Virtualization

Otherwise the author wouldn't say "you can skip this step" and then the next steps go on with how to switch to Linux containers on a Windows Server 2016.

[IlyaFinkelshteyn]

@philippgille AppVeyor build workers are Hyper-V VMs.

[philippgille]

@IlyaFinkelshteyn: Well, then you can turn on nested virtualization to make it work, as mentioned in that article.

[scara]

you can turn on nested virtualization to make it work, as mentioned in that article.

On the Visual Studio 2017 image which looks, like to me, running on top of w10 1709, now: https://www.appveyor.com/updates/2017/11/18/ and #1554.
Never tried by myself, I'm interested too.

TIA,
Matteo

[solvingj]

I tried myself, looks like it still doesn't have the DockerCLI.exe tool to switch the mode.

https://ci.appveyor.com/project/solvingj/appveyor-docker-linux-test/build/1.0.9#L4

[cmdkoh]

Also would like to have Docker support for running cross platform test (currently using Travis for Linux and Appveyor for Windows, would like to just use one CI with private repo).

[RouR]

Any progress here?
I guess Appveyor team must configure build servers.
https://forums.docker.com/t/linux-container-in-w2k16/26321

[jhecking]

I'm interested in the ability to run Linux-based Docker images on the Windows build servers as well. My use case is client server software, where the client runs on Windows and Linux but the server only runs on Linux. To run the client's integration tests, I need to be able to spin up a server instance using a Linux container.

[ojii]

We would also like to run Linux containers on appveyor to test windows clients interacting with our linux servers.

[asampat3090]

Is there any progress here? We're also trying to run linux containers on the Windows build.

[solvingj]

After working with LCOW on Jenkins for a bit, I can tell you that it will likely be a little while. I don't expect appveyor to try to support it before it's GA, and that seems a long ways away based on various comments i've seen. For example:
moby/moby#36469 (comment)

[xt0rted]

I'd like to see linux container support too. I'm trying to run Azurite for some of my tests but the npm package blocks so the build script hangs, and the docker image is using node:alpine which results in an error of

docker : image operating system "linux" cannot be used on this platform

[kiview]

@FeodorFitsner As this issue has been closed, does this mean we are now able to switch the daemon, or run it in hybrid mode?

[IlyaFinkelshteyn]

Please check this blog, sorry log post :)
https://ci.appveyor.com/project/appveyor-tests/docker-ce/build/1.0.29

[xt0rted]

@IlyaFinkelshteyn linux containers are only available with a paid upgrade, are there plans to bring support for this to the existing workers as well?

[FeodorFitsner]

@xt0rted Currently, supporting Linux Containers on Windows (LCOW) requires running "MobyLinuxVM" Hyper-V VM (with actual Linux Docker in it) - it's not just installing Docker CE. While existing "Standard"-tier VMs with 2 CPU cores and 4 GB of dynamic (burstable) RAM are great for the most CI/CD tasks their resources, unfortunately, are not enough for smooth run of the main build flow and any containers inside nested "MobyLinuxVM" VM (taking 2 GB itself).

"Premium"-tier VMs have 4 CPU cores and 7 GB of static RAM. Besides LCOW these beefy VMs have other applications such as faster builds of heavy code-bases, faster parallel tests that can't be broken into multiple jobs.

[xt0rted]

Thanks for the clarification @FeodorFitsner