New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker Swarm - Multiplatform Containers Will Not Start in Windows Server 2016 #27612

Open
walked opened this Issue Oct 21, 2016 · 8 comments

Comments

Projects
None yet
6 participants
@walked

walked commented Oct 21, 2016

Description

In a two node scenario (as simple as I can make it) with a linux swarm manager node, and a Windows Server 2016 worker node: when starting linux containers, published ports are made available to swarm services. Services published that utilize Windows containers will only start without published ports; if there is a published port the container will NOT start (in fact, never even gets created on the Windows Server 2016 host)

Steps to reproduce the issue:

  1. Create a 2 node swarm; Ubuntu 16.10 host as the swarm manager, and Windows Server 2016 as a worker node
  2. Create a service that uses the windows host; using a -p or --publish flag
  3. The container will never be created on the Windows Server 2016 worker node, and will always show 0/# replicas.

Describe the results you received:

Describe the results you expected:
The container should be published with a port exposed the same way; and with the actual container created and started on the Server 2016 worker node.

Additional information you deem important (e.g. issue happens only occasionally):
Ubuntu 16.10 manager node
Server 2016 worker node
Container does not matter; they will be instantiated correctly without published ports.

Output of docker version:
Manager node:
Client:
Version: 1.12.2
API version: 1.24
Go version: go1.6.3
Git commit: bb80604
Built: Tue Oct 11 18:29:41 2016
OS/Arch: linux/amd64

Server:
Version: 1.12.2
API version: 1.24
Go version: go1.6.3
Git commit: bb80604
Built: Tue Oct 11 18:29:41 2016
OS/Arch: linux/amd64

Worker node:
Client:
Version: 1.12.2-cs2-ws-beta
API version: 1.25
Go version: go1.7.1
Git commit: 050b611
Built: Tue Oct 11 02:35:40 2016
OS/Arch: windows/amd64

Server:
Version: 1.12.2-cs2-ws-beta
API version: 1.25
Go version: go1.7.1
Git commit: 050b611
Built: Tue Oct 11 02:35:40 2016
OS/Arch: windows/amd64

Output of docker info:
Manager node:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1
Server Version: 1.12.2
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 5
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge overlay null host
Swarm: active
NodeID: du6nihw6jhqfn88lns6goo8uu
Is Manager: true
ClusterID: 49b6idfno5ji7x2r1re93dlc2
Managers: 1
Nodes: 2
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Node Address: 192.168.1.229
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.8.0-22-generic
Operating System: Ubuntu 16.10
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.935 GiB
Name: ubuntu
ID: IXOY:3UZX:HUPF:5ILV:PH3V:44YV:O5SV:XXPM:YIXR:RIIV:GX2A:G3D4
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
127.0.0.0/8

Worker node:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 22
Server Version: 1.12.2-cs2-ws-beta
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: nat null overlay
Swarm: active
NodeID: 5esk3np5yjdyt8r2ufxk89xr8
Is Manager: false
Node Address: 192.168.1.89
Default Isolation: process
Kernel Version: 10.0 14393 (14393.321.amd64fre.rs1_release_inmarket.161004-2338)
Operating System: Windows Server 2016 Standard
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 3.999 GiB
Name: S2016-Docker
ID: D4EB:GDU7:6QCG:S4UQ:B2O6:MQNM:KFUE:GQ7D:ORMU:PBGW:SHQA:MMUU
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):
All exists within VMware workstation; but has been replicated on physical hosts as well.

@walked

This comment has been minimized.

Show comment
Hide comment
@walked

walked Oct 21, 2016

Additionally; if after a windows container is brought up (without publishing a port), executing:
docker service update --publish-add ### contId
Will immediately cause the container to exit on the windows worker

walked commented Oct 21, 2016

Additionally; if after a windows container is brought up (without publishing a port), executing:
docker service update --publish-add ### contId
Will immediately cause the container to exit on the windows worker

@friism

This comment has been minimized.

Show comment
Hide comment
@friism

friism Oct 21, 2016

Contributor

@walked thanks for trying this out - you made a lot of progress!

There are some limitations to what Docker can do on Windows and for now, those limitations mean that swarm-mode and multi-host overlay networking networking won't work on Windows. Neither a pure Windows swarm nor a mixed Linux-master/Windows-worker is expected work.

Reading this and recognizing the amount of work you put into trying to make this work, I realize that this should be in an FAQ or perhaps the daemon should warn.

Thanks again for trying to make this work and posting your progress!

Contributor

friism commented Oct 21, 2016

@walked thanks for trying this out - you made a lot of progress!

There are some limitations to what Docker can do on Windows and for now, those limitations mean that swarm-mode and multi-host overlay networking networking won't work on Windows. Neither a pure Windows swarm nor a mixed Linux-master/Windows-worker is expected work.

Reading this and recognizing the amount of work you put into trying to make this work, I realize that this should be in an FAQ or perhaps the daemon should warn.

Thanks again for trying to make this work and posting your progress!

@walked

This comment has been minimized.

Show comment
Hide comment
@walked

walked Oct 21, 2016

@friism that's what I expected to be the outcome. I was hopeful because you can join Windows as a worker and start services there would be networking support too! Darn.

Feel free to give me a shout if I can assist with anything on this front!

walked commented Oct 21, 2016

@friism that's what I expected to be the outcome. I was hopeful because you can join Windows as a worker and start services there would be networking support too! Darn.

Feel free to give me a shout if I can assist with anything on this front!

@friism

This comment has been minimized.

Show comment
Hide comment
@friism
Contributor

friism commented Oct 22, 2016

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Oct 22, 2016

Member

/cc @nishanttotla wondering if resolving the digest works from the client for multi-platform (perhaps it does, just checking)

Member

thaJeztah commented Oct 22, 2016

/cc @nishanttotla wondering if resolving the digest works from the client for multi-platform (perhaps it does, just checking)

@KingOfSpades

This comment has been minimized.

Show comment
Hide comment
@KingOfSpades

KingOfSpades Mar 1, 2017

Same issue still on Ubuntu + Server 2016 swarm

image

Any progress on this issue since?

KingOfSpades commented Mar 1, 2017

Same issue still on Ubuntu + Server 2016 swarm

image

Any progress on this issue since?

@pbosch75

This comment has been minimized.

Show comment
Hide comment
@pbosch75

pbosch75 Mar 3, 2017

Hi, I stumbled upon this problem myself when setting up a docker datacenter trial. Ive setup a Linux vm (since docker datacenter only works in Azure, AWS and local linux) with Docker Datacenter on it and tried to add my windows 2016 docker server as a worker node. This failed as you know.
This has to do with the overlay driver.
in case #30051 i found this has been picked up by MS and for windows 10 they incorporated it in changed in an update (which is the fastest way to deploy such a change).. So from the Microsoft perspective it's not possible (yet) from Windows 2016 server. But is possible in Windows 10 with the creator update. Should you have any questions regarding the Windows part, please contact Kalie (contact info in the case) .. more info also available in the case
Full url to the case : #30051
Hope this helps.
Kind regards,
Paul

pbosch75 commented Mar 3, 2017

Hi, I stumbled upon this problem myself when setting up a docker datacenter trial. Ive setup a Linux vm (since docker datacenter only works in Azure, AWS and local linux) with Docker Datacenter on it and tried to add my windows 2016 docker server as a worker node. This failed as you know.
This has to do with the overlay driver.
in case #30051 i found this has been picked up by MS and for windows 10 they incorporated it in changed in an update (which is the fastest way to deploy such a change).. So from the Microsoft perspective it's not possible (yet) from Windows 2016 server. But is possible in Windows 10 with the creator update. Should you have any questions regarding the Windows part, please contact Kalie (contact info in the case) .. more info also available in the case
Full url to the case : #30051
Hope this helps.
Kind regards,
Paul

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Mar 6, 2017

Member

I'm closing this issue, because overlay networking (thus swarm mode) is not yet supported on Windows Server 2016; it will be added in some future update (sorry, I don't have inside information on when that will be), but support has been added to Windows 10 in the "Creators Update", see the blog post here; https://blogs.technet.microsoft.com/virtualization/2017/02/09/overlay-network-driver-with-support-for-docker-swarm-mode-now-available-to-windows-insiders-on-windows-10/

Until an update for Windows Server 2016 arrives, Swarm mode is not supported.

I'm locking the conversation on this issue, to prevent it from becoming a kitchen sink of anything "swarm", "network" and "windows" related reports. If you have issues with the current Docker release, and suspect there's a bug at hand, please open a new issue with details.

Member

thaJeztah commented Mar 6, 2017

I'm closing this issue, because overlay networking (thus swarm mode) is not yet supported on Windows Server 2016; it will be added in some future update (sorry, I don't have inside information on when that will be), but support has been added to Windows 10 in the "Creators Update", see the blog post here; https://blogs.technet.microsoft.com/virtualization/2017/02/09/overlay-network-driver-with-support-for-docker-swarm-mode-now-available-to-windows-insiders-on-windows-10/

Until an update for Windows Server 2016 arrives, Swarm mode is not supported.

I'm locking the conversation on this issue, to prevent it from becoming a kitchen sink of anything "swarm", "network" and "windows" related reports. If you have issues with the current Docker release, and suspect there's a bug at hand, please open a new issue with details.

@moby moby locked and limited conversation to collaborators Mar 6, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.