Cannot set hosts in daemon configuration

[johnnyhoffman]

Expected behavior

Start docker with the value of "hosts" from the Daemon config

Actual behavior

Failure to start daemon because of conflict with -H flag

Information

• Diagnostic ID: D6EBFDA6-3FCC-4AD3-BD57-560DCB6D3905/2017-01-31_17-30-49
• Windows 10 Pro, Version 1607
• Docker version 1.13.0, build 49bf474

Steps to reproduce the behavior

1. Go to Settings in GUI
2. Go to the Daemon tab
3. Switch from Basic to Advanced
4. input the following config:

{
  "hosts": [
    "tcp://0.0.0.0:2376",
    "npipe://"
  ]
}

5. Press Apply

The error is unable to configure the Docker daemon with file C:\ProgramData\docker\config\daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [npipe:////./pipe/docker_engine_windows], from file: [tcp://0.0.0.0:2376 npipe://])

From this stack overflow post, and the GitHub issue reference from it, I can gather that this is because docker is by default running with a -H flag. However, the suggested fix does not seem to apply to Windows. Is there a Windows fix equivalent to setting DOCKER_OPTS to ""?

[dgageot]

@johnnyhoffman Thanks for the report. hosts settings is not supported on Docker for Windows. Can I ask you why you need this feature?

[johnnyhoffman]

@dgageot I wanted the feature so that I could securely communicate with docker on my test server (via the .NET Client for Docker Remote API). I assumed that hosts is/was supported in Docker for Windows because I was following this blog post which uses it in the same way I am attempting to use it.

The Daemon tab in the docker settings GUI tool points directly to this section of the documentation, which explicitly mentions hosts as an allowed configuration option. Perhaps the documentation should be updated, or the link in the GUI should point to a more up-to-date page.

[colltoaction]

I'm also interested in this feature. I have a powerful desktop machine and a laptop on the same network, and I'm trying to open up my desktop docker daemon so I can run all my Linux containers there.

[simonferquel]

We have now an internal ticket tracking this issue. Can't give a date on when it will be enabled though.

[friism]

@johnnyhoffman @tinchou if we enabled this, how would you want authentication to work?

• Windows AD login
• docker login, using your Docker ID
• None (not a good idea)

[colltoaction]

Whatever's best for cross-platform! If one day I move that daemon to a Linux host or a Mac host, ideally it'd be transparent to me. I'm only talking to a docker daemon to run Linux Containers, I don't care about the host.

[johnnyhoffman]

Thanks for the updates!
I agree with @tinchou - it would be preferable to have something OS agnostic.
My team doesn't use Windows AD, and it might be overkill to set it up for this use case.

[willseward]

Also without that option, I believe running things like the visualizer tool and portainer are prohibited. Correct me if I'm wrong.

[ericwj]

Perhaps this can be done really simply by putting the -H argument on the service itself so that it can be tweaked through Start parameters in Service Control Manager. You hid it and that makes this a bug...

@dgageot Can I ask you why you need this feature? I want to run swarm and to not be required to spend gigabytes of memory running docker on my dev box.

And I tried to run Docker on Nanoserver in Azure. But the docs I followed appear to be broken over this issue, as I found Docker remains inaccessible whatever I try. The error mentioned above is only visible running Docker from the start menu on Windows 10. The event log is swamped with useless events every 2 seconds, but this error isn't in it. That's a loss on Nanoserver certainly. The service should probably fail with an exit code as well. I suspect it exits gracefully even though dockerd crashed.

@friism Please, since you started about authentication, on Windows, get rid of the friggin PEM files and CA bundles is step 1. The Windows way is to find a suitable (root) certificate from dir Cert:\localmachine -Recurse and to bridge the Linux and Windows ways, configure a plain thumbprint in a config file - on Linux too.

One way to do authentication is to allow client certificates on TCP port 2376. And document a suitable New-SelfSignedCertificate call or two in the docs to create the root cert and those used for server and client authentication.

For dev machines, it's completely OK to allow remote access on TCP port 2375 without authentication. The firewall is on by default and blocks edge traversal. So only the local network can be given access even if docker is allowed through. Docker by default listens only on localhost, which is OK without authentication if *:2376 is configured with TLS and some form of authentication. But there is no reason to block configuring *:2375 instead of localhost:2375. My security is my problem. You can just warn.

The other option which enables at least Azure AD and possibly Docker and a world of alternatives on top of that is OpenID Connect.

[rioux602]

+1

[margic]

+1
Want this issue to run jenkins slaves in the background of a windows box.

[louisn]

+1
I also need this feature to run jenkins slaves as windows build agents. It's a blocker.

[mshareghi]

+1
The lack of support here presumably prevents configuring TLS over port 2376 (as documented here). If it is supported but these instructions are wrong, please advise with the correct method.

[chvndb]

Like @johnnyhoffman, I have been banging my head against the wall getting this to work, following this blog post.

This bug blocks me in my deploy process. I setup, develop and test my application locally. Then, I create a release of my application by creating tagged images which I push to a private remote docker registry. From my client I use my build system to then upgrade from the previous version of my application to the new. This fails as I need to be able to connect remotely to the docker host running on a Windows Server.

Is there any workaround in the meanwhile?

[norbinsh]

So, it's unclear what's the best approach? running on Debian, do we ignore the /etc/default/docker message saying not to configure the daemon there?
Should we edit the systemd unit instead?

[ericwj]

One way to hurt Microsoft is to let this drag so long people give up and switch to Linux?

[kiahmed]

Looks like they still haven't enabled it. I wanted to use host to be able to use docker engine api. So, are we saying api can't be used from a docker windows host? That't a bummer! Why auth is an issue? It could use any basic auth also with tls

[nicferrier]

I am having issues with this as well. I was trying to use docker for windows from WSL (WIndows Subsystem for Linux) because there are plenty of blog posts around saying it's possible.

There is also this FAQ https://docs.docker.com/docker-for-windows/faqs/#how-do-i-connect-to-the-remote-docker-engine-api which suggests that the host option was turned on for Docker for Windows at one point.

So I'm a bit confused by this thread which suggests that it wasn't.

Any clarification?

[kiahmed]

I created thread weeks ago on docker forum with screenshot but no response. Its turned off on windows what does that mean it works on Linux and mac? I have it on mac but mac has its own issue doesn't connect with network bridge (at least I couldn't final a solution yet).. Did anybody succeed using the engine api on Linux hosts?

[clivecrous]

Is there absolutely no way to get docker for windows to listen externally then? This is a huge issue for me.

[amrun]

Does anybody have an update on this? It is an issue of high importance, regarding that docker containers are intended to be highly portable and so on... :(

[msmith-techempower]

/remove-lifecycle stale

[petrosmm]

For anyone looking: I was able to get portainer connected and hosts working with LCOW (linux containers on windows) on using tcp docker.for.win.localhost:2375 instead of tcp localhost:2375.

Update: I have confirmed using this method can link containers while using LCOW... (eg. I had to link Matomo to a MariaDB instance)

[STaRDoGG]

..... sweet Moses .... this issue is almost 2 and a half years old, and still an ISSUE. Come on guys.

I'm trying to connect to a Docker Desktop Win 10 host via Portainer on a different machine. From reading this entire thread I can hack my way around it by switching to Windows containers, but apparently that uses LCOW which has some issues. Not a good solution.

[sentinel09]

I found docker settings "expose daemon on tcp://loclhost:2375 with TLS" checked may be work with Pycharm

This is ok, if you run docker staff locally. But I need to connect to this machine from another, and this does not work.

[atline]

Come on, guys, what happened? No official say YES or NO?

[brunofin]

Oh wow. I can't believe this had been going on for years.
Guys, I just want to connect to Docker running on another computer in my local network. It is not even NAT'ed on my router so I don't care about CA.

It's really simple. I just need to move some workload to another computer...

[kekru]

If you need to get access to a remote Docker Desktop machine, running in Linux mode, then a helper container should work:

• For unsecured connection: https://hub.docker.com/r/jarkt/docker-remote-api
• For TLS connection: https://hub.docker.com/r/kekru/docker-remote-api-tls

Be sure to open your Windows Firewall

[brunofin]

#453 (comment) @kekru thanks I'll make sure to check that out in the morning.

[atline]

@kekru Tried just now, this solution works.

[petrosmm]

If you need to get access to a remote Docker Desktop machine, running in Linux mode, then a helper container should work:

• For unsecured connection: https://hub.docker.com/r/jarkt/docker-remote-api
• For TLS connection: https://hub.docker.com/r/kekru/docker-remote-api-tls

Be sure to open your Windows Firewall

starred this, will def have a look!

[dmportella]

this been an "feature" for a long time. it is the same issue in mac setup too.
I dont think it will ever change.

[Aisuko]

Desktop for Mac "hosts": Cannot be used in Docker Desktop, 🤨

[omidkrad]

How can I get Docker Client on a Windows Server 2019 VM to connect to the Docker Engine on the host machine that is using Linux containers in a Moby VM? Anyone kind enough to give the steps to get this working?

[petrosmm]

@omidkrad https://stackoverflow.com/a/40789612/3997521 see the comment by Datz Apr 18 '18 at 10:38. I can confirm this has worked for me.

[omidkrad]

@petrosmm it's not from within a container, it's from a VM. I've added more details in this stackoverflow post. Thanks!

[omidkrad]

I got the answer to my question here: Make docker client in VM to attach to docker daemon on the host

Here's the summary:

Using Docker remotely on Windows

You can use this method for a configuration where you want to use Linux containers on a Hyper-V VM that does not run Linux containers locally.

Setup docker for remote connection

On the host machine run:

docker run -d --name docker-remote-api -p 2375:2375 -v /var/run/docker.sock:/var/run/docker.sock jarkt/docker-remote-api

On the client machine run:

Replace DOCKERHOST with the name of the host machine that is running docker engine.

docker -H=DOCKERHOST:2375 ps

or

$env:DOCKER_HOST="tcp://DOCKERHOST:2375"
docker ps

From here you can use docker commands on the local terminal. However, note that since the containers are hosted on the remote computer, any port you expose on the containers are also on the remote computer, and they are not available or tunneled on the local computer.

[abhishekmishra]

If you need to get access to a remote Docker Desktop machine, running in Linux mode, then a helper container should work:

* For unsecured connection: https://hub.docker.com/r/jarkt/docker-remote-api

* For TLS connection: https://hub.docker.com/r/kekru/docker-remote-api-tls

Be sure to open your Windows Firewall

This worked for me.

[JiajunZhou-ws]

If you need to get access to a remote Docker Desktop machine, running in Linux mode, then a helper container should work:

• For unsecured connection: https://hub.docker.com/r/jarkt/docker-remote-api
• For TLS connection: https://hub.docker.com/r/kekru/docker-remote-api-tls

Be sure to open your Windows Firewall

This worked for me! Such a great idea to proxy the request.

[docker-robott]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[ericblade]

FWIW, this is now over 3 years old, and still does not have a conclusive answer -- is this intended to be supported? What is the correct way for accessing a Windows Docker running Linux containers, from a remote machine? Why did it work several years ago, then stop?

[MrBuddyCasino]

Why is this closed? I was hoping I could run some containers on my beefy Windows machine, because my Macbook is struggling. But apparently not? This is weak.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked