Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support host.docker.internal DNS name to host #264

Open
Mahoney opened this issue Mar 31, 2018 · 207 comments
Open

Support host.docker.internal DNS name to host #264

Mahoney opened this issue Mar 31, 2018 · 207 comments

Comments

@Mahoney
Copy link

@Mahoney Mahoney commented Mar 31, 2018

  • This is a bug report
  • This is a feature request
  • I searched existing issues before opening this one

Expected behavior

As in docker-for-mac and docker-for-windows, inside a container, the DNS name host.docker.internal resolves to an IP address allowing network access to the host (roughly the output of ip -4 route list match 0/0 | cut -d' ' -f3 inside the same container).

Actual behavior

host.docker.internal resolves to nothing

Steps to reproduce the behavior

Execute docker run --rm alpine nslookup host.docker.internal

See it returns nslookup: can't resolve 'host.docker.internal': Name does not resolve

Output of docker version:

Client:
 Version:	18.03.0-ce
 API version:	1.37
 Go version:	go1.9.4
 Git commit:	0520e24
 Built:	Wed Mar 21 23:10:09 2018
 OS/Arch:	linux/amd64
 Experimental:	false
 Orchestrator:	swarm

Server:
 Engine:
  Version:	18.03.0-ce
  API version:	1.37 (minimum version 1.12)
  Go version:	go1.9.4
  Git commit:	0520e24
  Built:	Wed Mar 21 23:08:36 2018
  OS/Arch:	linux/amd64
  Experimental:	false

Output of docker info:

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 18.03.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfd04396dc68220d1cecbe686a6cc3aa5ce3667c
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.13.0-37-generic
Operating System: Ubuntu 17.10
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.947GiB
Name: rob-VirtualBox
ID: 3L2C:BTV3:TQO2:4SAG:XVW5:744G:MPWQ:62FK:56DP:KH3Z:EQ7Z:TBR5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.)

Running Ubuntu in VirtualBox 5.2.8 on OS/X 10.13.4

@bscheshirwork
Copy link

@bscheshirwork bscheshirwork commented Apr 2, 2018

same situation. I want to use this in docker-compose.yml for replace remote_host

    environment:
      XDEBUG_CONFIG: "remote_host=192.168.0.83 remote_port=9001 var_display_max_data=1024 var_display_max_depth=5"
$ docker-compose -f ~/projects/docker-yii2-app-advanced/docker-run/docker-compose.yml run --rm --entrypoint nslookup php "host.docker.internal"
Creating network "dockerrun_default" with the default driver
Creating dockerrun_mysql_1 ... done
Creating dockerrun_db_1    ... done
nslookup: can't resolve '(null)': Name does not resolve

nslookup: can't resolve 'host.docker.internal': Name does not resolve
...
Kernel Version: 4.4.0-116-generic
Operating System: Ubuntu 16.04.4 LTS
@bscheshirwork
Copy link

@bscheshirwork bscheshirwork commented Apr 2, 2018

https://docs.docker.com/docker-for-mac/networking/#use-cases-and-workarounds

I want to connect from a container to a service on the host

The host has a changing IP address (or none if you have no network access). From 18.03 onwards our recommendation is to connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host.

The gateway is also reachable as gateway.docker.internal.

@JuxhinDB
Copy link

@JuxhinDB JuxhinDB commented Apr 11, 2018

Same situation, when taking a closer look at /etc/hosts I notice the following:

172.17.0.2 f85e063d8c3e

Which suggests that it's just setting it to a random ID rather than host.docker.internal which is what I need.

@Zomis
Copy link

@Zomis Zomis commented Apr 15, 2018

When I tried to find out how to connect to host from Docker I found this question/answer on Stack Overflow: https://stackoverflow.com/a/43541732/1310566

I was not aware at the time that it only applied to macOS and Windows (it was just recently edited)

@atolia
Copy link

@atolia atolia commented May 1, 2018

While this feature not on linux I use

web:
  image: httpd:2.4
  volumes:
    - ......
  entrypoint: 
  - /bin/sh
  - -c 
  - ip -4 route list match 0/0 | awk '{print $$3" host.docker.internal"}' >> /etc/hosts && httpd-foreground 
@bscheshirwork
Copy link

@bscheshirwork bscheshirwork commented May 2, 2018

@atolia
This look like work with docker-compose exec web and don't work with docker-compose run --rm --entrypoint /bin/bash web

@hernandev
Copy link

@hernandev hernandev commented May 8, 2018

Mimic on that can be done with:

echo -e "`/sbin/ip route|awk '/default/ { print $3 }'`\tdocker.host.internal" | sudo tee -a /etc/hosts > /dev/null

Notice @atolia already provided a similar solution, but he is not considering non-privileged default USER, this one will work for non-root images with passwordless sudo available, for images where default user is root, just remove the sudo part.

This command will make docker.host.internal available regardless of the Docker version OR execution mode. I'm using this on entrypoint files.

@acuthbert
Copy link

@acuthbert acuthbert commented May 15, 2018

Firstly - docker is an amazing tool and so thank you for all who work tirelessly on it! As a leader of a large mixed team of linux and mac engineers this has been one of the biggest "why did we leave vagrant" questions I get hammered about when we called time on vagrant. It is very frustrating that this exists on mac and windows and not linux. We need connect back for xdebug and for letting selenium running in a container to access local urls for acceptance testing. This difference is bloating our build scripts with more and more fragility so it would be great if this was standardised. Is it not a worry if the same version of docker engineer on the 3 platforms can deviate in feature set?

@FX-HAO
Copy link

@FX-HAO FX-HAO commented May 16, 2018

docker.host.internal is still unavailable on my mac. And I can't connect to my host with 172.17.0.1.
My docker version:

Client:
 Version:      18.03.1-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   9ee9f40
 Built:        Thu Apr 26 07:13:02 2018
 OS/Arch:      darwin/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.03.1-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   9ee9f40
  Built:        Thu Apr 26 07:22:38 2018
  OS/Arch:      linux/amd64
  Experimental: true
@chaospixel
Copy link

@chaospixel chaospixel commented May 16, 2018

In case you missed it: docker.for.mac.host.internal and docker.for.mac.localhost do work - but only on docker for mac...

@FX-HAO
Copy link

@FX-HAO FX-HAO commented May 16, 2018

From 18.03 onwards our recommendation is to connect to the special DNS name host.docker.internal

Older aliases are deprecated in favor of this one. And I tried them, not working.

@mnd999
Copy link

@mnd999 mnd999 commented May 16, 2018

Not working here either (unsurprisingly).

@mnd999
Copy link

@mnd999 mnd999 commented May 16, 2018

@JuxhinDB it's not a random number, it's the container id. Still not useful though.

@jcoutch
Copy link

@jcoutch jcoutch commented May 18, 2018

I'm running a microsoft/dotnet-framework container on a Windows host (v18.03.1-cd-win65 17513), and host.docker.internal does not work.

@kunal-bhatia
Copy link

@kunal-bhatia kunal-bhatia commented May 22, 2018

Any idea, when will linux support for connecting to special DNS will be fixed?

@bscheshirwork
Copy link

@bscheshirwork bscheshirwork commented May 22, 2018

@kunalbhatia87 wait 4 resolve this issue? 😆

@rfay
Copy link

@rfay rfay commented May 23, 2018

There are several comments (and workarounds) here that mistakenly use "docker.host.internal", which I don't think was ever supported. The hostname we want to be supported is the one that's supported in Docker for Windows and Docker for Mac, "host.docker.internal"

@rfay
Copy link

@rfay rfay commented May 23, 2018

@Mahoney I think you should check "This is a bug report" in the OP. This is a bug. Docker team, please acknowledge it, thanks!

@Mahoney
Copy link
Author

@Mahoney Mahoney commented May 23, 2018

@rfay I'm not aware of this ever being a documented feature of Linux docker - as far as I can see it's only documented for docker-for-mac and docker-for-windows, and only as a recent change in each case. I couldn't find any discussion around the choice or anything to suggest it had been agreed as something all versions of Docker should implement, though it would make sense to me if it were.

So as far as I can see "feature request" rather than "bug" is the correct categorisation.

@brunosaboia
Copy link

@brunosaboia brunosaboia commented May 31, 2018

@Mahoney while I do agree that this is a feature request, it's an important one. In the end, you want your dev team to use a shared config file all across — the underlying OS should be indifferent.

For me, running Linux on a Mac-based team, it's very bad to have to either create and ignore the changes to a config file, or to have to create hosts entries in each VM to mimic Docker for Mac behavior. I think it's simpler to just have an additional entry on the Docker networking so that the host is always reachable using the same hostname.

@jtreminio
Copy link

@jtreminio jtreminio commented Jun 1, 2018

@brunosaboia What I do while waiting for this to be resolved is create Linux-specific config files and mount them via docker-compose.override.yml.

Not the perfect workaround, but it is the best solution right now.

@mrpink76
Copy link

@mrpink76 mrpink76 commented Jun 14, 2018

I'm running a windows container on a windows host:
microsoft/dotnet:2.1-aspnetcore-runtime-nanoserver-1709

host.docker.internal does not work to connect to a service on the host.

@Mahoney
Copy link
Author

@Mahoney Mahoney commented Jun 14, 2018

Issues with docker for windows should be raised in the docker/for-win repo after an appropriate search such as https://github.com/docker/for-win/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+host.docker.internal

@atrauzzi
Copy link

@atrauzzi atrauzzi commented Jun 14, 2018

Yeah, this definitely needs to be implemented so that we can have consistency.

Really, containers should always have had something like this anyway.

@webbby
Copy link

@webbby webbby commented Mar 28, 2021

Adding

    extra_hosts:
      - "host.docker.internal:host-gateway"

indocker-compose.yml to my container result in blocking all my curl request in container's Dockerfile
So the creation of the container just hangs at

Step 14/21 : RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
 ---> Running in 

and stops there

Ubuntu 18.04
Docker version 20.10.5, build 55c4c88
docker-compose version 1.26.2, build eefe0d31
@eitsupi
Copy link

@eitsupi eitsupi commented Mar 28, 2021

@webbby You may need to add that in the "build" section too (docker/compose#7323).

version: "3"
services:
  myservice:
    build:
      extra_hosts:
        - "host.docker.internal:host-gateway"
    extra_hosts:
      - "host.docker.internal:host-gateway"
@webbby
Copy link

@webbby webbby commented Mar 28, 2021

@eitsupi
That's invalid option

ERROR: The Compose file '././docker-compose.yml' is invalid because:
services.myservice.build contains unsupported option: 'extra_hosts'
@eitsupi
Copy link

@eitsupi eitsupi commented Mar 29, 2021

@webbby It may be because docker-compose is an older version.

$ docker --version
Docker version 20.10.5, build 55c4c88
$ docker-compose --version
docker-compose version 1.28.5, build c4eb3a1f
@jat001
Copy link

@jat001 jat001 commented Apr 4, 2021

@webbby It may be because docker-compose is an older version.

$ docker --version
Docker version 20.10.5, build 55c4c88
$ docker-compose --version
docker-compose version 1.28.5, build c4eb3a1f
Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 33, in <module>
    sys.exit(load_entry_point('docker-compose==1.28.5', 'console_scripts', 'docker-compose')())
  File "/usr/lib/python3.9/site-packages/compose/cli/main.py", line 80, in main
    command_func()
  File "/usr/lib/python3.9/site-packages/compose/cli/main.py", line 192, in perform_command
    handler(command, command_options)
  File "/usr/lib/python3.9/site-packages/compose/metrics/decorator.py", line 18, in wrapper
    result = fn(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/compose/cli/main.py", line 357, in build
    self.project.build(
  File "/usr/lib/python3.9/site-packages/compose/project.py", line 519, in build
    build_service(service)
  File "/usr/lib/python3.9/site-packages/compose/project.py", line 501, in build_service
    service.build(no_cache, pull, force_rm, memory, build_args, gzip, rm, silent, cli, progress)
  File "/usr/lib/python3.9/site-packages/compose/service.py", line 1131, in build
    all_events = list(stream_output(build_output, output_stream))
  File "/usr/lib/python3.9/site-packages/compose/progress_stream.py", line 22, in stream_output
    for event in utils.json_stream(output):
  File "/usr/lib/python3.9/site-packages/compose/utils.py", line 50, in split_buffer
    for data in stream_as_text(stream):
  File "/usr/lib/python3.9/site-packages/compose/utils.py", line 26, in stream_as_text
    for data in stream:
  File "/usr/lib/python3.9/site-packages/compose/service.py", line 1880, in build
    for host, ip in extra_hosts.items():
AttributeError: 'list' object has no attribute 'items'

extra_hosts in build only accept this format:

      extra_hosts:
        host.docker.internal: host-gateway
@djs55
Copy link

@djs55 djs55 commented Apr 8, 2021

@kpcyrd there's a fix in Docker Desktop for Mac 3.3.0 for host-gateway which should map the DNS name to the host's virtual IP, so it should be possible to connect to services running there. Previously host-gateway was accidentally mapped to a Linux IP which wasn't particularly useful. Let me know if it works any better or not!

@kpcyrd
Copy link

@kpcyrd kpcyrd commented Apr 9, 2021

@djs55 that's working nicely on both Linux and MacOS, thank you very much! 🙏

@carlwain74
Copy link

@carlwain74 carlwain74 commented Apr 12, 2021

Are the changes mentioned above for extra_hosts in 19.03 release?
I have to support Docker on EC2 and the latest version I see is 19.03.13ce-1.amzn2

@phillipuniverse
Copy link

@phillipuniverse phillipuniverse commented Apr 20, 2021

Thanks to the note from @djs55 after upgrading Docker for Mac to 3.3.0 I have a single compose file that works great on both macOS and Ubuntu 20.04 for host.docker.internal:

# Any version >= 2.3 will work here
version: '3'

services:
  pingtest:
    image: alpine
    extra_hosts:
      host.docker.internal: host-gateway

To verify, run it with:

docker-compose run --rm pingtest ping host.docker.internal

Are the changes mentioned above for extra_hosts in 19.03 release?

Yes, the extra_hosts option was added in 2.3 of the compose file format. According to these release notes this is compatible with Docker Engine 17.06.0+ and Docker Compose 1.16.0+.

@Favna Favna mentioned this issue Apr 20, 2021
8 of 8 tasks complete
@tmds
Copy link

@tmds tmds commented Apr 21, 2021

@phillipuniverse have you checked you can access TCP services that are bound on the host? Do these services need to be publicly accessible (e.g. 0.0.0.0 bound), or can you also access localhost services (e.g. 127.0.0.1 bound)?

@kpcyrd
Copy link

@kpcyrd kpcyrd commented Apr 21, 2021

@tmds yes, but you need to bind them to 0.0.0.0.

@tmds
Copy link

@tmds tmds commented Apr 22, 2021

yes, but you need to bind them to 0.0.0.0.

That makes them accessible from the physical network.
Is this also needed with Docker on Windows when using host.docker.internal? Or can those services use localhost bindings?

@phillipuniverse
Copy link

@phillipuniverse phillipuniverse commented Apr 24, 2021

@phillipuniverse have you checked you can access TCP services that are bound on the host? Do these services need to be publicly accessible (e.g. 0.0.0.0 bound), or can you also access localhost services (e.g. 127.0.0.1 bound)?

tl;dr - from what I can tell you can access localhost services using host.docker.internal.

I didn't fully validate this on Linux or WIndows, validated only on running Docker in macOS.

To test, I ran a Django service on my host macOS machine and bound to specific address on start. I added host.docker.internal and 192.168.0.96 to ALLOWED_HOSTS. On my local network my IP address is 192.168.0.96 and I tried accessing services using this IP from another machine on my network. Here are my results:

Bind to all IP addresses

python manage.py runserver 0.0.0.0:8025
  • wget localhost and wget 127.0.0.1 - 200
  • wget host.docker.internal:8025 from a container - 200
  • wget 192.168.0.96 - 200

Bind to localhost

python manage.py runserver 127.0.0.1:8025
  • wget localhost and wget 127.0.0.1 - 200
  • wget host.docker.internal:8025 from a container - 200 🎉
  • wget 192.168.0.96 - connection refused

Same results if I runserver localhost:8025.

Bind to macOS physical network IP

python manage.py runserver 192.168.0.96:8025
  • wget localhost and wget 127.0.0.1 - connection refused
  • wget host.docker.internal:8025 from a container - connection refused
  • wget 192.168.0.96 - 200

Bind to the macOS IP address on the docker network

I figured this out by looking at what host.docker.internal actually resolved to inside the container

python manage.py runserver 192.168.65.2:8025

Error when starting up Django:

Error: That IP address can't be assigned to.
@tmds
Copy link

@tmds tmds commented Apr 27, 2021

yes, but you need to bind them to 0.0.0.0.

I didn't fully validate this on Linux or WIndows, validated only on running Docker in macOS.

Based on these comments, it seems Docker on macOS and Windows allow containers to access services bound to the localhost.

On Linux, only services bound to the any address can be accessed.

The first is interesting during development. The second is more secure in production.

@dfinkel
Copy link

@dfinkel dfinkel commented Apr 27, 2021

Just to be clear, services can bind to the docker0 interface's address (often 172.17.0.1, but not guaranteed) instead of all interfaces (0.0.0.0).
The reason loopback bound services don't work on Linux, but they do on Windows and Mac is because there is a VM boundary which is proxying requests on those other two OSes. On Linux, the loopback is a private interface and the docker bridge network has no involvement with the lo0 interface.

@lucasbasquerotto
Copy link

@lucasbasquerotto lucasbasquerotto commented Apr 29, 2021

For anyone who wants to easily make sure that the host-gateway magical string works in their installed docker, just run:

docker run --rm --add-host host.docker.internal:host-gateway alpine cat /etc/hosts

Then see if it runs successfully and shows host.docker.internal pointing to the correct ip (in linux it will probably be 172.17.0.1).

@brjadams
Copy link

@brjadams brjadams commented May 5, 2021

even though I add

    extra_hosts:
        - "host.docker.internal:host-gateway"

When my gateway attempts to talk to a application running on the host machine at port 4200, i get:

Connection refused: host.docker.internal/172.17.0.1:4000

Is there any additional configuration needed that I am missing?

@lucasbasquerotto
Copy link

@lucasbasquerotto lucasbasquerotto commented May 5, 2021

running on the host machine at port 4200

host.docker.internal/172.17.0.1:4000

@brjadams It seems the port being used is wrong somewhere (or it's just a mistake in your post).

If the port is wrong only in your post (that is, the ports are correct in your setup), then make sure that host.docker.internal maps to a ip in the hosts file (enter your container with docker-compose exec <service> /bin/sh and then run cat /etc/hosts inside the container).

If the host host.docker.internal is mapping to an ip, make sure the service in the host machine is actualy running. If it's running, make sure it accepts connections from the container ip (or you can set it as 0.0.0.0 to accept from all ips, if this is not a security concern in your case), because if the service in the host only accepts from 127.0.0.1 you probably won't be able to connect from a container.

If it still doesn't work, then it might be that the host gateway ip is somehow wrong. You might try to update docker in this case, but I advise doing this only if all other cases haven't worked.

@bjmi
Copy link

@bjmi bjmi commented May 6, 2021

@brjadams Could it be a firewall problem? E.g. if Ubuntu UFW is installed then INPUT chain default policy is deny i.e. ports on the host machine interfaces aren't reachable until they are opened in the firewall.

hastinbe added a commit to hastinbe/laradock that referenced this issue May 7, 2021
Fixes laradock#2966 where current versions of laradock containers are already making DNS requests
to host.docker.internal which resolving NXDOMAIN when it should resolve.

See docker/for-linux#264
    moby/moby#40007
    https://docs.docker.com/engine/release-notes/#networking-2 for
    docker-engine 20.10

Signed-off-by: Beau Hastings <beau@saweet.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet