Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal Host DNS cannot be resolved (Windows Container 1803) #1976

Open
n-junge opened this issue Apr 26, 2018 · 73 comments

Comments

@n-junge
Copy link

commented Apr 26, 2018

Expected behavior

According to the Docs host.docker.internal resolves the host's ip.

Actual behavior

Ping request could not find host host.docker.internal
Network works fine. Pinging Host IP directly (ipconfig) works as expected.

Host: Windows 10 Enterprise
Container: microsoft/windowsservercore:latest

Neither stable (18.03) nor edge (18.04) works.

Probably important side note: DNS can be resolved in Linux Container (Ubuntu)

 PS C:\> ping host.docker.internal
 Ping request could not find host host.docker.internal. Please check the name and try again.
 PS C:\> ping gateway.docker.internal
 Ping request could not find host gateway.docker.internal. Please check the name and try again.
 PS C:\> ping google.com
 
 Pinging google.com [172.217.16.78] with 32 bytes of data:
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 
 Ping statistics for 172.217.16.78:
     Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
 Approximate round trip times in milli-seconds:
     Minimum = 31ms, Maximum = 31ms, Average = 31ms
C:\WINDOWS\system32>docker version
Client:
 Version:       18.03.0-ce
 API version:   1.37
 Go version:    go1.9.4
 Git commit:    0520e24
 Built: Wed Mar 21 23:06:28 2018
 OS/Arch:       windows/amd64
 Experimental:  false
 Orchestrator:  swarm

Server:
 Engine:
  Version:      18.03.0-ce
  API version:  1.37 (minimum version 1.24)
  Go version:   go1.9.4
  Git commit:   0520e24
  Built:        Wed Mar 21 23:21:06 2018
  OS/Arch:      windows/amd64
  Experimental: false
@jasonbivins

This comment has been minimized.

Copy link

commented Apr 26, 2018

Hi @4xiz
Thanks for reporting - I can repro and have reported it as a bug

@ebriney ebriney self-assigned this May 2, 2018
@ebriney

This comment has been minimized.

Copy link
Member

commented May 2, 2018

It's a windows daemon limitation and needs to be fix there but we will add a workaround waiting MS to fix it upstream.

@n-junge

This comment has been minimized.

Copy link
Author

commented May 17, 2018

Release 18.05.0-ce-win66 works. Thanks.

@n-junge n-junge closed this May 17, 2018
@atomaras

This comment has been minimized.

Copy link

commented May 17, 2018

Tested it on Windows 10, Docker 18.05.0-ce-win66 (17760) and it doesn't work

image

image

image

@n-junge

This comment has been minimized.

Copy link
Author

commented May 18, 2018

@atomaras
Well the dns resolution seems to work for you too. So that's a different problem. I propose you open another issue.

@JoshCollinsMSFT

This comment has been minimized.

Copy link

commented Jun 20, 2018

I just tried this and it isn't working for me. Version info:

Windows
Microsoft Windows [Version 10.0.17134.112]

Docker

Client:
 Version:      18.05.0-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   f150324
 Built:        Wed May  9 22:12:05 2018
 OS/Arch:      windows/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.05.0-ce
  API version:  1.37 (minimum version 1.24)
  Go version:   go1.10.1
  Git commit:   f150324
  Built:        Wed May  9 22:29:00 2018
  OS/Arch:      windows/amd64
  Experimental: false

Ping output:

C:\>docker exec b61746027bc9 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.
@cpumanaz

This comment has been minimized.

Copy link

commented Jun 21, 2018

@JoshCollinsMSFT I have the same windows build (17134.112) and docker (18.05.0-ce) and am also not able to resolve host.docker.internal. No joy with connection strings.

@prochnowc

This comment has been minimized.

Copy link

commented Aug 14, 2018

Still not working for me with 18.06.0-ce:

docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.
@Heurazio

This comment has been minimized.

Copy link

commented Aug 19, 2018

Same problem. docker.host.internal not working.

@prochnowc

This comment has been minimized.

Copy link

commented Aug 20, 2018

@n-junge @ebriney could you please re-open this issue?

@kierzo

This comment has been minimized.

Copy link

commented Aug 23, 2018

So it seems the way this was fixed is that it must write out to the local hosts file on the host machine.

I have a bit of a situation as Symantec Endpoint blocks write access to the hosts file :(.
Is there a better dns way of doing this?
Thanks

@ebriney

This comment has been minimized.

Copy link
Member

commented Aug 24, 2018

They are 2 separate things.

First, for dockerd to get host.docker.internal resolution we must put it in windows hosts file...
So if you have endpoint protection or similar you must add an exception on com.docker.service executable.

Second, in the windows containers, we do a docker exec as soon as the container is started (using docker event) to patch the hosts file in the container (I don't know what happened if you are running in process isolation, because we don't support Windows server). So if you overwrite it at runtime you can lose those entries.

@Heurazio this is host.docker.internal, not docker.host.internal. you also have gateway.docker.internal but that is the same ip in windows containers.

@prochnowc can you try docker run microsoft/nanoserver powershell "Start-Sleep -s 2 ; if (! $(IPCONFIG /displayDNS | Select-String -Quiet -SimpleMatch host.docker.internal)) { exit 1 }"

@prochnowc

This comment has been minimized.

Copy link

commented Aug 24, 2018

@ebriney Interestingly ... if i wait 2 seconds after startup i can ping host.docker.internal. I will try this again within my traefik for windows container. traefik needs the gateway address.

@prochnowc

This comment has been minimized.

Copy link

commented Aug 24, 2018

Unfortunately it does not work.. traefik_1 | time="2018-08-25T01:30:05+02:00" level=error msg="Failed to retrieve information of the docker client and server host: error during connect: Get http://host.docker.internal:2375/v1.24/version: dial tcp: lookup host.docker.internal: getaddrinfow: This is usually a temporary error during hostname resolution and means that the local server did not receive a response from an authoritative server."

Does the host.docker.internal stuff require minimum nanoserver version?

@ebriney

This comment has been minimized.

Copy link
Member

commented Aug 27, 2018

@prochnowc, we execute that script as soon as we receive the start event from any windows containers:

$hostsFile = "C:\Windows\System32\drivers\etc\hosts"
$src = [System.IO.File]::ReadAllLines($hostsFile)
$a = $src += ""
For ($i=0; $i -le $a.length; $i++) {
	if ($a[$i].Contains("host.docker.internal"))
	{
		$a[$i] = "%s host.docker.internal"
		$a[$i+1] = "%s gateway.docker.internal"
		[System.IO.File]::WriteAllLines($hostsFile, [string[]]$a)
		exit
	}
}
$a = $a += "# Added by Docker for Windows"
$a = $a += "%s host.docker.internal"
$a = $a += "%s gateway.docker.internal"
$a = $a += "# End of section"
[System.IO.File]::WriteAllLines($hostsFile, [string[]]$a)

Perhaps the hosts file doesn't exist and there is an exception reading it.

@prochnowc

This comment has been minimized.

Copy link

commented Aug 27, 2018

@ebriney Am I right when saying that this requires working PowerShell in the windows container? My traefik container does not have PowerShell.

@ebriney

This comment has been minimized.

Copy link
Member

commented Aug 27, 2018

@prochnowc Arfff, that's why it didn't work. Do you have a suggestion on how to solve that?

@prochnowc

This comment has been minimized.

Copy link

commented Aug 27, 2018

@ebriney So actually this fix only works on windows containers which have PowerShell installed (which is not the case for windows nanoserver). I dont think the hosts file editing can be done with "standard" windows command line utils. My idea would be to implement this in the docker DNS.

@ebriney

This comment has been minimized.

Copy link
Member

commented Aug 27, 2018

@prochnowc hummm, nanoserver includes powershell (see command line above).
And yes your image must include powershell for now.
Microsoft is managing all the network part on WCOW so except if they handle it upstream we cannot do anything (but to do that I think it must be generalized in moby/moby).
On Linux containers the vm traffic is going throught vpnkit so we can tweak dns requests ...

@prochnowc

This comment has been minimized.

Copy link

commented Aug 29, 2018

@ebriney Sorry, but nanoserver does not include PowerShell by default. See https://docs.microsoft.com/en-us/windows-server/get-started/nano-in-semi-annual-channel. Your command line didnt work with pure nanoserver image. I had to use mcr.microsoft.com/powershell:6.0.2-nanoserver.

@ebriney

This comment has been minimized.

Copy link
Member

commented Aug 29, 2018

and so why docker run microsoft/nanoserver powershell "Start-Sleep -s 2 ; if (! $(IPCONFIG /displayDNS | Select-String -Quiet -SimpleMatch host.docker.internal)) { exit 1 }" is working?

@ebriney

This comment has been minimized.

Copy link
Member

commented Aug 29, 2018

ok that's in 1803 image

@kierzo

This comment has been minimized.

Copy link

commented Sep 3, 2018

@ebriney , Would you be able to update your powershell script slightly to only write to the host file if needed?

I can see you do an if string contains "host.docker.internal" but then also add the comment strings to the file after the check by the looks of it.

Because of this second write to the file outside of this condition, this one is causing me a bit of pain because I've had my I.T update my hosts file manually to fix this, but because of this second write statement that happens unconditionally, when I click switch to Windows containers, there is a "can't write to hosts file error" still and crashes Docker.

Also in my case the wrong host ip was getting selected too as we have multiple network adaptors.

A suggestion if you could make if possible, would be to first check the DNS resolves first, if it doesn't then check if the hosts file has the additions it needs, and write to it if its not in there, also would move the second write in to the if statement.

Hope this makes sense.

I've coded and tested my suggestions above (W10) in the below, so feel free to use parts or all of it if needed:
Really appreciate any help as ping ponging with my IT dept here on this which is painful.

$hostsFile = "${env:windir}\system32\drivers\etc\hosts"
# Check DNS resolves before touching the hosts file.
if((Resolve-DnsName -Name host.docker.internal) -And (Resolve-DnsName -Name gateway.docker.internal)){}
else {
    # Check if in the hosts file and write it in if its not there...
    if((cat $hostsFile | Select-String -Pattern "host.docker.internal") -And (cat $hostsFile | Select-String -Pattern "gateway.docker.internal")){}
    else {
        $ip = (Resolve-DNSName (Hostname) | Select IPAddress | select IPAddress -expandproperty IPAddress | select -last 1)
        $addtohost = "`n#Added by Docker for Windows`n$ip    host.docker.internal`n$ip    gateway.docker.internal`n# End of section`n"
        echo $addtohost | Out-File -Append -filepath $hostsFile
    }
}
@perithompson

This comment has been minimized.

Copy link

commented Jan 28, 2019

Hi @perithompson ,
I'm facing the same problem as you do, but using your hostname workaround leads my services to resolve using ipv6 only. Have you managed getting it work using ipv4?
Unfortunatly there is not option in docker for windows to disable ipv6 in general - as far as I'm aware.

Hi @noraab, Yeah, it resolves on ipv6 for me too, but that hasn't caused me a problem for what I needed, I don't know if there is a way to disable it, unless you can change the settings on the hosts virtual switch?

@noraab

This comment has been minimized.

Copy link

commented Jan 29, 2019

To provide more details:
This happens on Docker Version 18.09.1
This does not happen on Docker Version 18.09.0

@noraab

This comment has been minimized.

Copy link

commented Feb 1, 2019

To provide more details:
This happens on Docker Version 18.09.1
This does not happen on Docker Version 18.09.0

And some more findings:

  • This does not happen if you use Docker's default network.
  • This does happen, if you configure your own network within compose-file

I just defined my own network to have static IPs for each container and to pass this IP into the container as an environment - as I need it there for some RMI stuff configuration.

Now, I removed the custom network, and used the following code within my entrypoint to find out the dynamically assigned IP and write it into some variable:

setlocal enabledelayedexpansion
rem throw away everything except the IPv4 address line
for /f "usebackq tokens=*" %%a in (`ipconfig ^| findstr /i "ipv4"`) do (
  rem we have for example "IPv4 Address. . . . . . . . . . . : 192.168.42.78"
  rem split on : and get 2nd token
  for /f delims^=^:^ tokens^=2 %%b in ('echo %%a') do (
    rem we have " 192.168.42.78"
	set IP_ADDRESS=%%b
	set IP_ADDRESS=!IP_ADDRESS:~1!
    )
  )
)
(
endlocal
set IP_ADDRESS=%IP_ADDRESS%
)

Credits to this article on superuser

@MagicShoebox

This comment has been minimized.

Copy link

commented Feb 13, 2019

While playing around with something completely different, I found something possibly related.

The host names are injected into the container by this automatically executed docker command (taken from the Windows Event Log with docker daemon debug set to true):
(192.168.XXX.XXX is the host IP)

debug: exec commandLine: cmd.exe /C "ECHO 192.168.XXX.XXX host.docker.internal >> %systemroot%\system32\drivers\etc\hosts & ECHO 192.168.XXX.XXX gateway.docker.internal >> %systemroot%\system32\drivers\etc\hosts" [...]

The POST call that triggered this command was this:

debug: form data: {"AttachStderr":true,"AttachStdin":false,"AttachStdout":true,"Cmd":["cmd.exe","/C","ECHO 192.168.XXX.XXX host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 192.168.XXX.XXX gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts"],"Detach":false,"DetachKeys":"","Env":null,"Privileged":false,"Tty":false,"User":"Administrator","WorkingDir":""}

The key part is "User":"Administrator".
On nanoserver:1803, the built-in Administrator account is enabled with no password and it works fine.
On servercore:1803, the built-in Administrator account is disabled (with a password) and it rejects the command.

CreateProcess: failure in a Windows system call: The user name or password is incorrect. (0x52e) [Event Detail: Provider: 00000000-0000-0000-0000-000000000000] [Event Detail: Provider: 00000000-0000-0000-0000-000000000000] [Event Detail: onecore\vm\compute\management\orchestration\vmhostedcontainer\processmanagement.cpp(174)\vmcomputeagent.exe!00007FF7B648C00A: (caller: 00007FF7B645ECEA) Exception(2) tid(36c) 8007052E The user name or password is incorrect. CallContext:[\Bridge_ProcessMessage\ComputeSystemManager_ExecuteProcess\VmHostedContainer_ExecuteProcess] Provider: 00000000-0000-0000-0000-000000000000]

Steps to reproduce
docker run --rm -it mcr.microsoft.com/windows/nanoserver:1803
(in container)
type %systemroot%\system32\drivers\etc\hosts
(Notice the hosts are listed correctly)

docker run --rm -it mcr.microsoft.com/windows/servercore:1803
(in container)
type %systemroot%\system32\drivers\etc\hosts
(Notice the hosts are missing)

Strangely, when I tried putting the "type ...hosts" directly after docker run, the hosts were missing from BOTH
docker run --rm -it mcr.microsoft.com/windows/[flavor]:1803 cmd /C type %systemroot%\system32\drivers\etc\hosts
(Notice the hosts are missing)

Unfortunately, this doesn't seem to actually fix the ping, merely the lookup.

Still, Docker should probably use "User":"ContainerAdministrator" instead of "User":"Administrator" when adding the hosts, at least on 1803. Unfortunately I can't test either flavor of 1809 to see if they behave the same way.

As a side note, if anyone wants to get the hosts working on servercore:

EDIT: In case it's not clear, this gives Administrator (aka root) a blank password, just like nanoserver. Be sure you understand the security implications.

# escape=`

FROM mcr.microsoft.com/windows/servercore:1803

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

RUN Enable-LocalUser Administrator; `
    SecEdit.exe /export /cfg secpol.cfg; `
    (Get-Content secpol.cfg).Replace('PasswordComplexity = 1', 'PasswordComplexity = 0') | `
    Out-File secpol.cfg; `
    SecEdit.exe /configure /db C:\Windows\Security\Local.sdb /cfg secpol.cfg /areas SECURITYPOLICY; `
    Remove-Item secpol.cfg; `
    Set-LocalUser Administrator -Password (New-Object SecureString)
@mx-bernhard

This comment has been minimized.

Copy link

commented Feb 18, 2019

RUN Enable-LocalUser Administrator; SecEdit.exe /export /cfg secpol.cfg;
(Get-Content secpol.cfg).Replace('PasswordComplexity = 1', 'PasswordComplexity = 0') | Out-File secpol.cfg;
SecEdit.exe /configure /db C:\Windows\Security\Local.sdb /cfg secpol.cfg /areas SECURITYPOLICY; Remove-Item secpol.cfg;
Set-LocalUser Administrator -Password (New-Object SecureString)

We have an internal tool that is run within a container used for our CI builds based on 1803 that seems to be affected by the absence of an administrator account.
Using the ps-script above we were able to get our container running again. But here is the peculiar thing:

  • The problem (CreateProcess: failure in a Windows system call: The user name or password is incorrect. (0x52e)) occurs after a period of about two weeks after installing docker to the machine the first time or going back to factory defaults. Either of the two fixes the problem but once a certain period has passed the issue reappears
  • When the issue reappears, using the powershell script above fixes the problem for a certain time period
  • Once this has been executed on this machine in one container and by using "--rm", it works for all other containers. So we do not have to execute the powershell script each time we want a working container but apparently only once within the above mentioned period. So there seems to be some kind of escaping the sandbox and usage of a cache on the docker host here. We could also observe that without the powershell script, the hosts file still contains two entries when the container is working and no entries when it is not working.

EDIT:

  • we suspect that the two weeks interval comes from a new dynamic ip adress of the docker host, this requires the hosts file to be altered as it contains the docker host ip adress
  • changing the docker host file from inside a container seems to require an administrator account but only then
  • updating the hosts file within the docker container seems to update the hosts file of the docker host and thus fixes the problem for a certain time period for all other containers as well

EDIT 2:

  • observed with Docker Desktop 2.0.0.3 / Engine 18.09.2
@mx-bernhard

This comment has been minimized.

Copy link

commented Feb 19, 2019

Unfortunately, it was not just the dynamic ip address. We could not reproduce when our tool is causing the issue and when it is not.

@oatsoda

This comment has been minimized.

Copy link

commented Mar 12, 2019

It's a windows daemon limitation and needs to be fix there but we will add a workaround waiting MS to fix it upstream.

@ebriney Is this still the case? Can we track the MS issue anywhere? Thanks

@Kralizek

This comment has been minimized.

Copy link

commented Mar 19, 2019

Getting this issue

PS> docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.

Docker version

Client: Docker Engine - Community
 Version:           18.09.2
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        6247962
 Built:             Sun Feb 10 04:12:31 2019
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.2
  API version:      1.39 (minimum version 1.24)
  Go version:       go1.10.6
  Git commit:       6247962
  Built:            Sun Feb 10 04:28:48 2019
  OS/Arch:          windows/amd64
  Experimental:     true
@AndyVulhop

This comment has been minimized.

Copy link

commented Apr 4, 2019

I have the same issue with my container based on microsoft/aspnet:4.7.2-windowsservercore-1803. Trying to use host.docker.internal as the server for a db connection string to a sql database running locally on my host fails due to not being able to resolve host.docker.internal. I am also able to confirm that the host file inside the running container does not have records for host.docker.internal, nor for gateway.docker.internal. They are created and updated on my host machine's host file properly.

Please advise on a preferred workaround or a fix timetable.

PS C:\WINDOWS\system32> docker info
Containers: 13
 Running: 3
 Paused: 0
 Stopped: 10
Images: 15
Server Version: 18.09.2
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 17134 (17134.1.amd64fre.rs4_release.180410-1804)
Operating System: Windows 10 Pro Version 1803 (OS Build 17134.677)
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 15.88GiB
Name: OH-DUB-LAP-48XS
ID: YEOQ:S5SV:YTJS:FJRW:CCE2:HZGV:SF2V:BXVR:AXZZ:P5AT:736C:KQZD
Docker Root Dir: C:\ProgramData\Docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: -1
 Goroutines: 53
 System Time: 2019-04-03T22:05:04.3714794-04:00
 EventsListeners: 1
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
@alexvaut

This comment has been minimized.

Copy link

commented May 9, 2019

I can confirm I can reproduce on 18.09.2 with a windows server core image, more precisely: image 4.7.2-20190409-windowsservercore-ltsc2019.
Workaround script provided by @MagicShoebox fixes the issue.

@Cloudmersive

This comment has been minimized.

Copy link

commented May 11, 2019

Is this still not fixed? It's 2019 guys

@cpumanaz

This comment has been minimized.

Copy link

commented May 13, 2019

Why can the docker-compose/swarm style DNS server not be used here? The A record can be managed by the container host instead of a host file. All docker containers get this DNS server entry first with the host's DNS servers appended. Just set the A record to the primary IPv4 address of the host.

@Dweeberly

This comment has been minimized.

Copy link

commented May 13, 2019

Still broke for me too:

PS > docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Unable to find image 'microsoft/nanoserver:1709' locally
1709: Pulling from microsoft/nanoserver
407ada6e90de: Already exists
9ef95ce817ec: Pull complete
Digest: sha256:97272be4e3abcd71727095a6ab1948afc4c8412ad0649abfe60912651769a67d
Status: Downloaded newer image for microsoft/nanoserver:1709
Ping request could not find host host.docker.internal. Please check the name and try again.
PS > docker info
Containers: 6
 Running: 0
 Paused: 0
 Stopped: 6
Images: 6
Server Version: 18.09.2
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 16299 (16299.637.amd64fre.rs3_release_svc.180808-1748)
Operating System: Windows 10 Enterprise Version 1709 (OS Build 16299.1029)
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 63.89GiB
Name: xxxxxxx
ID: QQQQ:37AP:xxxx:MLOQ:Z4FE:xxxx:yyyy:RRYJ:xxxx:ZCCR:KKKK:xxxx
Docker Root Dir: C:\ProgramData\Docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: -1
 Goroutines: 39
 System Time: 2019-05-13T16:54:40.6304645-04:00
 EventsListeners: 2
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

I'll note that it was working but I had an extra 20GB in windowsfilter that I couldn't get rid of so I did a "factory reset". After that a running container could not resolve DNS, I could tracert my host by ip, but not by name. I brought down another (unrelated) vm I had running in hyperv, reset docker desktop and found this command still failed:

PS C:\Users\grgran\docker\nanops> docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.

However, it did start resolving DNS and I could tracert www.google.com without issues. Don't know if this helps but it appears ping'ing host.docker.internal is not a reliable test.

@jorioux

This comment has been minimized.

Copy link

commented May 28, 2019

Pinging host.docker.internal is not a reliable test. A true DNS test would be to ping another docker-compose service.

The issue is present on 10.0.16299, and I confirm it works on 10.0.17763.

@kiiwii

This comment has been minimized.

Copy link

commented Jul 5, 2019

Pinging host.docker.internal is not a reliable test. A true DNS test would be to ping another docker-compose service.
The issue is present on 10.0.16299, and I confirm it works on 10.0.17763.

agree: services of docker stack/swarm/compose, cannot ping to each other‘ internal DNS name from different swarm node.
but also disagree: not working on 10.0.17763.253, confirem as above ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

@danieldaeschle

This comment has been minimized.

Copy link

commented Aug 7, 2019

Is there any update?
It's still not working.

@achrjulien

This comment has been minimized.

Copy link

commented Aug 8, 2019

I don't believe it will ever be. Try to compile the windows binary yourself, copy it over the other one and it should work. https://github.com/rgl/docker-ce-windows-binaries-vagrant

@Heurazio

This comment has been minimized.

Copy link

commented Aug 11, 2019

Any update on this? After it has been resolved for some time, the problem is back after the latest update to 2.1.0.1 (37199).

@Heurazio

This comment has been minimized.

Copy link

commented Aug 11, 2019

Reinstallation of Docker for Windows solved the problem.

@kmukul123

This comment has been minimized.

Copy link

commented Sep 5, 2019

I am still seeing the problem, I see people have commented about the hostfile, the hostfile has an entry on the docker machine but not on the container machine so will it work on the container?
I have tried reinstalling restarting, resetting etc.

without this you cant have part of the solution working locally and part of it on docker containers.

@gillesgalipeau

This comment has been minimized.

Copy link

commented Sep 17, 2019

The problem still persist in our environment (moby 39110) since April. Does anyone have a fix for us?

@spex66

This comment has been minimized.

Copy link

commented Sep 27, 2019

Solution approach which seems to work is to write the gateway IP address (from within the container) to "drivers\etc\hosts" files, to make DNS resolving of "host.docker.internal" and "gateway.docker.internal" possible.

As the PS script from @kierzo has not worked for me using PowerShell Core

Most of the modules that ship as part of Windows (for example, DnsClient, Hyper-V, NetTCPIP, Storage, etc.) and other Microsoft products including Azure and Office have not been explicitly ported to .NET Core yet.
https://docs.microsoft.com/en-us/powershell/scripting/whats-new/what-s-new-in-powershell-core-60?view=powershell-6

here is a minimum version working for PWSH Core v6 (for me):

$hostsFile = "${env:windir}\system32\drivers\etc\hosts"
$ip = (ipconfig | where-object {$_ –match "Default Gateway"} | foreach-object{$_.Split(":")[1]})
$addtohost = "`n#Added by Docker for Windows`n$ip    host.docker.internal`n$ip    gateway.docker.internal`n# End of section`n"
echo $addtohost | Out-File -Append -filepath $hostsFile

added it to our entrypoint.ps1 script and from there one used host.docker.internal to connect to services running on host (outside container).

Now back to work after spending an hour to dig through all posts and options around this issue :)

Waiting for a proper solution from MSFT to replace this hack.

@ki11roy

This comment has been minimized.

Copy link

commented Sep 29, 2019

I am experiencing a random docker build failures (~40 minutes long builds on a build machine) and it seems that those somehow connected with this error (whole build fails because of the failed ECHO command ?)

[23:03:06.612][WindowsDaemon     ][Info   ] debug: start() completed [module=libcontainerd namespace=moby container=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd
04ef0c2d3e32039e3823ba0]
[23:03:06.612][WindowsDaemon     ][Info   ] sending event [module=libcontainerd namespace=moby event-info={eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3
e32039e3823ba0 init 25660 0 0001-01-01 00:00:00 +0000 UTC false <nil>} container=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0 event=start
]
[23:03:06.612][WindowsDaemon     ][Info   ] debug: Calling HEAD /_ping
[23:03:06.612][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/containers/eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0/json
[23:03:06.612][WindowsDaemon     ][Info   ] debug: Calling POST /v1.40/containers/eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0/exec
[23:03:06.612][WindowsDaemon     ][Info   ] debug: form data: {"AttachStderr":true,"AttachStdin":false,"AttachStdout":true,"Cmd":["cmd.exe","/C","ECHO 10.128
.2.243    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 10.128.2.243    gateway.docker.internal \u003e\u003e %sys
temroot%\\system32\\drivers\\etc\\hosts"],"Detach":false,"DetachKeys":"","Env":null,"Privileged":false,"Tty":false,"User":"Administrator","WorkingDir":""}
[23:03:06.612][WindowsDaemon     ][Info   ] debug: Calling POST /v1.40/exec/46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6db44e85a4388b0/start
[23:03:06.612][WindowsDaemon     ][Info   ] debug: form data: {"Detach":false,"Tty":false}
[23:03:06.612][WindowsDaemon     ][Info   ] debug: starting exec command 46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6db44e85a4388b0 in container eb719a
6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0
[23:03:06.612][WindowsDaemon     ][Info   ] debug: exec commandLine: cmd.exe /C "ECHO 10.128.2.243    host.docker.internal >> %systemroot%\system32\drivers\e
tc\hosts & ECHO 10.128.2.243    gateway.docker.internal >> %systemroot%\system32\drivers\etc\hosts" [exec=46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6d
b44e85a4388b0 module=libcontainerd namespace=moby container=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0]
[23:03:06.612][WindowsDaemon     ][Info   ] debug: hcsshim::ComputeSystem::CreateProcess - Begin Operation [cid=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef
0c2d3e32039e3823ba0]
[23:03:06.612][WindowsDaemon     ][Info   ] debug: HCS ComputeSystem Process Document [cid=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0 j
son={"CommandLine":"cmd.exe /C \"ECHO 10.128.2.243    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 10.128.2.243
   gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDirectory":"C:\\build","Environment":{"
COMPLUS_NGenProtectedProcess_FeatureEnabled":"0","GIT_ASKPASS":"C:\\git-credential-helper.bat","HOST_ADDRESS":"10.128.2.243","HOST_TOKEN":"683cb9d70552cbf762
20861aa96ba28f","NUGET_VERSION":"4.4.1","ROSLYN_COMPILER_LOCATION":"C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\MSBuild\\Current\\Bin
\\Roslyn"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]}]
[23:03:06.612][WindowsDaemon     ][Info   ] debug: attach: stderr: begin
[23:03:06.612][WindowsDaemon     ][Info   ] debug: attach: stdout: begin
[23:03:06.612][WindowsDaemon     ][Info   ] debug: HCS Result [json={"Error":-2147023570,"ErrorMessage":"The user name or password is incorrect."}]
[23:03:06.612][WindowsDaemon     ][Error  ] hcsshim::ComputeSystem::CreateProcess - End Operation - Error [cid=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0
c2d3e32039e3823ba0 error=CreateProcess eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0: The user name or password is incorrect.

and after that:

[23:03:06.613][WindowsDaemon     ][Error  ] Error running exec 46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6db44e85a4388b0 in container: container eb719
a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0 encountered an error during CreateProcess: failure in a Windows system call: The user name or pas
sword is incorrect. (0x52e) extra info: {"CommandLine":"cmd.exe /C \"ECHO 10.128.2.243    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\
etc\\hosts \u0026 ECHO 10.128.2.243    gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDir
ectory":"C:\\build","Environment":{"COMPLUS_NGenProtectedProcess_FeatureEnabled":"0","GIT_ASKPASS":"C:\\git-credential-helper.bat","HOST_ADDRESS":"10.128.2.2
43","HOST_TOKEN":"683cb9d70552cbf76220861aa96ba28f","NUGET_VERSION":"4.4.1","ROSLYN_COMPILER_LOCATION":"C:\\Program Files (x86)\\Microsoft Visual Studio\\201
9\\BuildTools\\MSBuild\\Current\\Bin\\Roslyn"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]}
[23:03:06.613][WindowsDaemon     ][Info   ] debug: attach: stdout: end
[23:03:06.613][WindowsDaemon     ][Info   ] debug: attach: stderr: end
[23:03:06.613][WindowsDaemon     ][Info   ] debug: attach done
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/exec/46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6db44e85a4388b0/json
[23:03:06.613][WindowsDaemon     ][Info   ] debug: clean 1 unused exec commands
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Client context cancelled, stop sending events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: clean 3 unused exec commands
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Client context cancelled, stop sending events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Client context cancelled, stop sending events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Build cancelled, killing and removing container: eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e
3823ba0
[23:03:06.614][WindowsDaemon     ][Info   ] debug: Sending kill signal 9 to container eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0

and the result is:

[23:03:07.641][WindowsDaemon     ][Info   ] debug: hcsshim::DestroyLayer - succeeded [path=D:\docker_windows\windowsfilter\0c92737e7b38f61d3bb4aed66d69280aa7
89283a35da07f5ce7802784ca8f6a6-removing]
[23:03:07.641][WindowsDaemon     ][Info   ] debug: Build cancelled (build cancelled): a non-zero code from ContainerWait: 3221225473

I've tried to apply @MagicShoebox script, but with no luck.

> docker info
Client:
 Debug Mode: false
 Plugins:
  app: Docker Application (Docker Inc., v0.8.0)
  buildx: Build with BuildKit (Docker Inc., v0.3.0-5-g5b97415-tp-docker)

Server:
 Containers: 3
  Running: 0
  Paused: 0
  Stopped: 3
 Images: 115
 Server Version: 19.03.2
 Storage Driver: windowsfilter (windows) lcow (linux)
  Windows:
  LCOW:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics l2bridge l2tunnel nat null overlay transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: process
 Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
 Operating System: Windows 10 Enterprise Version 1809 (OS Build 17763.737)
 OSType: windows
 Architecture: x86_64
 CPUs: 12
 Total Memory: 63.95GiB
 Name: BY1-CABuilds-01
 ID: RS5M:6DEO:RNHL:MYF4:CBRW:T6XD:E6CT:HMVF:HRXN:FSKU:MSJN:5GRM
 Docker Root Dir: D:\docker_windows
 Debug Mode: true
  File Descriptors: -1
  Goroutines: 33
  System Time: 2019-09-29T23:25:41.1348512+03:00
  EventsListeners: 1
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine
@brianharwell

This comment has been minimized.

Copy link

commented Oct 3, 2019

I am updating the hosts file when the container starts but there is an issue with that because the ip resolved in a container where hosts.docker.internal works is different from the gateway ip address used in the workaround.

@brianharwell

This comment has been minimized.

Copy link

commented Oct 17, 2019

I think I found the cause of the issue, when looking at the Docker logs for a different issue I found this...

[Event Detail: onecore\vm\compute\management\orchestration\vmhostedcontainer\processmanagement.cpp(173)\vmcomputeagent.exe!00007FF6000D9ADB: (caller: 00007FF60008DF1A) Exception(2) tid(4c4) 8007052E The user name or password is incorrect.
    CallContext:[\Bridge_ProcessMessage\VmHostedContainer_ExecuteProcess] 
 Provider: 00000000-0000-0000-0000-000000000000]
(extra info: {"CommandLine":"cmd.exe /C \"ECHO 192.168.1.204    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 192.168.1.204    gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDirectory":"/","Environment":{"COMPLUS_NGenProtectedProcess_FeatureEnabled":"0","ROSLYN_COMPILER_LOCATION":"c:\\\\RoslynCompilers\\\\tools"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]})]

The important thing to note is the error The user name or password is incorrect and the command seems to be trying to add host.docker.internal into the hosts file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.