DNS lookup is slow

[larsnystrom]

Expected behavior

Near same latency connecting to example.com from container as from host.

Actual behavior

Connecting from container has ~2.5s latency.

Information

This is probably a dupe of #446 (which was closed due to lacking repro).

I can't provide the output of the diagnosis tool because it crashes my computer. I'm running version 17.03.1-ce-mac12 (17661)
Channel: stable
d1db12684b
on macOS 10.12.5

Steps to reproduce the behavior

Code to reproduce can be found at: https://github.com/larsnystrom/dfm-network-latency

Clone it, and then compare running curl from host and from the container. Here are my results:

$ curl -w '@format.txt' -o /dev/null -s http://example.com
    time_namelookup:  0,004
       time_connect:  0,099
    time_appconnect:  0,000
   time_pretransfer:  0,099
      time_redirect:  0,000
 time_starttransfer:  0,193
                    ----------
         time_total:  0,193
$ docker-compose up
Starting test-speed_1
Attaching to test-speed_1
test-speed_1  |     time_namelookup:  2.521
test-speed_1  |        time_connect:  2.622
test-speed_1  |     time_appconnect:  0.000
test-speed_1  |    time_pretransfer:  2.622
test-speed_1  |       time_redirect:  0.000
test-speed_1  |  time_starttransfer:  2.720
test-speed_1  |                     ----------
test-speed_1  |          time_total:  2.720
test-speed_1 exited with code 0

[larsnystrom]

An update: Since I wrote this ticket, I found #1112 which also seems to be a dupe of this.

I've also looked into where the bottleneck is, and it seems like it's Docker for Mac's built in DNS server that's slow.

Inside the container, dig says it takes ~2000ms to resolve google.com:

# dig google.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35735
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		206	IN	A	216.58.211.142

;; Query time: 2002 msec
;; SERVER: 192.168.65.1#53(192.168.65.1)
;; WHEN: Wed Jun 07 10:02:52 UTC 2017
;; MSG SIZE  rcvd: 44

However, if I switch from the DNS server at 192.168.65.1 to 8.8.8.8 the latency goes back to normal:

(On host):

$ docker build . -t test-speed
$ docker run -it --dns=8.8.8.8 test-speed
    time_namelookup:  0.029
       time_connect:  0.125
    time_appconnect:  0.000
   time_pretransfer:  0.125
      time_redirect:  0.000
 time_starttransfer:  0.221
                    ----------
         time_total:  0.222
$ docker run -it test-speed
    time_namelookup:  2.518
       time_connect:  2.619
    time_appconnect:  0.000
   time_pretransfer:  2.620
      time_redirect:  0.000
 time_starttransfer:  2.720
                    ----------
         time_total:  2.720

So, it seems it's Docker for Mac's DNS server that's just really slow.

[djs55]

I notice you're running the latest stable version. The latest DNS code in "edge" (which will soon be released to stable) has many bug fixes and improvements -- in particular it tries to use the Mac's native resolver rather than talk to upstreams itself. Could you try "edge" (available from https://docs.docker.com/docker-for-mac/install/#download-docker-for-mac) and let me know if the problem still manifests? If it still manifests, could you upload a fresh diagnostic?

Thanks for your report and for using Docker for Mac!

[larsnystrom]

I manually added "dns": ["192.168.0.1", "0.0.0.0"] (the same values that I have in the macOS Settings -> Network -> Advanced -> DNS) to the docker settings and the DNS resolution time went down to normal, so I've worked around the problem for now.

I'm a little hesitant to install beta software as I kind of need things working. Can I downgrade to stable after having installed edge?

[djs55]

@larsnystrom it's possible to downgrade from edge to stable but it forces you to reset to factory defaults. If you want to keep your containers and images and you have a workaround, then perhaps stick with stable -- there's a new update coming soon.

[larsnystrom]

I'm not sure what's happening, but today the DNS completely stopped working, so I guess my work-around was no good after all.

I'll try the edge version and report back.

[larsnystrom]

Now the container uses 127.0.0.11 to resolve domain names, and it only takes 4ms, so that looks good 👍 I'll run this for a while and go back to stable after the next release. Do you have an ETA for the stable update?

There's also some issue which seems to add a tab or something after each line break? But that has nothing to do with this issue. It makes the shell a little harder to use though.

$ docker-compose exec my-service bash -l
root@b3b2822669ad:/var/www/html# dig google.com

                                               ; <<>> DiG 9.9.5-9+deb8u11-Debian <<>> google.com
                ;; global options: +cmd
                                       ;; Got answer:
                                                     ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7095
                             ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

                 ;; QUESTION SECTION:
                                     ;google.com.			IN	A

                                                                               ;; ANSWER SECTION:
                 google.com.		137	IN	A	216.58.211.142

                                                                              ;; Query time: 1 msec
                   ;; SERVER: 127.0.0.11#53(127.0.0.11)
                                                       ;; WHEN: Thu Jun 08 09:53:35 UTC 2017
            ;; MSG SIZE  rcvd: 44

                                 root@b3b2822669ad:/var/www/html# 

[djs55]

Thanks for the update! Sorry about the terminal issue -- that'll be fixed in the next release (hopefully today/tomorrow). It's #1672

A workaround is to run reset in the terminal.

[djs55]

I believe this has been fixed in edge. Thanks for your report!

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked