504 when connecting to backend on docker-machine

[gabriel403]

Running a backend httpd server with

docker run -it --rm -e VIRTUAL_HOST=dev-app.test.com -e VIRTUAL_PORT=80 -p 8080:80 -v $(PWD):/usr/local/apache2/htdocs/ httpd

And a hosts file entry of

$ cat /etc/hosts
127.0.0.1   localhost
255.255.255.255 broadcasthost
::1             localhost


192.168.99.100  dev-app.test.com

With the proxy set up as

  web-proxy:
    container_name: web-proxy
    image: jwilder/nginx-proxy
    ports:
     - "80:80"
     - "443:443"
    volumes:
     - /var/run/docker.sock:/tmp/docker.sock:ro

visiting http://dev-app.test.com:8080/ responds as expected, visiting http://dev-app.test.com/ responds with

504 Gateway Time-out

nginx/1.9.9

Looking at the logs for the proxy container I see

web-proxy | nginx.1    | dev-app.test.com 192.168.99.1 - - [16/Feb/2016:21:39:17 +0000] "GET / HTTP/1.1" 504 584 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.109 Safari/537.36"
web-proxy | nginx.1    | 2016/02/16 21:40:18 [error] 48#48: *16 upstream timed out (110: Connection timed out) while connecting to upstream, client: 192.168.99.1, server: dev-app.test.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.17.0.2:80/favicon.ico", host: "dev-app.test.com", referrer: "http://dev-app.test.com/"
web-proxy | nginx.1    | dev-app.test.com 192.168.99.1 - - [16/Feb/2016:21:40:18 +0000] "GET /favicon.ico HTTP/1.1" 504 584 "http://dev-app.test.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.109 Safari/537.36"

Would really appreciate any help with this, I've got it running fine in production on ubuntu host (though different setup), but struggling somewhat when getting it going on docker machine on mac os

[wader]

Hmm weird, can you try from another container e.g. docker run --rm busybox wget http://$(docker inspect -f '{{.NetworkSettings.IPAddress}}' <httpd-container-id>):80?

Do you know if you have inter-container communication disabled? (--icc=false)

[gabriel403]

@wader

$ docker ps -al
CONTAINER ID        IMAGE               COMMAND              CREATED             STATUS              PORTS                  NAMES
eae2679ae16f        httpd               "httpd-foreground"   20 seconds ago      Up 19 seconds       0.0.0.0:8080->80/tcp   romantic_mestorf

gabriel at Etruscan in ~
$ docker run --rm busybox wget http://$(docker inspect -f '{{.NetworkSettings.IPAddress}}' romantic_mestorf):80
Connecting to 172.17.0.2:80 (172.17.0.2:80)
index.html           100% |*******************************| 94071   0:00:00 ETA

[wader]

Hmm even weirder 😏 can you try from the running nginx-proxy container? docker exec <nginx-proxy-container-id> wget http://$(docker inspect -f '{{.NetworkSettings.IPAddress}}' <httpd-container-id>):80

Can you see if the generated nginx config looks sane? correct IP:s etc? docker exec <nginx-proxy-container-id> cat /etc/nginx/conf.d/default.conf

[gabriel403]

@wader good news! it looks like it was down to starting the proxy via docker-compose and the other web container via a normal docker run. Could connect into the proxy container from other containers, but the proxy container doesn't appear to be able to reach outer containers. When both started as normal docker runs it worked fine, both started via docker-compose and the nginx config seems a bit off, will open a different issue for that though

[gabriel403]

@wader ah looks like I'm suffering from this now #304

[wader]

Ah ok, so you can't mix "old" docker networking and the new docker networking feature (used by docker-compose config version 2)?