Support for communicating with Docker over TCP

[ryfow]

I'm trying to set up the nginx reverse proxy as a docker container.

To do that, it'd be nice if docker-gen supported the DOCKER_HOST environment variable or a command-line parameter specifying how to communicate with Docker.

[thaJeztah]

+1 Was just trying to achieve the same thing to test this thing out

Communicating via TCP should not be required, mounting docker.sock inside the container should work, e.g.:

docker run -v /var/run/docker.sock:/docker.sock ….

Note that mounting docker.sock inside a container at /var/run/ does not work, other locations do (for example /docker.sock). Currently, the location of docker.sock is hardcoded inside https://github.com/jwilder/docker-gen/blob/master/docker-gen.go#L213 and https://github.com/jwilder/docker-gen/blob/master/docker_client.go#L43

I tried to change those lines and re-build docker-gen but couldn't get it to work. Only spent a couple of minutes so maybe it's just my lack of go-experience or a misconfiguration of the docker-gen parameters.

@jwilder any chance on providing a Dockerized nginx proxy on index.docker.io? (preferably a 'trusted build')

I like the simplicity of your software for simple development stuff (compared to more advanced systems like skydock etc).

[jwilder]

@ryfow I agree. The hard-coded unix domain socket is not ideal.

[jwilder]

@thaJeztah I'll look into getting docker-gen to run as a container. I have not tried that since I usually just run it on the host. An trusted build would be convenient though.

[thaJeztah]

@jwilder thanks a lot in advance! Now that I got the 'hang' of docker, I'd like keep my host clean, especially when testing new software :)

[jwilder]

I pushed some changes to master that add support for DOCKER_HOST as well as a new -endpoint flag via the command-line. The format is tcp://127.0.0.1:4243 or unix:///var/run/docker.sock.

[cer]

As far as I can tell the -endpoint flag will always ignored:

if endpoint == "" && os.Getenv("DOCKER_HOST") != "" {
    endpoint = os.Getenv("DOCKER_HOST")
} else {
    endpoint = "unix:///var/run/docker.sock"
}

[jwilder]

@cer You're right. Should be fixed now.

[jwilder]

@thaJeztah Created a trusted build: https://index.docker.io/u/jwilder/nginx-proxy/ Can you give it a try?

[bradleyg]

This works really well!

[ekristen]

Works for me.

[thaJeztah]

@jwilder Not really sure what the problem is on my side; I'm running a container that exposes two ports (22 and 80) nginx-proxy tries to use port 22 (i.e. the 49xxx port that is assigned for port 22) as the upstream port.

If I don't expose port 22, nginx-proxy doesn't pick-up the port and tries to use http://ip-address: (note the trailing :) as the upstream address.

When exposing both port 22 and 80:

2014/05/08 17:21:56 [error] 30#0: *1 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: somedomainname.tld, request: "GET / HTTP/1.1", upstream: "http://172.17.42.1:49232/", host: "somedomainname.tld"

2014/05/08 17:21:56 [error] 30#0: *1 readv() failed (104: Connection reset by peer) while reading upstream, client: xxx.xxx.xxx.xxx, server: somedomainname.tld, 
request: "GET / HTTP/1.1", upstream: "http://172.17.42.1:49232/", host: "somedomainname.tld"

Which is to be expected, since this is the SSH port.

If I only expose port 80;

2014/05/08 15:44:48 [emerg] 38#0: invalid port in upstream "172.17.42.1:" in /etc/nginx/sites-enabled/default:6

And this is the configuration that was generated;

upstream somedomainname.tld {
        server 172.17.42.1:;
}

server {
        gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

        server_name somedomainname.tld;

        location / {
            proxy_pass http://somedomainname.tld;
            include /etc/nginx/proxy_params;
        }
}

_Note:_

Replaced the actual domain-name with 'somedomainname.tld'. I'm using a '.xip.io' domain-name for testing, e.g. something.123.123.123.123.xip.io, but don't think this is related

[jwilder]

@thaJeztah I'll see if I can reproduce that. My guess is that the template is just assuming one port is exposed and this line: https://github.com/jwilder/nginx-proxy/blob/master/nginx.tmpl#L5 is just picking the first address.

[thaJeztah]

@jwilder that was what I was thinking. Therefore tried to only expose port 80, in which case it completely stripped the port from the configuration, which was weird.

[ekristen]

Wouldn't it be better to provide a list of ports to docker-gen that are suppose to be exposed and then have docker-gen look up the exposed host port and then use that instead of trying to use all ports all the time?

[jwilder]

Yeah. That's sounds like a better idea.

[thaJeztah]

Basically, the proxy only needs to look for port 80 since it's used for web, unless a container contains two web-services on different ports?

[ekristen]

Not all apps use port 80 inside the container, most of the time in my experience they use different ports. I use everything from port 3012, to 8000, to 8080, etc ...

No assumption should be made, a config option is best IMO.

[thaJeztah]

Agreed that it may be configurable. (However, using port 8080 etc most of the time is a workaround for port 80 already being in use?). If each container uses a different exposed port for web-traffic, nginx-proxy will need another env-variable per container to specify which port it should pick.

[ekristen]

Agreed. I think that is the best approach.

[thaJeztah]

..but 80 should be the default if no port is specified 😄

[ekristen]

I think that is a reasonable assumption.

[jwilder]

How about this?

1. If a container only publishes 1 port, use that port.
2. If a container publishes multiple ports, use the mapped port that matches a VIRTUAL_PORT env variable for the container, if that env variable is defined in the container.
3. Otherwise, default to the published host port that maps to port 80 in the container.

The first item keeps the config simple so existing users don't need to go back and add a VIRTUAL_PORT env variable. If a container publishes port 8080 and it's the only published port, it'll be used. Second item provides a way to specify which port to proxy to if your container publishes multiple ports. Third item seems like a reasonable default if multiple ports are published by a VIRTUAL_PORT env is not set.

Also, just to clarify, I say published vs exposed because the containers need to be started w/ -p/-P to publish their exposed ports to the host.

[thaJeztah]

Looks good to me

Although;
Containers publishing a single port, e.g. a MySQL container publishing port. 3306 should not be taken into account, so slightly still in favour of detecting port 80 and only using other ports if they are explicitly marked via the VIRTUAL_PORT variable.

On second thought (sorry!) containers without VIRTUAL_HOST are ignored, so probably not a problem..LOL

[jwilder]

I've updated nginx-proxy to address the multiple port issue. Closing this since this issue is in docker-gen and originally about DOCKER_HOST.