nginx.conf does not get updated with overlay network IPs

[schmunk42]

I created an overlay-network for my swarm and can ping and wget a hello-world webserver in the swarm from the reverseproxy container.

This is my /etc/hosts

10.0.1.7        mynetwork_hello_1

And this the relevant part of nginx.conf

upstream ~my.test.com {
                        # mynetwork_hello_1
                        server :80;
}

As you can see the IP is missing.

Looks like somewhere around here the IP does not get written correctly.

Any idea how to fix this or is this an issue with docker-gen?

Tested on: docker 1.9.1, docker-compose 1.5.1

[schmunk42]

Related issue: #97

[thaJeztah]

This could be related to the new networking in docker, where a container can be attached to multiple networks. Only the default "bridge" networking propagates the old fields for IP-address of the container, but custom networks will not, and the IP address is returned in a nested list, see nginx-proxy/docker-gen#129

[schmunk42]

@thaJeztah Thank you for the feedback. I am not a go-programmer (yet), but I looked a bit through the code and played around with docker-gen.

So it looks to me, that we first need support of the new networking feature from fsouza/go-dockerclient. Not until then we could use the VXLan IPs in the nginx template. Right?

I thought about an additional ENV variable like VIRTUAL_NETWORK to define which IP should be used, any thoughts about this?

[md5]

@schmunk42 See fsouza/go-dockerclient#407

[schmunk42]

@thaJeztah @jwilder @md5 Some updates from my side.

1. I updated the client and added a very basic struct for the networks. This is working as intended so far. These are my very first steps in Go, so don't be too harsh ;) nginx-proxy/docker-gen@master...schmunk42:feature/docker-network
2. With this template I am able to insert the correct IPs, but with some glitches ... I need to specify VIRTUAL_NETWORK and VIRTUAL_PORT at the moment. I think I could get rid of the port definition, but I think we need something which defines the network to use.
3. I wasn't really able to put all the pieces together and use the current nginx template with my patched docker-gen, that was mainly because of the different structures of Network and Address.

My plan would be to connect the reverseproxy to all docker-compose networks of my applications.

So .. would love to hear some thoughts from your side about this.

[md5]

@schmunk42 That looks like a great start. Glad to see that the necessary changes have landed in fsouza/go-dockerclient. I think if you add the IPv6-related fields to the struct, your changes to docker-gen will be in good shape for a PR.

As for the changes to the nginx.tmpl in this repo, I haven't looked closely enough to see what the right approach is to making it work, but I imagine it should be doable without any significant changes to docker-gen. Off the top of my head, I think that Address will no longer be used and that some more complicated logic about which Network to pick will be needed. It may also allow some consolidation around how Swarm is being handled, though I haven't looked into how the Swarm networking information is exposed in the new Docker API.

[schmunk42]

I played around with it a little bit more today...

Here's another approach without any changes to docker-gen or nginx-proxy, updated nginx.tmpl

{{ define "upstream" }}
    {{ if .Address }}
        {{/* If we got the containers from swarm and this container's port is published to host, use host IP:PORT */}}
        {{ if and .Container.Node.ID .Address.HostPort }}
            # {{ .Container.Node.Name }}/{{ .Container.Name }}
            server {{ .Container.Node.Address.IP }}:{{ .Address.HostPort }};
        {{/* If there is no swarm node or the port is not published on host, use container's IP:PORT */}}
        {{ else }}
            {{ if .Container.Env.VIRTUAL_NETWORK }}
                # {{ .Container.Env.VIRTUAL_NETWORK }} - {{ .Container.Name }}
                server {{ .Container.Name }}:{{ .Address.Port }};
            {{ else }}
            # {{ .Container.Name }}
            server {{ .Address.IP }}:{{ .Address.Port }};
        {{ end }}
        {{ end }}
    {{ else }}
        # {{ .Container.Name }}
        server {{ .Container.IP }} down;
    {{ end }}
{{ end }}

This is the changed section # {{ .Container.Env.VIRTUAL_NETWORK }} - {{ .Container.Name }}.
It still uses VIRTUAL_NETWORK, but with true/false values, which could also be read as USE_CONTAINER_NAME_AS_ADDRESS :)

The glitch at the moment is, that we have to connect the proxy to the app-network before we start it, otherwise we get an error (host not found).

@md5 But in general that goes in the same direction as you mentioned - swarm handling can be consolidated.

[schmunk42]

And I think we need to wait for the fix here docker-archive/classicswarm#1402 first

[modius]

@schmunk42 would your earlier suggestions nginx-proxy/docker-gen@fdf2922 fix jwilder/nginx-proxy for local networking with docker network? ie #305

[schmunk42]

I haven't continued working on the PR, because I found the above solution without changing the image.

See above, for the first lines from our modified nginx.tmpl.

If a container is started with VIRTUAL_NETWORK=1 the reverse proxy uses the container name to resolve the upstream IP.

This works fine in conjunction with docker-compose and overlay networking.

Any thoughts about this? Do we need the these changes in the PR at all?

[update] removed duplicated code

You could give the above template a try with local docker networking, I at least can't remember having problems with it.

[fbender]

This should work automatically without an extra env var to be specified. Isn't the type of networking available via the container info? Though it probably requires knowing if the proxy container is on the same network as the app container …

[md5]

@fbender I don't think the preferred networking type is specified in the container info. The container will have NetworkSettings.Networks set in the underlying response from the Docker API and that will include all network addresses, including the bridge network as well as any others.

I haven't fully thought through this, but I think that instead of a VIRTUAL_NETWORK setting on each container, the jwilder/nginx-proxy container will instead need to be launched with a setting that indicates its preferred network(s) for connecting to all the containers it can see. This will imply that if some containers aren't connected to that network (or networks) they will not be included in the config regardless of whether they have a VIRTUAL_HOST environment variable

[josteinbf]

I think that instead of a VIRTUAL_NETWORK setting on each container, the jwilder/nginx-proxy container will instead need to be launched with a setting that indicates its preferred network(s) for connecting to all the containers it can see. This will imply that if some containers aren't connected to that network (or networks) they will not be included in the config regardless of whether they have a VIRTUAL_HOST environment variable

This sounds like a good approach to me. It would allow other networks that are used by the proxied container to remain inaccessible to the nginx-proxy container, which I think is useful.

[Ant59]

Same issue with Docker 1.10 and Docker Compose 1.6. Missing IP address in upstream block. Using default network. Compose config version 2.

[gabriel403]

Yeh I just run into this too, same as @Ant59, it seems both the proxy and any webapp need starting in the singular way, with docker run rather than with docker-compose

[Ant59]

Not using Compose isn't an option for me.

For some reason, docker-gen is no longer able to determine the IP address of the container when started using Compose.

[wader]

@Ant59 the problem is that nginx-proxy does not yet support the new docker networking feature that docker compose version 2 uses. Try to use the previous compose config format instead.

[Ant59]

Perfect. Older Compose config works. Thanks.

[meyer]

@wader thank yooooou. I wasted a few hours on this silly issue. Really needs to be mention in the README.

[wader]

@meyer no probleeeem 😄

[meetmatt]

Didn't know that this is a confirmed issue. Should've searched better!
As a work around I just check for the Address.IP in returned Container structure, which allows to support both version 1 and 2 of docker-compose files:

{{ define "upstream" }}
    ...
    {{ if .Address.IP }}
        server {{ .Address.IP }}:{{ .Address.Port }};
    {{ else }}
        server {{ .Container.Name }}:{{ .Address.Port }};
    {{ end }}
    ...
{{ end }}

[almereyda]

I am now also using the following declaration to get working results:

{{ define "upstream" }}
        {{ if .Address }}
                {{/* If we got the containers from swarm and this container's port is published to host, use host IP:P$
                {{ if and .Container.Node.ID .Address.HostPort }}
                        # {{ .Container.Node.Name }}/{{ .Container.Name }}                                             
                        server {{ .Container.Node.Address.IP }}:{{ .Address.HostPort }};
                {{/* If there is no swarm node or the port is not published on host, use container's IP:PORT */}}
                {{ else if .Address.IP }}
                        # {{ .Container.Name }}
                        server {{ .Address.IP }}:{{ .Address.Port }};
                {{ else }}
                        # {{ .Container.Name }}
                        server {{ .Container.Name }}:{{ .Address.Port }};
                {{ end }}
        {{ else }}
                # {{ .Container.Name }}
                server {{ .Container.IP }} down;
        {{ end }}
{{ end }}

[kop]

So is there any temporary solution to make nginx-proxy work with compose version 2 configs?

[llitfkitfk]

For local dev

...
    volumes:
      - ./conf.d:/etc/nginx/conf.d
      - /var/run/docker.sock:/tmp/docker.sock:ro
...

just change default.conf to

...
upstream subdomain.domain.com {
            # dockersummerweb_app_1
            server your-proxy-container-name:8080;
}
...

[nikashitsa]

Created PR #425 from @duck1123 code. Please, check.

[orlade]

If #425 is merged, is this issue resolved?

[mickaelperrin]

Hi,

Just ran into the same issue as the OP with the latest version of all containers, but only when using a decoupled version of docker-gen and nginx. I provide a test configuration at https://github.com/mickaelperrin/docker-nginx-proxy-icc-bug/tree/bug304.

Hope it helps.

[jice-lavocat]

Hi @mickaelperrin , thanks for the example repo. I learned a new way to organize my services/networks.

[artworkad]

👍 same problem here with docker 1.10 and latest nginx-proxy

[Nepoxx]

What's the status on this issue?

#425 was closed and #337 was merged

[suizman]

I've tested the changes from #425 and it works like a charm xD nginx-proxy

[artworkad]

@Nepoxx looks like this has not beed published on docker hub yet

[boggyhole]

Any news on this? Or do you plan to integrate #425 at all?

[jwilder]

Fixed via #425 #337

[Nepoxx]

The README should be updated accordingly (it refers to this issue).

[murbanowicz]

@mickaelperrin did you solve it? I am using nginx,nginx-gen and companion and I have no host...

[arefaslani]

my compose file looks like this:

version: "3.1"

services:
  app:
    image: registry.gitlab.com/arefaslani/influencer-marketing-platform
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
    volumes:
      - .:/usr/src/app
    ports:
      - 3000:3000
    environment:
      - VIRTUAL_HOST=api.myapp.dev

  nginx:
    image: jwilder/nginx-proxy:alpine
    ports:
      - 80:80
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
    depends_on:
      - app

when I deploy the stack using docker stack deploy command, my upstream is look like this:

upstream api.influencer2.dev {
		## Can be connected with "chanka_default" network
		# my_app.1.xqykftg8k3ba74e3ovchlqhql
			server 10.0.2.5 down;
}

According to
https://github.com/jwilder/nginx-proxy/blob/master/nginx.tmpl#L5-L8
it's possible to use overlay networks for nginx-proxy image. But why my server cant get a correct IP and Port?