Requests show Docker IP adresses instead of real client IP

[bllngr]

Welcome!

• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you do?

Basically, running the example from https://doc.traefik.io/traefik/user-guides/docker-compose/basic-example/ results in (only) Docker IP addresses being reported, instead of the real client IPs.

What did you see instead?

$ ip route get 1.2.3.4 | awk '{print $7}'
192.168.178.60

$ curl -H "Host:whoami.localhost" localhost:80
Hostname: 4123d0073d4c
IP: 127.0.0.1
IP: 172.21.0.2
RemoteAddr: 172.21.0.3:60392
GET / HTTP/1.1
Host: whoami.localhost
User-Agent: curl/7.75.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.21.0.1
X-Forwarded-Host: whoami.localhost
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: 2b9b375b9ae0
X-Real-Ip: 172.21.0.1

The same IP is shown in the logs. It doesn't matter from which client the request comes, the IP is always the one from the Docker network.

This is problematic, since it completely prevents middlewares that rely on the client IP, such as IPWhiteList.

What version of Traefik are you using?

2.8.1

What is your environment & configuration?

# Unmodified basic example from https://doc.traefik.io/traefik/user-guides/docker-compose/basic-example/
version: "3.3"

services:

  traefik:
    image: "traefik:v2.8"
    container_name: "traefik"
    command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

  whoami:
    image: "traefik/whoami"
    container_name: "simple-service"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`whoami.localhost`)"
      - "traefik.http.routers.whoami.entrypoints=web"

Docker version: 20.10.3
docker-compose version 1.28.5

If applicable, please paste the log output in DEBUG level

traefik    | time="2022-07-23T14:42:24Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"User-Agent\":[\"curl/7.75.0\"],\"X-Forwarded-Host\":[\"whoami.localhost\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"126774ff0647\"],\"X-Real-Ip\":[\"172.21.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.localhost\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.1:46176\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik    | time="2022-07-23T14:42:24Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"User-Agent\":[\"curl/7.75.0\"],\"X-Forwarded-Host\":[\"whoami.localhost\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"126774ff0647\"],\"X-Real-Ip\":[\"172.21.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.localhost\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.1:46176\",\"RequestURI\":\"/\",\"TLS\":null}" ForwardURL="http://172.21.0.2:80"
traefik    | time="2022-07-23T14:42:24Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"User-Agent\":[\"curl/7.75.0\"],\"X-Forwarded-Host\":[\"whoami.localhost\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"126774ff0647\"],\"X-Real-Ip\":[\"172.21.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.localhost\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.1:46176\",\"RequestURI\":\"/\",\"TLS\":null}"

[bllngr]

See also #614, #7063, and #8139, where the same issue is reported (last comment).

If that is the expected behaviour, this should probably documented in the basic example and in the documentation for IPWhiteList, with suggestions how to configure a container to be able to use the IPWhiteList middleware.

[traefiker]

Hi! I'm Træfiker 🤖 the bot in charge of communication regulation.

Thanks for your interest in Traefik!

We dedicate the issue tracker to bug reports and feature requests only. My advanced AI has spotted that your issue might be a configuration problem or relates to something that doesn't look like a bug.

To confirm this, please join our Community Forum and reach out to us on the Traefik section.

In case I'm wrong (well, that would be embarrassing 😅), my developers will re-open the issue and fix me!

In the meantime, you can double check Traefik's documentation.

[bllngr]

Please don't do this, this is at least an oversight in the documentation. I'm only using the example docker-compose configuration, and both the whoami output and the IPWhiteList middleware are not working as documented.

I understand that this might be an issue with docker itself, but can at least the documentation be adjusted? @rtribotte maybe?

[rtribotte]

Hello @bllngr,

Thanks for your interest in Traefik!

Please don't do this, this is at least an oversight in the documentation. I'm only using the example docker-compose configuration, and both the whoami output and the IPWhiteList middleware are not working as documented.

This issue seems to be indeed a configuration issue and is not related to Traefik behavior, as it seems that the client IP here is the docker0 interface IP.

But to get some help on configuration please reach out on our community forum. For instance, you can take a look at this thread: https://community.traefik.io/t/whitelist-swarm-cant-get-real-source-ip/3897.

I understand that this might be an issue with docker itself, but can at least the documentation be adjusted? @rtribotte maybe?

Regarding the lack of documentation, as it is not related to Traefik itself and more to the operation of Traefik, we don't see an obvious way to address it in our documentation.
Thus, if you feel this should be documented and see a good place to do it, we gladly welcome contributions, so please feel free to open a pull request.