how to get user real ip when use docker service / dns loadbalance

[marech]

Have problem while getting user real IP - it always shows as container ip.

Im using docker 1.12 in swarm mode, 3 services with 2 replicas each.
2 gateway service replicas - whic dontains nginx and forward requests based on host
2 app1 service replicas - lets call it app1.example.com
2 app2 service replicas - app2.example.com

Thats how i created these services

docker service create --name gateway    --replicas=2 -p 80:80 -p 443:443    --network ingress my.registry.com:5000/general/gateway
docker service create --name app1       --replicas=2                        --network ingress my.registry.com:5000/general/app1
docker service create --name app2       --replicas=2                        --network ingress my.registry.com:5000/general/app2

Ingress network:
docker network create --driver overlay ingress

Gateway nginx config looks something like this:

server {
    listen       80;
    server_name app1.example.com;

    proxy_redirect off;
    proxy_set_header Host $host;

    location / {
        set $target app1;
        proxy_pass http://$target;
    }
}

server {
    listen       80;
    server_name app2.example.com;

    proxy_redirect off;
    proxy_set_header Host $host;

    location / {
        set $target app2;
        proxy_pass http://$target;
    }
}

So based on requested host i redirect to appropriate docker service.
Problem is - in nginx logs and in php application client IP shows as gateway container IPs e.g. 10.255.0.7
Any ideas how can I restore original client ip if i use docker loadbalancing?

Docker version 1.12.1, build 23cf638

Containers: 17
 Running: 5
 Paused: 0
 Stopped: 12
Images: 7
Server Version: 1.12.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 97
 Dirperm1 Supported: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host null overlay
Swarm: active
 NodeID: 24wvq2ae4pdair4qkvc61rfne
 Is Manager: true
 ClusterID: eaybzefous060vugnsv8o0q50
 Managers: 1
 Nodes: 1
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
 Node Address: 172.31.16.58
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 3.13.0-95-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 992.5 MiB
Name: ip-172-31-16-58
ID: Z7AI:KPZ4:ALEJ:APOL:6KY2:3QW4:6PXY:RQ26:BTB2:KPCA:Z5T7:B4YU
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

[justincormack]

This is a duplicate of #25526 I think.

[marech]

yep, its #25526 dup. Thanks!

[struanb]

We've now released v3.1.0 of https://github.com/newsnowlabs/docker-ingress-routing-daemon, which modifies docker's ingress mesh routing to expose true client IPs to service containers:

• implemented purely through routing and firewall rules; and so
• without the need for running any additional application layers like traefik or other reverse proxies; and so
• there's no need to reconfigure your existing application.

As far as I know, the docker-ingress-routing-daemon is the most lightweight way to access client IPs from within containers launched by docker services.

Summary of features:

• Support for replacing docker's masquerading with routing on incoming traffic either for all published services, or only for specified services on specified TCP or UDP ports
• Support for recent kernels (such as employed in Google Cloud images) that set rp_filter=1 (strict) inside service containers
• Automatic installation of kernel tweaks that improve IPVS performance in production (though this can be disabled)

Please check it out and raise any issues you find.