New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document how to get real remote client ip for service running in container #15086

Open
Silex opened this Issue Jul 28, 2015 · 73 comments

Comments

Projects
None yet
@Silex
Copy link

Silex commented Jul 28, 2015

Hello,

I have an hard time figuring out how to get the real remote ip of a client connecting to a (web) service inside a container which port is exposed to a port on the host.

So far the solution seems to be either one of:

  • use --net=host
  • disable the userland proxy
  • configure nginx on the host to forward the real ip address?

Here are related issues:

#7540
jwilder/nginx-proxy#130
jwilder/nginx-proxy#133

None of the current solutions seems to be "right" and they all seem temporary. Can you clarify/explain how things should be, even if that means that the current docker 1.7.1 is bugged in that regard and that we have to use hack X or Y until things are fixed in 1.8.0.

@GordonTheTurtle

This comment has been minimized.

Copy link

GordonTheTurtle commented Jul 28, 2015

Hi!

Please read this important information about creating issues.

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.

If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.

This is an automated, informational response.

Thank you.

For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues


BUG REPORT INFORMATION

Use the commands below to provide key information from your environment:

docker version:
docker info:
uname -a:

Provide additional environment details (AWS, VirtualBox, physical, etc.):

List the steps to reproduce the issue:
1.
2.
3.

Describe the results you received:

Describe the results you expected:

Provide additional info you think is important:

----------END REPORT ---------

#ENEEDMOREINFO

@Silex

This comment has been minimized.

Copy link
Author

Silex commented Jul 28, 2015

Description of problem:

I have an hard time figuring out how to get the real remote ip of a client connecting to a (web) service inside a container which port is exposed to a port on the host.

So far the solution seems to be either one of:

  • use --net=host
  • disable the userland proxy
  • configure nginx on the host to forward the real ip address?

Here are related issues:

#7540
jwilder/nginx-proxy#130
jwilder/nginx-proxy#133

None of the current solutions seems to be "right" and they all seem temporary. Can you clarify/explain how things should be, even if that means that the current docker 1.7.1 is bugged in that regard and that we have to use hack X or Y until things are fixed in 1.8.0.

docker version:

Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

docker info:

Containers: 1
Images: 508
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 510
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-58-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 4
Total Memory: 15.64 GiB
Name: philippe-desktop
ID: QLBB:VYAR:TKYL:NZQN:MGON:SNU4:7BLV:XFEJ:AIGR:LZY6:LCIF:GHBU

uname -a:

Linux philippe-desktop 3.13.0-58-generic #97-Ubuntu SMP Wed Jul 8 02:56:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Environment details (AWS, VirtualBox, physical, etc.):

Physical

How reproducible:

Easily.

Steps to Reproduce:

  1. FIXME

Actual Results:

We get docker0 bridge ip.

Expected Results:

We get the remote client ip.

Additional info:

Creating issues in this format is a PITA and goes against github flow.

@phemmer

This comment has been minimized.

Copy link
Contributor

phemmer commented Jul 28, 2015

disable the userland proxy

This is the correct answer, and is soon to become the default action (see #14856)

Edit: If this solution seems unsatisfactory, can you explain your thoughts on the matter?

@Silex

This comment has been minimized.

Copy link
Author

Silex commented Jul 29, 2015

Alright, thanks!

@Silex Silex closed this Jul 29, 2015

@oleynikd

This comment has been minimized.

Copy link

oleynikd commented Nov 4, 2015

@phemmer

Edit: If this solution seems unsatisfactory, can you explain your thoughts on the matter?

the problem is that if you use --iptables=false you can not disable userland proxy. Please take a look here
Will appreciate any suggestions. Thanks.

@colllin

This comment has been minimized.

Copy link

colllin commented Aug 10, 2016

Ok, what's the latest recommended solution if --userland-proxy=false is unworkable (as described in #14856)?

  • use --net=host
  • disable the userland proxy
  • configure nginx on the host to forward the real ip address?
@colllin

This comment has been minimized.

Copy link

colllin commented Aug 10, 2016

Also, can we re-open this issue if #14856 was reverted? Or has it been documented somewhere?

@dimaqq

This comment has been minimized.

Copy link

dimaqq commented Sep 7, 2016

+1 for reopen, as "no userland proxy" is not viable at the moment.

@ciarans

This comment has been minimized.

Copy link

ciarans commented Sep 23, 2016

+1 to reopen from me. Looking for the real IP address to set SSL via a reverse proxy

@hazam

This comment has been minimized.

Copy link

hazam commented Oct 1, 2016

+1 this issue is really scary.

@hardware

This comment has been minimized.

Copy link

hardware commented Oct 6, 2016

+1 we need more documentation on this subject and workable solution.

@j-schumann

This comment has been minimized.

Copy link

j-schumann commented Oct 10, 2016

+1
receiving the wrong source IP inside the container results in problems with SPF validatio, spam scoring, access statistics etc.

@deanml

This comment has been minimized.

Copy link

deanml commented Oct 10, 2016

+1
Need a way to handle geolocated requests.

@Coyzz

This comment has been minimized.

Copy link

Coyzz commented Nov 2, 2016

+1 need to know which IP source connect to my app, and fail2ban if needed

@adimit

This comment has been minimized.

Copy link

adimit commented Nov 4, 2016

Since we already have so many +1s, let's have another. And a minimal setup I used to test this & exhibit the problem: http://serverfault.com/questions/813298/

@djflux

This comment has been minimized.

Copy link

djflux commented Nov 11, 2016

Another +1 here. Need to have true client IP address for logging and security purposes. Re-open with some documentation please :)

@romanoaugusto88

This comment has been minimized.

Copy link

romanoaugusto88 commented Nov 11, 2016

+1 here. Really need to get true client IP address.

1 similar comment
@rafaelsq

This comment has been minimized.

Copy link

rafaelsq commented Nov 11, 2016

+1 here. Really need to get true client IP address.

@cicerocomp

This comment has been minimized.

Copy link

cicerocomp commented Nov 11, 2016

+1 here.

$ docker version
Client:
 Version:      1.12.3
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   6b644ec
 Built:        Wed Oct 26 21:44:32 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.3
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   6b644ec
 Built:        Wed Oct 26 21:44:32 2016
 OS/Arch:      linux/amd64
$ uname -r
3.13.0-100-generic
@DominicBoettger

This comment has been minimized.

Copy link

DominicBoettger commented Nov 17, 2016

Failed using the geoip module....

@BenediktS

This comment has been minimized.

Copy link

BenediktS commented Nov 23, 2016

+1 The NAT should only be used on outbound traffic and not on inbound traffic

@aledelgo

This comment has been minimized.

Copy link

aledelgo commented Dec 6, 2016

+1 here

@ishan-marikar

This comment has been minimized.

Copy link

ishan-marikar commented Dec 9, 2016

+1 here. Need the real IP address for access statistics, logging and security purposes.

@ciarans

This comment has been minimized.

Copy link

ciarans commented Dec 11, 2016

+1 This is a real problem. We need the real IP for our firewall

@phpid

This comment has been minimized.

Copy link

phpid commented Dec 15, 2016

+1. Please reopen the issue.

@k2xl

This comment has been minimized.

Copy link

k2xl commented Oct 3, 2017

Found the solution in my wsgi app
request.environ.get('HTTP_X_REAL_IP', request.remote_addr)

@mukuld

This comment has been minimized.

Copy link

mukuld commented Oct 3, 2017

@caoli5288

This comment has been minimized.

Copy link

caoli5288 commented Oct 15, 2017

@k2xl not all app based on http :(

@gabrielmocan

This comment has been minimized.

Copy link

gabrielmocan commented Oct 16, 2017

I really need this to get working to separate different flow streams from routers (they're identified by their source IPs...)

@mikehaertl

This comment has been minimized.

Copy link

mikehaertl commented Oct 18, 2017

Here's a solution that works for me. It's based on classic port forwarding (which @michallohnisky already pointed out above).

Principles

  • The docker iptables option is disabled as we manage iptables rules ourselves
  • We use a user-defined docker network with a dedicated private IP address range
  • Each container gets a static IP address in that network
  • The container does not expose (--expose) or publish (--publish or -p) any ports

Setup on Ubuntu 16.04

Disable docker's iptables

sudo systemctl stop docker

Create a file /etc/systemd/system/docker.service.d/noiptables.conf with this content:

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// --iptables=false
sudo systemctl daemon-reload
sudo systemctl start docker

Create user-defined docker network

docker network create -d bridge --subnet 192.168.0.0/24 --gateway 192.168.0.1 dockernet

Note: The --gateway option makes the host reachable under 192.168.0.1 from the containers.

Add custom rules to ufw (iptables)

We use ufw to manage our firewall. But it would also work with any other iptables based firewall.

First change the default forward policy to ACCEPT in /etc/default/ufw:

DEFAULT_FORWARD_POLICY="ACCEPT"

Then we add custom rules to the iptables NAT table. We use masquerading to let the containers connect to the internet and DNAT to forward ports from the host to specific containers. This is achieved by adding the following to the top of /etc/ufw/before.rules:

# nat Table rules
*nat

# Set default policies
:POSTROUTING ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]

# Masquerade traffic from dockernet
-A POSTROUTING -s 192.168.0.0/24 -o eth0 -j MASQUERADE
# ... and from default docker network
-A POSTROUTING -s 172.17.0.0/16 -o eth0 -j MASQUERADE

# Port forwarding for specific services to docker container
-A PREROUTING -p tcp -i eth0 --dport 25 -j DNAT --to-destination 192.168.0.2:25
-A PREROUTING -p tcp -i eth0 --dport 80 -j DNAT --to-destination 192.168.0.2:80
# ... more DNAT rules here ...

# don't delete the 'COMMIT' line or these nat table rules won't be processed
COMMIT

Note Our public facing interface on the host is eth0. Your mileage may vary.

Now we can open the specified ports on the host and start the firewall:

sudo ufw allow 25
sudo ufw allow 80
sudo ufw enable

Configure container to use this network

Here's an example docker-compose.yml:

version: '3'
services:
    app:
        build: ./
        networks:
            dockernet:
                ipv4_address: 192.168.0.2
networks:
    dockernet:
        external: true
@goetas

This comment has been minimized.

Copy link

goetas commented Jan 23, 2018

So far the only solution that worked for me was --net=host.
(that makes you not being able to use most of the other docker features....).
Any other strategy (without going crazy with iptables or other network "hacks"...) failed miserably.

@Dean-Christian-Armada

This comment has been minimized.

Copy link

Dean-Christian-Armada commented Jan 24, 2018

I agree with @goetas . What I did for this was yes put the port mode to host

ports:
      - target: 8001
        published: 8001
        mode: host

But this will disable the ingress networking where you can access the port on any node of the Swarm. Basically, you will only access this/these ports if you type the actual address of that server on the browser's address bar. But the benefit of using mode host is it has lesser latency in the networking point of view. This is perfect for Nginx gateways.

@hikmat30ce

This comment has been minimized.

Copy link

hikmat30ce commented Feb 5, 2018

in php yii2 framwork below code workded for me:
<?= isset($_SERVER['HTTP_CF_CONNECTING_IP']) ?$_SERVER['HTTP_CF_CONNECTING_IP'] : null;?>

My setup is as below:

  • I am using cloudflare dns
  • I am using apache as reverse proxy which forward request to my containers
  • I am using yii2 as php framwork (code is inside docker)
@Xplouder

This comment has been minimized.

Copy link

Xplouder commented Feb 10, 2018

+1

@basicdays

This comment has been minimized.

Copy link

basicdays commented Feb 14, 2018

So I actually did go and disable the userland proxy in the Docker daemon. This didn't actually seem to fix getting the remote IP when navigating via localhost. In addition, since Chrome wants to resolve localhost to IPv6, requests won't actually get to the container without going through all the problems of getting IPv6 working within Docker. Firefox and CURL can be used to force an IPv4 localhost request though.

Setting the container to use host networking does work, but is suboptimal. I would be absolutely ok with disabling userland proxy if it fixes anything.

My current setup:

  • Kubuntu 17.10
  • Linux kernel 4.13.0-32
  • Docker 17.12.0-ce
@Steiniche

This comment has been minimized.

Copy link

Steiniche commented Mar 5, 2018

I have tried disabling the userland proxy which renders my setup unusable as only the server can see access the port mapped containers.
All calls made externally to the server failed.
I have also tried to disable IPv6 as it was suggested in hardware/mailserver#43 by doing the following:

/etc/sysctl.conf

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

However, it seemed to have no effect on my system.

@basicdays

This comment has been minimized.

Copy link

basicdays commented Mar 6, 2018

@Steiniche When I ran into that, I found that if I added another entry in my /etc/hosts file with 127.0.0.1 localhost2, and I navigated to localhost2:<exposed_port>, I could actually get a request to the Docker instance. No clue as to why this is, but I guess the localhost hostname is super special. :/

@hfreville

This comment has been minimized.

Copy link

hfreville commented Mar 6, 2018

Hello, got similar problem running rsyslog inside container. Workaround for me is to install conntrack tools on host and flush table after starting container (with conntrack -F).

lexa-uw added a commit to lexa-uw/ansible-role-nginx-in-docker that referenced this issue Apr 13, 2018

@chris13524

This comment has been minimized.

Copy link

chris13524 commented May 15, 2018

In our case, we have connections coming in via IPv6, but Docker is replacing those with the IPv4 address of the docker0 gateway.

@yi-huan

This comment has been minimized.

Copy link

yi-huan commented Jun 19, 2018

This should be the same question, right?

Correctly express "any address" to iptables.

@treydibler

This comment has been minimized.

Copy link

treydibler commented Feb 8, 2019

3 years and counting.... +1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment