New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gunicorn w/ Django on nginx + unix socket: REMOTE Adress is "b''" (a string containing the three characters b'') #1755

Closed
anx-ckreuzberger opened this Issue Apr 24, 2018 · 7 comments

Comments

Projects
None yet
2 participants
@anx-ckreuzberger
Contributor

anx-ckreuzberger commented Apr 24, 2018

I just had a huge "wtf moment" when I was setting up a Django Middleware to parse the HTTP_X_FORWARDED_FOR header. For security reasons (more a precaution) my middleware would check the content of the REMOTE_ADDR header, to verify that the request is actually coming from a trusted proxy (e.g., 127.0.0.1).

Now, as already discussed in issue #797 gunicorn does not provide the REMOTE_ADDR anymore when used with an unix socket, and instead an empty string is being sent (see comment by @benoitc on Oct 9, 2014: "To summarise the context, the remote address is right now empty by default when using the unix socket. ").

So I thought I'll just make this little thing in a Django Middleware:

if request.META['REMOTE_ADDR'] == "": # empty string
    request.META['REMOTE_ADDR'] = request.META['HTTP_X_FORWARDED_FOR'].split(",")[0].strip()

Oh well... It seems that I was wrong.

Would you like to know what request.META['REMOTE_ADDR'] contains? It's actually the three characters b'' (no, not the byte representation b'', but actually the three characters!).

I believe this is not the intended behaviour, though I'm not involved enough with the whole gunicorn/nginx unix socket setup to point out where the error might be.

Here are some Details of the software that is being used:

  • Debian
  • Python 3.5.3
  • gunicorn 19.7.1
  • nginx 1.10.3
  • Django 1.11.12

Gunicorn is started via systemd with the following command:
gunicorn --workers 3 --bind unix:/run/gunicorn/socket application.wsgi

Nginx is configured as follows:

upstream web {
        ip_hash;
        server unix:/run/gunicorn/socket fail_timeout=0;
}

        location @proxy {
                # proxy stuff
                proxy_read_timeout 600;
                proxy_connect_timeout 600;
                proxy_pass_request_headers on;
                proxy_set_header        X-Real-IP $remote_addr;
                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header        Host $http_host;
                proxy_pass              http://web;
                proxy_redirect          off;
                # tell the upstream that we are using https (or whatever scheme)
                proxy_set_header        X-Forwarded-Proto $scheme;
        }

@anx-ckreuzberger anx-ckreuzberger changed the title from Gunicorn Gunicorn with Django on nginx + unix socket, REMOTE Adress is "b''" (a string containing the three characters b'') to Gunicorn w/ Django on nginx + unix socket: REMOTE Adress is "b''" (a string containing the three characters b'') Apr 24, 2018

@anx-ckreuzberger

This comment has been minimized.

Contributor

anx-ckreuzberger commented Apr 24, 2018

edit: I've added several details into the ticket.

@tilgovi

This comment has been minimized.

Collaborator

tilgovi commented Apr 28, 2018

I've tried this locally with Python 3.6 and environ['REMOTE_ADDR'] is empty for me. I have not tested with Django. This could be a bug with Django. If you can provide a minimal test I can take a look.

@anx-ckreuzberger

This comment has been minimized.

Contributor

anx-ckreuzberger commented Apr 30, 2018

I'll try to provide a docker setup where this bug happens, stay tuned.

@anx-ckreuzberger

This comment has been minimized.

Contributor

anx-ckreuzberger commented May 2, 2018

I've created an example repo:
https://github.com/anx-ckreuzberger/gunicorn-nginx-socket-docker

Feel free to check it out, it should be easy to reproduce.

@anx-ckreuzberger

This comment has been minimized.

Contributor

anx-ckreuzberger commented May 2, 2018

This issue seems to be appearing with Python 3.5.

Python 3.6.0 also has this issue.
Python 3.6.5 does not have this issue. I'm trying to see at which version of Python 3.6.* this issue was introduced.

@anx-ckreuzberger

This comment has been minimized.

Contributor

anx-ckreuzberger commented May 2, 2018

It seems like this issue is related to the python version, rather than gunicorn.

Affected Versions:
Python 3.5.2, 3.5.3
Python 3.6.0, 3.6.1

Non-affected versions:
Python 3.5.4+
Python 3.6.2+

I'm just making an educated guess here, but I believe that python issue 30205 is the cause (patch has been included with Python 3.5.4 and Python 3.6.2).

I'm not 100% sure if you can fix this in gunicorn. However, the problem is appearing on a debian 9.4, so this issue is probably affecting quite a bunch of users.

anx-ckreuzberger added a commit to anx-ckreuzberger/gunicorn that referenced this issue May 2, 2018

tilgovi added a commit that referenced this issue May 2, 2018

Merge pull request #1769 from anx-ckreuzberger/master
Fix for #1755: Decode remote addr if it is a byte
@tilgovi

This comment has been minimized.

Collaborator

tilgovi commented May 2, 2018

Closed by #1769.

@tilgovi tilgovi closed this May 2, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment