New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrading to 19.7.0 means can't bind anymore #1480

Closed
hartym opened this Issue Mar 10, 2017 · 16 comments

Comments

6 participants
@hartym

hartym commented Mar 10, 2017

Upgraded to latest gunicorn and could not bind ip address anymore (simple wsgi app, ipv4 or ipv6 same story).

foo@xxx:~$ cd current
foo@xxx:~/current$ . env/bin/activate
foo@xxx:~/current$ gunicorn my.wsgi.package
[2017-03-10 18:50:46 +0000] [9242] [INFO] Starting gunicorn 19.7.0
[2017-03-10 18:50:46 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:47 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:48 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:49 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:50 +0000] [9242] [ERROR] Retrying in 1 second.
[2017-03-10 18:50:51 +0000] [9242] [ERROR] Can't connect to ('127.0.0.1', 8000)

Reverting to 19.6.0 fixes the problem.

System version : Linux xxx 3.2.0-4-amd64 #1 SMP Debian 3.2.68-1+deb7u1 x86_64 GNU/Linux

Let me know if I can provide more info, 19.6.0 is acceptable solution for me but I guess that's a regression you wanna at least understand ?

@lee-pai-long

This comment has been minimized.

Show comment
Hide comment
@lee-pai-long

lee-pai-long Mar 13, 2017

Adding to @hartym comment using --log-level debug I get the following:

[2017-03-13 15:49:39 +0100] [7175] [INFO] Starting gunicorn 19.7.0
[2017-03-13 15:49:39 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:39 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:40 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:40 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:41 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:41 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:42 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:42 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:43 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:43 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:44 +0100] [7175] [ERROR] Can't connect to ('0.0.0.0', 8000)

lee-pai-long commented Mar 13, 2017

Adding to @hartym comment using --log-level debug I get the following:

[2017-03-13 15:49:39 +0100] [7175] [INFO] Starting gunicorn 19.7.0
[2017-03-13 15:49:39 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:39 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:40 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:40 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:41 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:41 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:42 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:42 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:43 +0100] [7175] [DEBUG] connection to ('0.0.0.0', 8000) failed: [Errno 92] Protocol not available
[2017-03-13 15:49:43 +0100] [7175] [ERROR] Retrying in 1 second.
[2017-03-13 15:49:44 +0100] [7175] [ERROR] Can't connect to ('0.0.0.0', 8000)
@tilgovi

This comment has been minimized.

Show comment
Hide comment
@tilgovi

tilgovi Mar 13, 2017

Collaborator

It's very hard to imagine why this would be. Are you positive that nothing else was bound on the port at that time?

Collaborator

tilgovi commented Mar 13, 2017

It's very hard to imagine why this would be. Are you positive that nothing else was bound on the port at that time?

@tilgovi

This comment has been minimized.

Show comment
Hide comment
@tilgovi

tilgovi Mar 13, 2017

Collaborator

Hmm. It does seem you're binding to 0.0.0.0 now, when you used to be using 127.0.0.1. Does this make a difference?

Collaborator

tilgovi commented Mar 13, 2017

Hmm. It does seem you're binding to 0.0.0.0 now, when you used to be using 127.0.0.1. Does this make a difference?

@tilgovi

This comment has been minimized.

Show comment
Hide comment
@tilgovi

tilgovi Mar 13, 2017

Collaborator

Oh, I see. Both logs are from 19.7.0.

Collaborator

tilgovi commented Mar 13, 2017

Oh, I see. Both logs are from 19.7.0.

@lee-pai-long

This comment has been minimized.

Show comment
Hide comment
@lee-pai-long

lee-pai-long Mar 14, 2017

@tilgovi For my part I double checked, on the VM and try on another "empty" VM and the result is the same, in both case using 19.6 worked perfectly.

lee-pai-long commented Mar 14, 2017

@tilgovi For my part I double checked, on the VM and try on another "empty" VM and the result is the same, in both case using 19.6 worked perfectly.

@tilgovi

This comment has been minimized.

Show comment
Hide comment
@tilgovi

tilgovi Mar 14, 2017

Collaborator

I suspect this is due to SO_REUSEPORT issues.

We probably need to make a change like this one in thriftpy: eleme/thriftpy#264

Collaborator

tilgovi commented Mar 14, 2017

I suspect this is due to SO_REUSEPORT issues.

We probably need to make a change like this one in thriftpy: eleme/thriftpy#264

@tilgovi

This comment has been minimized.

Show comment
Hide comment
@tilgovi

tilgovi Mar 14, 2017

Collaborator

Any chance you can test this?

The change would be in gunicorn/sock.py in the set_options method of BaseSocket class.

Collaborator

tilgovi commented Mar 14, 2017

Any chance you can test this?

The change would be in gunicorn/sock.py in the set_options method of BaseSocket class.

tilgovi added a commit that referenced this issue Mar 14, 2017

@tilgovi

This comment has been minimized.

Show comment
Hide comment
@tilgovi

tilgovi Mar 14, 2017

Collaborator

I've pushed a branch for this.

Collaborator

tilgovi commented Mar 14, 2017

I've pushed a branch for this.

tilgovi added a commit that referenced this issue Mar 14, 2017

tilgovi added a commit that referenced this issue Mar 16, 2017

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Mar 16, 2017

Owner

@hartym @lee-pai-long can you test it ? If OK we will bump a 0.17.1 .

Owner

benoitc commented Mar 16, 2017

@hartym @lee-pai-long can you test it ? If OK we will bump a 0.17.1 .

@tilgovi

This comment has been minimized.

Show comment
Hide comment
@tilgovi

tilgovi Mar 17, 2017

Collaborator

@hartym @lee-pai-long branch for testing is over at #1483.

Collaborator

tilgovi commented Mar 17, 2017

@hartym @lee-pai-long branch for testing is over at #1483.

@benoitc benoitc added this to Acknowledged in Bugs Mar 20, 2017

@hartym

This comment has been minimized.

Show comment
Hide comment
@hartym

hartym Mar 20, 2017

Woops, sorry i did not see the thread before. I'm going to see if I can pull something quickly to test the suggested change.

hartym commented Mar 20, 2017

Woops, sorry i did not see the thread before. I'm going to see if I can pull something quickly to test the suggested change.

@hartym

This comment has been minimized.

Show comment
Hide comment
@hartym

hartym Mar 20, 2017

Works for me.

I bumped the gunicorn version number back to 19.7.0, deployed to check the breakage was still there, it was indeed broken, applied the patch, restarted the server and the port bind now works again.

Thank you, will bumb to 19.7.1 when it will be available.

hartym commented Mar 20, 2017

Works for me.

I bumped the gunicorn version number back to 19.7.0, deployed to check the breakage was still there, it was indeed broken, applied the patch, restarted the server and the port bind now works again.

Thank you, will bumb to 19.7.1 when it will be available.

@tilgovi tilgovi closed this in #1483 Mar 20, 2017

@tilgovi

This comment has been minimized.

Show comment
Hide comment
@tilgovi

tilgovi Mar 21, 2017

Collaborator

19.7.1 should be available now

Collaborator

tilgovi commented Mar 21, 2017

19.7.1 should be available now

@RazZziel

This comment has been minimized.

Show comment
Hide comment
@RazZziel

RazZziel Apr 25, 2017

19.7.1 is still broken for me:

root@server:/var/www/service# .venv/bin/gunicorn wsgi -c ../gunicorn.conf.py
[2017-04-25 08:56:45 +0000] [1234] [INFO] Starting gunicorn 19.7.1
Traceback (most recent call last):
  File "/var/www/service/.venv/lib/python3.5/site-packages/gunicorn/sock.py", line 44, in set_options
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
OSError: [Errno 92] Protocol not available

RazZziel commented Apr 25, 2017

19.7.1 is still broken for me:

root@server:/var/www/service# .venv/bin/gunicorn wsgi -c ../gunicorn.conf.py
[2017-04-25 08:56:45 +0000] [1234] [INFO] Starting gunicorn 19.7.1
Traceback (most recent call last):
  File "/var/www/service/.venv/lib/python3.5/site-packages/gunicorn/sock.py", line 44, in set_options
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
OSError: [Errno 92] Protocol not available
@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Apr 25, 2017

Owner
Owner

benoitc commented Apr 25, 2017

@petermorlion

This comment has been minimized.

Show comment
Hide comment
@petermorlion

petermorlion May 12, 2017

I can verify the master works for me (while 19.7.1 doesn't).

petermorlion commented May 12, 2017

I can verify the master works for me (while 19.7.1 doesn't).

fofanov pushed a commit to fofanov/gunicorn that referenced this issue Mar 16, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment