New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close File Handles on HUP Signal #627

Closed
avaitla opened this Issue Oct 10, 2013 · 3 comments

Comments

Projects
None yet
2 participants
@avaitla
Contributor

avaitla commented Oct 10, 2013

This is related to #624 (comment). Please let me know if any additional details are required, or if I can assist in some way.

I first proposed closing explicitly, but probably not the best way to do it. Another option is to hold on to the same logger object, and just update the cfg, and call reopen on it? This is probably more complicated though...

The issue was first noticed on CentOS release 6.4 (Final) x86_64 machine, it can still be reproduced on that os, it has sporadically failed on OS X 10.8:

$ gunicorn -c conf.py myapp:app &
[1] 18042
$ lsof -p `cat gunicorn.pid` | grep accesslog_test.txt
gunicorn 18042 avaitla    5w   REG    202,2        0     3305 /home/avaitla/gunicorn/accesslog_test.txt
$ lsof -p `cat gunicorn.pid` | grep errorlog_test.txt
gunicorn 18042 avaitla    1u   REG    202,2     2522     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    2u   REG    202,2     2522     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    3u   REG    202,2     2522     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    4w   REG    202,2     2522     3304 /home/avaitla/gunicorn/errorlog_test.txt

$ kill -HUP `cat gunicorn.pid`
$ lsof -p `cat gunicorn.pid` | grep accesslog_test.txt
gunicorn 18042 avaitla    5w   REG    202,2        0     3305 /home/avaitla/gunicorn/accesslog_test.txt
gunicorn 18042 avaitla   15w   REG    202,2        0     3305 /home/avaitla/gunicorn/accesslog_test.txt
$ lsof -p `cat gunicorn.pid` | grep errorlog_test.txt
gunicorn 18042 avaitla    1u   REG    202,2     3145     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    2u   REG    202,2     3145     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    3u   REG    202,2     3145     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    4w   REG    202,2     3145     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla   13u   REG    202,2     3145     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla   14w   REG    202,2     3145     3304 /home/avaitla/gunicorn/errorlog_test.txt

$ kill -HUP `cat gunicorn.pid`
$ lsof -p `cat gunicorn.pid` | grep accesslog_test.txt
gunicorn 18042 avaitla    5w   REG    202,2        0     3305 /home/avaitla/gunicorn/accesslog_test.txt
gunicorn 18042 avaitla   11u   REG    202,2        0     3305 /home/avaitla/gunicorn/accesslog_test.txt
gunicorn 18042 avaitla   15w   REG    202,2        0     3305 /home/avaitla/gunicorn/accesslog_test.txt
$ lsof -p `cat gunicorn.pid` | grep errorlog_test.txt
gunicorn 18042 avaitla    1u   REG    202,2     3768     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    2u   REG    202,2     3768     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    3u   REG    202,2     3768     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    4w   REG    202,2     3768     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla    9u   REG    202,2     3768     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla   10u   REG    202,2     3768     3304 /home/avaitla/gunicorn/errorlog_test.txt
gunicorn 18042 avaitla   14w   REG    202,2     3768     3304 /home/avaitla/gunicorn/errorlog_test.txt
@avaitla

This comment has been minimized.

Show comment
Hide comment
@avaitla

avaitla Oct 13, 2013

Contributor

The reason I think this doesn't have to do with close_on_exec, is because the Master process doesn't get recreated when HUP signal is received, only the workers. So I am thinking that close_on_exec doesn't apply in this case, since we are dealing with the masters handles and not the workers.

Contributor

avaitla commented Oct 13, 2013

The reason I think this doesn't have to do with close_on_exec, is because the Master process doesn't get recreated when HUP signal is received, only the workers. So I am thinking that close_on_exec doesn't apply in this case, since we are dealing with the masters handles and not the workers.

benoitc added a commit that referenced this issue Nov 19, 2013

makes sure that file handles are correctly reopened on HUP
This change makes sure we don't recreate the Logger class here so we
don't reinitialise the log handlers.

fix #627
@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Nov 19, 2013

Owner

@avaitla the branch fix/627 should fix it. On my machine I have now:

$ lsof -p 63703 |grep error.log 
python2.7 63703 benoitc    1u   REG                1,2       389 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    2u   REG                1,2       389 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    3u   REG                1,2       389 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    4w   REG                1,2       389 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
$ kill -HUP 63703
$ lsof -p 63703 |grep error.log 
python2.7 63703 benoitc    1u   REG                1,2       883 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    2u   REG                1,2       883 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    3u   REG                1,2       883 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    4w   REG                1,2       883 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
Owner

benoitc commented Nov 19, 2013

@avaitla the branch fix/627 should fix it. On my machine I have now:

$ lsof -p 63703 |grep error.log 
python2.7 63703 benoitc    1u   REG                1,2       389 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    2u   REG                1,2       389 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    3u   REG                1,2       389 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    4w   REG                1,2       389 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
$ kill -HUP 63703
$ lsof -p 63703 |grep error.log 
python2.7 63703 benoitc    1u   REG                1,2       883 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    2u   REG                1,2       883 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    3u   REG                1,2       883 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
python2.7 63703 benoitc    4w   REG                1,2       883 12552334 /Users/benoitc/work/gunicorn_env/src/gunicorn/examples/error.log
@avaitla

This comment has been minimized.

Show comment
Hide comment
@avaitla

avaitla Dec 2, 2013

Contributor

Thanks for taking a look. It works great on my end as well. Thanks!

Contributor

avaitla commented Dec 2, 2013

Thanks for taking a look. It works great on my end as well. Thanks!

@avaitla avaitla closed this Dec 2, 2013

benoitc added a commit that referenced this issue Sep 23, 2015

makes sure that file handles are correctly reopened on HUP
This change makes sure we don't recreate the Logger class here so we
don't reinitialise the log handlers.

fix #627

benoitc added a commit that referenced this issue Sep 23, 2015

makes sure that file handles are correctly reopened on HUP
This change makes sure we don't recreate the Logger class here so we
don't reinitialise the log handlers.

fix #627

fofanov pushed a commit to fofanov/gunicorn that referenced this issue Mar 16, 2018

makes sure that file handles are correctly reopened on HUP
This change makes sure we don't recreate the Logger class here so we
don't reinitialise the log handlers.

fix #627
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment