New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StopIteration Exceptions for Gevent worker #790

Closed
dsoprea opened this Issue Jun 14, 2014 · 15 comments

Comments

Projects
None yet
4 participants
@dsoprea

dsoprea commented Jun 14, 2014

I'm getting this, every time.

This is a test project that elicits the same result (it's a simple, unexceptional project using gevent). Traditionally, I don't use gevent workers. However, I did on a recent project, though I didn't see any errors until last night (which I can't explain).

Nginx:

    upstream gevent_gunicorn_test {
        server unix:/tmp/gevent.gunicorn.sock fail_timeout=0;
    }

    server {
            server_name gevent.local;
            keepalive_timeout 5;

            location /favicon.ico {
                return 404;
            }

            location / {
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_redirect off;

                proxy_pass http://gevent_gunicorn_test;
            }
    }

Config:

bind = 'unix:/tmp/gevent.gunicorn.sock'
worker_class = 'gevent'

Code:

import web

class HandleMe(object):
    def GET(self):
        return 'xyz'

URLS = (
    '/', HandleMe,
)

app = web.application(
            URLS, 
            globals())

wsgi = app.wsgifunc()

Command-line:

$ gunicorn -c gunicorn_conf.py main:wsgi

Request:

$ curl http://gevent.local

Output:

2014-06-14 15:01:50 [42624] [INFO] Starting gunicorn 19.0.0
2014-06-14 15:01:50 [42624] [DEBUG] Arbiter booted
2014-06-14 15:01:50 [42624] [INFO] Listening at: unix:/tmp/gevent.gunicorn.sock (42624)
2014-06-14 15:01:50 [42624] [INFO] Using worker: gevent
2014-06-14 15:01:50 [42629] [INFO] Booting worker with pid: 42629
2014-06-14 15:02:16 [42629] [DEBUG] GET /
2014-06-14 15:02:16 [42629] [ERROR] Error handling request
Traceback (most recent call last):
  File "/Users/dustin/development/python/guni_gevent_test/lib/python2.7/site-packages/gunicorn/workers/async.py", line 116, in handle_request
    raise StopIteration()
StopIteration
2014-06-14 15:02:16 [42629] [DEBUG] Closing connection. 

Versions:

gevent==1.0.1
greenlet==0.4.2
gunicorn==19.0.0
web.py==0.37
wsgiref==0.1.2

I get a result back, but it's always followed-up by this exception.

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Jun 14, 2014

Owner

what is the return when you're using the --config-check setting?

Owner

benoitc commented Jun 14, 2014

what is the return when you're using the --config-check setting?

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Jun 14, 2014

Owner

also the StopIteration here is normal since the request return an error. However the log should be more useful...

Owner

benoitc commented Jun 14, 2014

also the StopIteration here is normal since the request return an error. However the log should be more useful...

@dsoprea

This comment has been minimized.

Show comment
Hide comment
@dsoprea

dsoprea Jun 14, 2014

The request shouldn't return an error. I don't understand... I'm getting the intended string back, and it's a 200:

$ curl -v http://gevent.local
* Adding handle: conn: 0x7ffeb980d000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7ffeb980d000) send_pipe: 1, recv_pipe: 0
* About to connect() to gevent.local port 80 (#0)
*   Trying 127.0.0.1...
* Connected to gevent.local (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: gevent.local
> Accept: */*
> 
< HTTP/1.1 200 OK
* Server nginx/1.4.7 is not blacklisted
< Server: nginx/1.4.7
< Date: Sat, 14 Jun 2014 19:25:03 GMT
< Transfer-Encoding: chunked
< Connection: keep-alive
< 
* Connection #0 to host gevent.local left intact

...Why is the log indicating an error, and why am I getting back exactly what's expected if it is an error?

The "--config-check" option doesn't appear valid. It's not in the configuration documentation, and I get an error when passing as an option:

$ gunicorn --config-check -c gunicorn_conf.py main:wsgi
usage: gunicorn [OPTIONS] [APP_MODULE]
gunicorn: error: unrecognized arguments: --config-check

How am I supposed to use it?

dsoprea commented Jun 14, 2014

The request shouldn't return an error. I don't understand... I'm getting the intended string back, and it's a 200:

$ curl -v http://gevent.local
* Adding handle: conn: 0x7ffeb980d000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7ffeb980d000) send_pipe: 1, recv_pipe: 0
* About to connect() to gevent.local port 80 (#0)
*   Trying 127.0.0.1...
* Connected to gevent.local (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: gevent.local
> Accept: */*
> 
< HTTP/1.1 200 OK
* Server nginx/1.4.7 is not blacklisted
< Server: nginx/1.4.7
< Date: Sat, 14 Jun 2014 19:25:03 GMT
< Transfer-Encoding: chunked
< Connection: keep-alive
< 
* Connection #0 to host gevent.local left intact

...Why is the log indicating an error, and why am I getting back exactly what's expected if it is an error?

The "--config-check" option doesn't appear valid. It's not in the configuration documentation, and I get an error when passing as an option:

$ gunicorn --config-check -c gunicorn_conf.py main:wsgi
usage: gunicorn [OPTIONS] [APP_MODULE]
gunicorn: error: unrecognized arguments: --config-check

How am I supposed to use it?

@benoitc benoitc closed this in f41f86c Jun 14, 2014

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Jun 14, 2014

Owner

Sorry the option was --check-config like the doc said. Te fix above should fix your issue.

Owner

benoitc commented Jun 14, 2014

Sorry the option was --check-config like the doc said. Te fix above should fix your issue.

@dsoprea

This comment has been minimized.

Show comment
Hide comment
@dsoprea

dsoprea Jun 16, 2014

Everything looks fine with the config. I haven't yet tested the other PR, so the version I'm testing with hasn't changed.

$ PYTHONPATH=rpipe gunicorn --check-config -c rpipe/resources/data/gunicorn_conf_server_dev.py rpipe.app.wsgi_server:wsgi
$ echo $?
0

I'm assuming that your commit fixed it.

dsoprea commented Jun 16, 2014

Everything looks fine with the config. I haven't yet tested the other PR, so the version I'm testing with hasn't changed.

$ PYTHONPATH=rpipe gunicorn --check-config -c rpipe/resources/data/gunicorn_conf_server_dev.py rpipe.app.wsgi_server:wsgi
$ echo $?
0

I'm assuming that your commit fixed it.

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Jun 16, 2014

Owner

@dsoprea can you test though? the more eyes we are the best it is :)

Owner

benoitc commented Jun 16, 2014

@dsoprea can you test though? the more eyes we are the best it is :)

@dsoprea

This comment has been minimized.

Show comment
Hide comment
@dsoprea

dsoprea Jun 16, 2014

I just hadn't had the opportunity yet.

Later, I went back to the application that experienced it to begin with,
updated the virtualenv to point to Gunicorn master, and still saw the
StopIteration messages.

On Sun, Jun 15, 2014 at 10:59 PM, Benoit Chesneau notifications@github.com
wrote:

@dsoprea https://github.com/dsoprea can you test though? the more eyes
we are the best it is :)


Reply to this email directly or view it on GitHub
#790 (comment).

dsoprea commented Jun 16, 2014

I just hadn't had the opportunity yet.

Later, I went back to the application that experienced it to begin with,
updated the virtualenv to point to Gunicorn master, and still saw the
StopIteration messages.

On Sun, Jun 15, 2014 at 10:59 PM, Benoit Chesneau notifications@github.com
wrote:

@dsoprea https://github.com/dsoprea can you test though? the more eyes
we are the best it is :)


Reply to this email directly or view it on GitHub
#790 (comment).

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Jun 16, 2014

Owner

I don't follow, do you mean that after fetching the last master and installing it you see the same message?

Owner

benoitc commented Jun 16, 2014

I don't follow, do you mean that after fetching the last master and installing it you see the same message?

@dsoprea

This comment has been minimized.

Show comment
Hide comment
@dsoprea

dsoprea Jun 16, 2014

Yes.
On Jun 16, 2014 4:11 AM, "Benoit Chesneau" notifications@github.com wrote:

I don't follow, do you mean that after fetching the last master and
installing it you see the same message?


Reply to this email directly or view it on GitHub
#790 (comment).

dsoprea commented Jun 16, 2014

Yes.
On Jun 16, 2014 4:11 AM, "Benoit Chesneau" notifications@github.com wrote:

I don't follow, do you mean that after fetching the last master and
installing it you see the same message?


Reply to this email directly or view it on GitHub
#790 (comment).

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Jun 16, 2014

Owner

mm don't reproduce it... will recheck asap.

Owner

benoitc commented Jun 16, 2014

mm don't reproduce it... will recheck asap.

@flupke

This comment has been minimized.

Show comment
Hide comment
@flupke

flupke Jun 25, 2014

I had the same issue and the current master fixes it.

flupke commented Jun 25, 2014

I had the same issue and the current master fixes it.

@fletom

This comment has been minimized.

Show comment
Hide comment
@fletom

fletom Jul 14, 2014

Having the same issue. Any idea when this fix will make it into a release on PyPI?

fletom commented Jul 14, 2014

Having the same issue. Any idea when this fix will make it into a release on PyPI?

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Jul 14, 2014

Owner

On Mon, Jul 14, 2014 at 10:47 PM, Fletcher Tomalty <notifications@github.com

wrote:

Having the same issue. Any idea when this fix will make it into a release
on PyPI?

sometimes this week.

Owner

benoitc commented Jul 14, 2014

On Mon, Jul 14, 2014 at 10:47 PM, Fletcher Tomalty <notifications@github.com

wrote:

Having the same issue. Any idea when this fix will make it into a release
on PyPI?

sometimes this week.

@fletom

This comment has been minimized.

Show comment
Hide comment
@fletom

fletom Jul 14, 2014

@benoitc Great, thanks!

fletom commented Jul 14, 2014

@benoitc Great, thanks!

@dsoprea

This comment has been minimized.

Show comment
Hide comment
@dsoprea

dsoprea Jul 14, 2014

Ditto. Thanks, Benoit.
On Jul 14, 2014 5:04 PM, "Fletcher Tomalty" notifications@github.com
wrote:

@benoitc https://github.com/benoitc Great, thanks!


Reply to this email directly or view it on GitHub
#790 (comment).

dsoprea commented Jul 14, 2014

Ditto. Thanks, Benoit.
On Jul 14, 2014 5:04 PM, "Fletcher Tomalty" notifications@github.com
wrote:

@benoitc https://github.com/benoitc Great, thanks!


Reply to this email directly or view it on GitHub
#790 (comment).

jcushman added a commit to harvard-lil/perma that referenced this issue Jul 16, 2014

Roll back gunicorn 19.0.
Gunicorn 19.0 has a weird bug (benoitc/gunicorn#790) -- wait for 19.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment