Skip to content

force pyparser when gevent is being used #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

muodov
Copy link

@muodov muodov commented Jul 30, 2014

C-implementation is not working well with gevent async library. Python implementation does work. I've added an additional check to force pyparser when gevent monkey patching is detected.

@muodov muodov changed the title force pyparser when gevent is used force pyparser when gevent is being used Jul 30, 2014
@dhagrow
Copy link

dhagrow commented Nov 4, 2014

Can you be more specific? I have been using gevent with the C implementation and have not noticed any problems.

@dhagrow
Copy link

dhagrow commented Nov 11, 2014

Your example is flawed. Change the last lines to:

for i in xrange(100):
    pool = Pool()
    pool.spawn(handle, i)
pool.join()

You must join after all your greenthreads have spawned. As you wrote
it, you are waiting for each to complete before spawning the next.

On Tue, Nov 11, 2014 at 8:54 AM, muodov notifications@github.com wrote:

Sure, check out this snippet:

from gevent.monkey import patch_all
patch_all()from gevent import spawnfrom gevent.pool import Poolimport socketfrom http_parser.reader import SocketReaderfrom http_parser.http import HttpStream

request = '''GET / HTTP/1.0\r\n\r\n'''
addr = 'wikipedia.org'
def handle(id):
print id, 'started'
conn = socket.create_connection((addr, 80), timeout=5)
conn.send(request)
reader = SocketReader(conn)
strm = HttpStream(reader)
resp = ''
for chunk in strm:
resp += chunk
print id, 'finished'
for i in xrange(100):
pool = Pool()
pool.spawn(handle, i)
pool.join()

The output is something like

0 started
0 finished
1 started
1 finished
2 started
2 finished
3 started
3 finished
...

So the gevent threads are not switching while we are iterating over
request chunks.


Reply to this email directly or view it on GitHub
#51 (comment).

@muodov
Copy link
Author

muodov commented Nov 11, 2014

Yes, I've already spotted it and removed the comment. I can't reproduce this issue now, but I am pretty sure I saw this behavior in the first place. I suppose I should find the actual code I was working on. Closing for now...

@muodov muodov closed this Nov 11, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants