urlopen calls on Python 3 for HTTPS are not concurrent

[andreibica]

I'm using eventlet (version 0.20.1) with Python 3.5.4 and its look like calling multiple urlopen in parallel is not working. Only one urlopen is executed at once.

Here is the provided urlopen example which was modified for Python 3 and it shows the issue.
Each request takes about one second, but doing them in parallel should take also around 1 second.
Still, the the total time is the sum required for each request.

import eventlet
from eventlet.green.urllib.request import urlopen

urls = [
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
    "https://httpstat.us/200?sleep=1000",
]


def fetch(url):
    print("opening", url)
    body = urlopen(url).read()
    print("done with", url)
    return url, body


pool = eventlet.GreenPool(200)
for url, body in pool.imap(fetch, urls):
    print("got body from", url, "of length", len(body))

[andreibica]

It looks like the problem appears only when using HTTPS.
For HTTP urlopen calls are concurent.

[temoto]

Thanks, I was confused about this bug report. Does it work with monkey_patch() ?

[andreibica]

It works with monkey_patch(), but I'm not sure if it is ok to apply monkey_patch on my code since it also contains code which is not eventlet based. I assume that monkey_patch affects all Python code running in the Python interpreter.

[temoto]

Yes, it contains code that is not eventlet based. For example, urllib from standard Python library.

Please, try this patch.

pip install https://github.com/eventlet/eventlet/archive/5739f73ce8c9af9cab128dc1258f2c6b1ee56cbb.zip

[andreibica]

The patch is working. The HTTPS requests are now concurrent without monkey_patch.

[temoto]

@andreibica thanks for testing.

And thank you very much for using from eventlet.green imports.

Fix will be merged into master when I construct a test case.