New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gaiohttp worker does not work with aiohttp >= 2.0.0 #1526

Closed
munircontractor opened this Issue Jun 13, 2017 · 6 comments

Comments

Projects
None yet
5 participants
@munircontractor

munircontractor commented Jun 13, 2017

The latest version of gunicorn and aiohttp are incompatible with each other when using the gaiohttp worker class. Gunicorn still uses the aiohttp.wsgi module even though that module has been removed from aiohttp >=2.0.0. I used a git clone of gunicorn and aiohttp==2.1.0 for the tests below, and test:app is just the test app from the gunicorn docs.

$ gunicorn --workers=2 --worker-class=gaiohttp -b localhost:8080 test:app 

Error: class uri 'gaiohttp' invalid or not found: 

[Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/gunicorn/util.py", line 134, in load_class
    mod = import_module('.'.join(components))
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/usr/lib/python3.6/site-packages/gunicorn/workers/gaiohttp.py", line 14, in <module>
    from gunicorn.workers._gaiohttp import AiohttpWorker
  File "/usr/lib/python3.6/site-packages/gunicorn/workers/_gaiohttp.py", line 19, in <module>
    from aiohttp.wsgi import WSGIServerHttpProtocol as OldWSGIServerHttpProtocol
ModuleNotFoundError: No module named 'aiohttp.wsgi'
]

Solution suggested in Issue #1489 also does not solve the problem. It keeps trying to spawn workers until I manually kill it.

$ gunicorn --workers=2 --worker-class=aiohttp.GunicornWebWorker -b localhost:8080 test:app
[2017-06-13 12:14:43 -0400] [4345] [INFO] Starting gunicorn 19.7.1
[2017-06-13 12:14:43 -0400] [4345] [INFO] Listening at: http://127.0.0.1:8080 (4345)
[2017-06-13 12:14:43 -0400] [4345] [INFO] Using worker: aiohttp.GunicornWebWorker
[2017-06-13 12:14:43 -0400] [4349] [INFO] Booting worker with pid: 4349
[2017-06-13 12:14:43 -0400] [4349] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/home/munir/repos/gunicorn/gunicorn/arbiter.py", line 578, in spawn_worker
    worker.init_process()
  File "/home/munir/python3-virtualenv/lib/python3.6/site-packages/aiohttp/worker.py", line 41, in init_process
    super().init_process()
  File "/home/munir/repos/gunicorn/gunicorn/workers/base.py", line 131, in init_process
    self.run()
  File "/home/munir/python3-virtualenv/lib/python3.6/site-packages/aiohttp/worker.py", line 49, in run
    self.loop.run_until_complete(self._runner)
  File "/usr/lib64/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/home/munir/python3-virtualenv/lib/python3.6/site-packages/aiohttp/worker.py", line 105, in _run
    handler = self.make_handler(self.wsgi)
  File "/home/munir/python3-virtualenv/lib/python3.6/site-packages/aiohttp/worker.py", line 68, in make_handler
    "aiohttp.wsgi is not supported anymore, "
RuntimeError: aiohttp.wsgi is not supported anymore, consider to switch to aiohttp.web.Application

Also, currently using the aiohttp workers creates an import loop where gunicorn imports modules from aiohttp, which then imports modules from gunicorn. This makes it hard to upgrade versions independently.

@munircontractor munircontractor changed the title from `gaiohttp` worker does not work with aiohttp >= 2.0.0 to gaiohttp worker does not work with aiohttp >= 2.0.0 Jun 13, 2017

@kennethreitz

This comment has been minimized.

Collaborator

kennethreitz commented Aug 26, 2017

i hit this today

@berkerpeksag

This comment has been minimized.

Collaborator

berkerpeksag commented Aug 26, 2017

I've opened #1569 to address this.

@kennethreitz

This comment has been minimized.

Collaborator

kennethreitz commented Aug 28, 2017

Fixed by downgrading aiohhttp.

@amirbehzad

This comment has been minimized.

amirbehzad commented Oct 5, 2017

same here

@orokusaki

This comment has been minimized.

orokusaki commented Nov 26, 2017

The only solution seems to be:

aiohttp==1.3.5  # Latest version that works
@berkerpeksag

This comment has been minimized.

Collaborator

berkerpeksag commented Nov 26, 2017

Yes, pinning aiohttp to 1.3.5 is the easiest solution. I documented all options at http://docs.gunicorn.org/en/latest/design.html#asyncio-workers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment