<lambda>() missing 1 required positional argument: 'environ' on elastic-apm==2.2.0

[bastbnl]

Just updated elasticapm to version 2.2.0 and ran into this issue:

2018-06-13 17:50:28,893 - ERROR - server - Exception inside application: <lambda>() missing 1 required positional argument: 'environ'
  File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
    result = coro.throw(exc)
  File "/var/www/brownpapersession/dev/env/lib/python3.5/site-packages/channels/http.py", line 190, in __call__
    await self.handle(body)
  File "/var/www/brownpapersession/dev/env/lib/python3.5/site-packages/asgiref/sync.py", line 108, in __call__
    return await asyncio.wait_for(future, timeout=None)
  File "/usr/lib/python3.5/asyncio/tasks.py", line 381, in wait_for
    return (yield from fut)
  File "/usr/lib/python3.5/asyncio/futures.py", line 380, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 304, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
    raise self._exception
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/var/www/brownpapersession/dev/env/lib/python3.5/site-packages/asgiref/sync.py", line 123, in thread_handler
    return self.func(*args, **kwargs)
  File "/var/www/brownpapersession/dev/env/lib/python3.5/site-packages/channels/http.py", line 200, in handle
    signals.request_started.send(sender=self.__class__, scope=self.scope)
  File "/var/www/brownpapersession/dev/env/lib/python3.5/site-packages/django/dispatch/dispatcher.py", line 178, in send
    for receiver in self._live_receivers(sender)
  File "/var/www/brownpapersession/dev/env/lib/python3.5/site-packages/django/dispatch/dispatcher.py", line 178, in <listcomp>
    for receiver in self._live_receivers(sender)
  <lambda>() missing 1 required positional argument: 'environ'

Some of the python packages I'm using on Python 3.5.3:

Django==2.0.6
celery==4.2.0
certifi==2018.4.16
cffi==1.11.5
channels==2.1.1
channels-redis==2.2.1
chardet==3.0.4
constantly==15.1.0
cryptography==2.2.2
csscompressor==0.9.5
daphne==2.1.2
redis==2.10.6
requests==2.19.0

The issue is not here when I revert to elastic-apm==2.1.1. Will try to do some digging

[bastbnl]

The issue disappears when I remove environ from
https://github.com/elastic/apm-agent-python/blob/v2.2.0/elasticapm/contrib/django/apps.py#L41:
lambda sender, environ, **kwargs: client.begin_transaction('request')

I guess this environ is intended to contain ELASTIC_APM['ENVIRONMENT']?

# APM
ELASTIC_APM = {
   'ENVIRONMENT': 'dev',
   'SERVICE_NAME': 'brwnppr',
...

[beniwohli]

Hey @bastbnl. Thanks for reporting this! I fear the problem here is with channels, or more precisely, our lack of test coverage for channels.

Django's own request_started signal provides two arguments, sender and a keyword argument environ, the latter being the WSGI environment.

Channels, on the other hand, provides sender and a keyword argument scope (I haven't found any docs for this, but here's the relevant code).

The good news is that we don't do anything with the environ dictionary, so we might get away with simply not specifying it in the lambda function.

While this shows the dire need of setting up some tests for channels, this will probably take some time. If I pushed the proposed fix to a branch in my fork, could you give it a try to see if it indeed fixes the issue?

[bastbnl]

Yes, happy to give it a try!

[beniwohli]

Awesome! You should be able to install it via

pip install git+https://github.com/beniwohli/apm-agent-python.git@fix-232

[bastbnl]

TL;DR: Looks good! The error is not there anymore, the application works and there are no new errors.

Just installed it with just 3 attempts

(env) bastb@bastb-vps:/var/www/brownpapersession/dev/brownpapersession$ pip install git+https://github.com/beniwohli/apm-agent-python.git@fix-232
Collecting git+https://github.com/beniwohli/apm-agent-python.git@fix-232
  Cloning https://github.com/beniwohli/apm-agent-python.git (to revision fix-232) to /tmp/pip-req-build-5182cbb5
Requirement already satisfied (use --upgrade to upgrade): elastic-apm==2.2.0 from git+https://github.com/beniwohli/apm-agent-python.git@fix-232 in /var/www/brownpapersession/dev/env/lib/python3.5/site-packages
Requirement already satisfied: urllib3 in /var/www/brownpapersession/dev/env/lib/python3.5/site-packages (from elastic-apm==2.2.0) (1.23)
Requirement already satisfied: certifi in /var/www/brownpapersession/dev/env/lib/python3.5/site-packages (from elastic-apm==2.2.0) (2018.4.16)
elasticsearch 6.2.0 has requirement urllib3<1.23,>=1.21.1, but you'll have urllib3 1.23 which is incompatible.
(env) bastb@bastb-vps:/var/www/brownpapersession/dev/brownpapersession$ pip install git+https://github.com/beniwohli/apm-agent-python.git@fix-232 --no-dependencies
Collecting git+https://github.com/beniwohli/apm-agent-python.git@fix-232
  Cloning https://github.com/beniwohli/apm-agent-python.git (to revision fix-232) to /tmp/pip-req-build-9q84ly1c
Requirement already satisfied (use --upgrade to upgrade): elastic-apm==2.2.0 from git+https://github.com/beniwohli/apm-agent-python.git@fix-232 in /var/www/brownpapersession/dev/env/lib/python3.5/site-packages
(env) bastb@bastb-vps:/var/www/brownpapersession/dev/brownpapersession$ pip install git+https://github.com/beniwohli/apm-agent-python.git@fix-232 --no-dependencies --upgrade
Collecting git+https://github.com/beniwohli/apm-agent-python.git@fix-232
  Cloning https://github.com/beniwohli/apm-agent-python.git (to revision fix-232) to /tmp/pip-req-build-5jbg_1cy
Installing collected packages: elastic-apm
  Found existing installation: elastic-apm 2.2.0
    Uninstalling elastic-apm-2.2.0:
      Successfully uninstalled elastic-apm-2.2.0
  Running setup.py install for elastic-apm ... done
Successfully installed elastic-apm-2.2.0

[beniwohli]

Awesome! I'll merge and push a release, will take about half an hour to an hour until 2.2.1 is on PyPI.

BTW, we added instrumentation of elasticsearch-py in 2.2, would be great to hear any feedback you might have on it (once you can actually run a working version :D )

[beniwohli]

2.2.1 is now on PyPI :) https://pypi.org/project/elastic-apm/2.2.1/

Thanks again for reporting this @bastbnl!

[bastbnl]

... and on my development server. All is looking well. Thx a lot for your swift action @beniwohli

btw happy to serve as a human testrunner to discover issues with the stack I'm using. I can try to contribute to automated testing using channels.