Logging in celery outside of flask context?

[moodh]

Hello, I got sent here by Rollbar support. We currently use rollbar in both flask (request context) and celery (worker context) and we get exceptions thrown by the rollbar package.

This is how we setup celery:

@worker_init.connect
@beat_init.connect
def initialize_rollbar(**kwargs):
    del kwargs

    rollbar.init(
        config['ROLLBAR_TOKEN'],
        config['ENVIRONMENT'],
        root=config['API_PATH'],
        allow_logging_basic_config=False,
    )

    def base_data_hook(request, data):
        del request

        data['framework'] = 'celery'
    rollbar.BASE_DATA_HOOK = base_data_hook

Running a log.warning() inside a celery task correctly logs the error to rollbar but it also floods our local logs with this exception:

*testexception*

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/moodh/.pyenv/versions/3.6.3/envs/flow/lib/python3.6/site-packages/rollbar/__init__.py", line 756, in _add_person_data
    person_data = _build_person_data(request)
  File "/Users/moodh/.pyenv/versions/3.6.3/envs/flow/lib/python3.6/site-packages/rollbar/__init__.py", line 770, in _build_person_data
    if hasattr(request, 'rollbar_person'):
  File "/Users/moodh/.pyenv/versions/3.6.3/envs/flow/lib/python3.6/site-packages/werkzeug/local.py", line 347, in __getattr__
    return getattr(self._get_current_object(), name)
  File "/Users/moodh/.pyenv/versions/3.6.3/envs/flow/lib/python3.6/site-packages/werkzeug/local.py", line 306, in _get_current_object
    return self.__local()
  File "/Users/moodh/.pyenv/versions/3.6.3/envs/flow/lib/python3.6/site-packages/flask/globals.py", line 37, in _lookup_req_object
    raise RuntimeError(_request_ctx_err_msg)
RuntimeError: Working outside of request context.

This typically means that you attempted to use functionality that needed
an active HTTP request.  Consult the documentation on testing for
information about how to avoid this problem.

It seems to try to load flask (since it's importable) and inject people data (there's no request to get), but since the code tries to access flask's request context it complains that it isn't in a request context.

Can I somehow turn off the flask parts when we're not actually inside flask? Maybe use a celery request instead (task.request).

[rivkahstandig3636]

This appears to be the same issue as #177

[mattbillenstein]

Hitting this same problem, but RuntimeError subclasses exception which seems to be caught here, so I'm not sure why this is raising for me.

[mattbillenstein]

Ok, figured out this is a very subtle issue with requests getting imported before patching gevent... yikes!

[moodh]

Hi, after more testing I'm quite sure this is a bug rather than a question. We have no easy way other than running a local copy of the rollbar code to suppress these warnings. The problem occurs in __init__.py on line 63 to 66:

try:
    from werkzeug.wrappers import Request as WerkzeugRequest
except (ImportError, SyntaxError):
    WerkzeugRequest = None

Replacing this section with a simple WerkZeugRequest = None suppresses the warnings and lets the celery part of our codebase work as intended, but of course that also destroys the person tracking in the Flask request context. I don't really have a good idea on how to solve the issue properly.

Since py3 keeps a stack of exceptions our exception stacks show these bad exceptions in real logs which causes much confusion and problems when using Rollbar.

Is there a real possibility of this being fixed in 0.14.0 and is there an ETA?

[rivkahstandig3636]

We will be working on v0.14.0 shortly, although I don't have a specific ETA on when it will be released. But within the next few weeks.