This is a simple monitoring tool that intercepts the results of Celery tasks and uploads the result to AWS CloudWatch.
Install from PyPi:
pip install celery-cloudwatch-logs
Configuration is done through environment variables. The following environment variables must be set in order for
celery-cloudwatch to work:
AWS_SECRET_KEY are optional. If not specified, Boto3 will handle the configuration for these keys, as described here:
Optionally, the following environment variables may be set:
The CloudWatch log group does not have to exist. If it doesn't exists, it will be created.
$ celery cloudwatch
This requires you to have ran
setup.py or installed this package through pip. Alternatively, run:
$ python -m celery_cloudwatch
How it works
Celery CloudWatch connects to your broker and monitors tasks in real time. When a task succeeded, failed was rejected or revoked, it uploads all available information about that task into a log stream on AWS CloudWatch Logs.
Based on the specified log group name in the
AWS_CLOUDWATCH_GROUP_NAME, a log group will be created. For each possible result, a separate log stream is created. For each task result, an entry is added to the log stream associated with the result.
celery cloudwatchis ignored.
- No descriptive way to specify other brokers than Redis.
All brokers supported by Celery will work, simply specify the broker URL through the
REDIS_URL environment variable.