Send CloudEvents (and others) to an ASGI or WSGI application
cloud-events-handler
converts CloudEvents and other vendor specific events into ASGI/WSGI calls for your applications
How does an ASGI/WSGI application handle events from the cloud provider of choice?
Events could be
- CloudEvents
- AWS Events
- Google, Azure events too
Examples:
CloudEvent -> Lambda that processes the data in the event
S3 event -> Lambda that processes the file added
EventBridge -> Lambda that processes the data in the event
A whole application has parts that are accessible via HTTP and some that are reactive to non-HTTP events. The latter kind of event may need to be handled or processed with components from your application, e.g. middleware, models, serializers, etc.
pip install cloud-events
AWS S3 Example
- Include
cloud-events
in the requirements - Set the handler of a lambda to
cloud_events.handler.handler
- Set
WSGI_APPLICATION
to the python path of the WSGI application - Process the S3 event at
/aws.s3
of your application
my-django-app
views
Books
GET /books - returns a list of books
POST /books - possible but what if the book is > 6 MB? Lambda cannot handle that.
So let us allow users to upload that book directly to S3.
Now how do we get the metadata in the database when the user uploads the book?
We configure an event for each S3 object creation in our bucket.
Therefore, when the S3 event is sent, we want our Lambda to have access to the same code that the rest of our application uses.
DELETE /books/{id} - deletes a book
In this example instead of reacting only to an API Gateway or ALB event, we can process any event in our WSGI application.
Two settings are used to configure the cloud-event handler.
Required
WSGI_APPLICATION
points to the python path of the WSGI applicationASGI_APPLICATION
points to the python path of the ASGI application
Optional WSGI_ENVIRON
points to the python path of a module and is expected to end at a dictionary.
Example WSGI_ENVIRON
# in app/config
environ = {
"HTTP_ACCEPT": "application/vnd.api+json",
"PATH_INFO": "/books/",
"REQUEST_METHOD": "get"
}
WSGI_ENVIRON = app.config.environ
This is merged with the default environ and passed to the WSGI application.
Default environ:
{
"CONTENT_TYPE": "application/json",
"REQUEST_METHOD": "post",
"PATH_INFO": "/",
"SCRIPT_NAME": "",
"SERVER_NAME": "localhost",
"SERVER_PORT": "80",
"SERVER_PROTOCOL": str("HTTP/1.1"),
"wsgi.input": None,
"wsgi.version": (1, 0),
"wsgi.run_once": False,
"wsgi.multiprocess": False,
"wsgi.multithread": False,
"wsgi.url_scheme": "http",
}
You can use a custom handler instead of the provided handler by importing the handler function and passing the event
and context
.
from cloud_events.handler import handler
def my_handler(event, context):
do_my_special_thing()
return handler(event, context)
The default handler returns a dictionary.
{
"status_code": status_code <str>,
"body": body <str>,
"headers": headers <dict>,
}
Event | Path Attribute | Dynamic WSGI Path |
---|---|---|
CloudEvent | $.source |
/myContext |
AWS EventBridge | $.source |
/com.my.event |
AWS CloudWatch Event | $.source |
/com.my.event |
AWS IOT Event | $.event.eventName |
/myChargedEvent |
Events with a dynamic WSGI path mean that the path called in the WSGI application is dependent on the data in the event message at the path attribute specified.
AWS Event | Pattern Attributes | Static WSGI Path |
---|---|---|
S3 | $.Records[0].eventSource with : replaced by . |
/aws.s3 |
CodeCommit | $.Records[0].eventSource |
/aws.codecommit |
DynamoDB | $.Records[0].eventSource |
/aws.dynamodb |
EC2 Lifecycle | $.source |
/aws.ec2 |
Kinesis | $.Records[0].eventSource with : replaced by . |
/aws.kinesis |
SNS | $.Records[0].eventSource with : replaced by . |
/aws.sns |
SQS | $.Records[0].eventSource with : replaced by . |
/aws.sqs |
SES | $.Records[0].eventSource with : replaced by . |
/aws.ses |
Lex | $.bot and $.outputDialogMode and $.currentIntent |
/aws.lex |
Kinesis Firehose | $.records[0].kinesisRecordMetadata |
/aws.kinesis.firehose |
Cognito | $.identityId and $.identityPoolId |
/aws.cognito |
Config | $.configRuleId |
/aws.config |
CloudFront | $.Records[0].cf |
/aws.cloudfront |
CloudFormation | $.LogicalResourceId |
/aws.cloudformation |
Alexa | $.header and $.payload |
/aws.alexa |
Events with a static WSGI path mean that the path called in the WSGI application is the same for each event matching the pattern. Some events are essentially static since the route is defined by the message but that data is from AWS and unlikely to change.
- Clone this repo
- Install tox
pip install tox
- Run tests
tox
All package code is in src
Does this have ASGI support?
Yes
Are the responses JSON serializable?
Yes. This is also to support SQS which needs a response from a lambda. The response code from the WSGI application is sent back in the default response i.e. a 500 in the WSGI app will then be sent to SQS if using the default handler.
Does this have API Gateway/ALB support?
No. For now use other libraries as those are more fully featured.
This uses the same class the mangum uses. If the app is ASGI that is a well suited library for API gateway events
For AWS ALB and API Gateway events serverless-wsgi, zappa, or awsgi are good options
Are the URLs I add to handle the events also callable?
The urls used for events could also be called via HTTP if they are exposed on an API Gateway
Does this encourage monolithic lambdas?
Maybe
This started out with the question, "Can we take an AWS event, in a generic way, and send it through to Django via a handler that makes full use of Django?"
Yes we can :)
+------------------+
| |
| Event Producer |
| |
+------------------+
|
|
+---------------------v---------------------+
| |
| Event Consumer |
| |
| +---------------------+ |
| |Handler | |
| | | |
| |Any compute that | |
| |handles/processes the| |
| |event | |
| +---------------------+ |
| | |
| v |
| +---------------------+ |
| |Adapter | |
| | | |
| |Adapts events into | |
| |CloudEvents format | |
| +---------------------+ |
| | |
| v |
| +---------------------+ |
| |Interface | |
| | | |
| |Interfaces for A/WSGI| |
| +---------------------+ |
| | |
| v |
| +---------------------+ |
| |Application | |
| | | |
| |A/WSGI application | |
| +---------------------+ |
| |
+-------------------------------------------+