Python library for building and testing serverless, event-driven, data processing pipelines with AWS Lambda and Python.
Cognition-Pipeline is a python interface to the Serverless Framework, a widely-adopted toolkit for building serverless applications via configuration yaml files and code snippets. Cognition-Pipeline abstracts the components of an AWS serverless application to provide an object-oriented interface for building customized data processing pipelines or services with Serverless Framework. Your code is deployed and executed exactly how its written.
from pipeline import Pipeline, events # Simple pipeline with "Hello World" lambda function class HelloWorldPipeline(Pipeline): def __init__(self): super().__init__() @events.invoke def hello_world(self, event, context): print("Hello World!") pipeline = HelloWorldPipeline() hello_world = pipeline.hello_world def deploy(): pipeline.deploy()
- Install library:
pip install git+https://github.com/geospatial-jeff/cognition-pipeline.git\
- Create a new pipeline in a new directory:
- Build your pipeline in the
<new-directyr>/handler.pyscript. You may add:
- Lambda Functions: Create a new lambda function by creating a new Pipeline method.
- Events: Configure the lambda function's event trigger with decorators.
- Resources: Configure other AWS resources (SQS, SNS etc.) used by your Pipeline.
- Deploy your pipeline to AWS via Serverless Framework
- Deploy test pipeline to AWS with
- Run the unittests in