Skip to content
AWS Event Fork Pipelines helps you build event-driven serverless applications by providing pipelines for common event-handling requirements, such as event backup, analytics, and replay. The pipelines are based on AWS SAM, and can deployed directly from AWS SAR into your AWS account.
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Creating initial file from template Oct 23, 2018
images Rename architecture image files Mar 19, 2019
pipelines Architecture diagram corrections Mar 27, 2019
.gitignore Convert apps to use sam build/publish Feb 27, 2019

AWS Event Fork Pipelines

Build Status

AWS Event Fork Pipelines is an architectural pattern where an event source, such as an Amazon SNS topic, is used to send events to multiple processing pipelines. The high-level architecture looks like this:

AWS Event Fork Pipelines Architecture

Each processing pipeline creates a separate subscription to the Amazon SNS topic. SNS Subscription Filter Policies can be applied for each subscription to ensure each pipeline only receives the messages they want to process.

This repository implements AWS Event Fork Pipelines as a suite of Serverless applications. Each application implements general purpose, reusable event processing pipelines. All of the apps have been published to the AWS Serverless Application Repository (SAR) and can easily be integrated into an existing AWS SAM application using Nested Applications. An example application is also included that demonstrates composing the different event processing pipeline apps together using nested apps.

Serverless Applications

This repository showcases the following AWS Event Fork Pipelines serverless applications:

  1. Event Storage and Backup Pipeline - Processing pipeline that saves topic messages to an Amazon S3 bucket for use as backups or other purposes, e.g., to query with Amazon Athena.
  2. Event Search and Analytics Pipeline - Processing pipeline that saves topic messages to an AWS Elasticsearch cluster for search and analytics.
  3. Event Replay Pipeline - Processing pipeline that saves topic messages to a replay buffer SQS queue. In a disaster recovery scenario, messages from up to 14 days ago can be replayed back to another processing pipeline's SQS queue.


This repository also contains the following example applications to demonstrate the AWS Event Fork Pipelines:

  1. E-Commerce App - An example of using the AWS Event Fork Pipeline apps in an e-commerce use case.

License Summary

This sample code is made available under a modified MIT license. See the LICENSE file.

You can’t perform that action at this time.