Skip to content

WIP Serverless Streaming Analytics data pipeline composed of AWS Kinesis Data Streams, AWS Kinesis Data Analytics (Flink), AWS S3, AWS Lambda, Snowflake and Python components. Provisioned using Terraform.

Notifications You must be signed in to change notification settings

shaunmcglinchey/serverless-streaming-analytics

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

WIP

Client run instructions

  • python3 -m venv streaming_env
  • source streaming_env/bin/activate
  • pip3 install -r requirements.txt
  • python3 stock.py

TODO

  • Add TF to create Snowflake resources
  • Update Lambda with logic to write to Snowflake resources
  • Update README.md with build instructions
  • Parameterise Flink app - pass source,sink, region and app names in from the environment - these will be set in TF

About

WIP Serverless Streaming Analytics data pipeline composed of AWS Kinesis Data Streams, AWS Kinesis Data Analytics (Flink), AWS S3, AWS Lambda, Snowflake and Python components. Provisioned using Terraform.

Topics

Resources

Stars

Watchers

Forks