Utility code for AWS Services. Sample use case could be querying data from KDS. This project is open to receive contribution.

Demo of running AWS Lambda and AWS Kineses locally using Localstack

Pinterest's experiment analytics data pipeline which runs thousands of experiments per day and crunches billions of datapoints to provide valuable insights to improve the product.

AWS framework to stream, classify, index and store data

AWS Kensis Destionation - Http Endpoint Demo

Developed an ETL pipeline for real-time ingestion of stock market data from the stock-market-data-manage.onrender.com API. Engineered the system to store data in Parquet format for optimized query processing and incorporated data quality checks to ensure accuracy prior to visualization.

Simple Consent System implementation using AWS services and Python 3.

Big Data Lab | Stream Delivery AWS Kinesis Firehose

Recover AWS Kinesis Firehose errors by running the COPY command to each of the manifests file in the S3 errors folders

A production grade data pipeline that streams real time data of an online retailer website into AWS S3 data lake. The pipeline uses AWS Kinesis to receive real time streaming data and a lambda consumer which puts data into S3 data-lake.

Python script that produces logs observed by the Amazon Kinesis Agent

클라우드(S3) 기반으로 데이터를 변환 압축해 API를 제공하는 데이터 파이프라인

Streaming data pipeline on aws, Tech session repository for hist

Setup an ETL from Twitter API to S3

A streaming data pipeline leveraging managed services from AWS & the serverless architecture

This sample application demonstrates Amazon SNS with AWS X-Ray active tracing

Shipping AWS RDS logs from s3 into a kinesis stream

you run a script to mimic multiple sensors publishing messages on an IoT MQTT topic, with one message published every second. The events get sent to AWS IoT, where an IoT rule is configured. The IoT rule captures all messages and sends them to Firehose. From there, Firehose writes the messages in batches to objects stored in S3. In S3, you set u…

Recovers Kinesis Firehose source records from S3, replaying them into the stream