Description

A pipeline in AWS to transform a CSV file to a Parquet file, once the CSV file is uploaded to the input directory. Then, the transformed file is moved to the output directory, and the input file is changed to intelligent-tiering after 3 days.

Technologies

• Python
• AWS SDK for Pandas
• AWS Lambda
• Amazon S3
• Amazon EventBridge

Architecture

Snippet

Result

Input

Output