Implemented Snowflake project on AWS for efficient data storage and transformation. Utilized JSON-to-CSV conversion, Snowpipe for real-time ingestion, and reader accounts for secure data access. Employed streams, tasks, and materialized views for data synchronization and optimization. Implemented masking policies for enhanced data security.

ETL with AWS Lambda and Snowflake with automation using Snowpipe.

Get the streaming data from the S3 bucket with SQS queue. Load into Snowflake with Snowpipe and modify the data with Snowflake task

Realtime data pipeline using Kafka + Spark + AWS S3 (Terraform) + Snowflake

Retail data analysis pipeline utilizing AWS S3, Snowflake, Python, SQL, and Tableau. It demonstrates data transformation and setup in Jupyter Notebook, integrates real-time retail insights via an automated Tableau dashboard with Snowflake, and employs a CRON job in Jupyter Lab connected to Amazon SQS for consistent data updates.

Spotify ETL pipeline on AWS

Building a Seamless Data Pipeline with AWS and Snowpipe

The goal is to build an example of a simple data collection pipeline which collects data from multiple customers and uploads the data to Snowflake could look like. The OpenMeteo Api acts as the "customer system" in this case.

❄️ Stream real-time data into Snowflake with Amazon Kinesis Firehose

Automatically move data from AWS S3 to Snowflake datawarehouse continuously with snowpipe

SQL, Databases, warehouses, Data lake, cloud storage, MYSQL, Data Pipeline

SQL Scripts related to my learning on the Snowflake data cloud provider

SQL, Databases, warehouses, Data lake, cloud storage, MYSQL, Data Pipeline

Generating Streaming Data for Snowpipe Streaming

CLI script for interacting with Snowpipe REST APIs