Glue scripts for converting AWS Service Logs for use in Athena

Build and deploy a serverless data pipeline on AWS with no effort.

The Sensitive Data Protection on AWS solution allows enterprise customers to create data catalogs, discover, protect, and visualize sensitive data across multiple AWS accounts. The solution eliminates the need for manual tagging to track sensitive data such as Personal Identifiable Information (PII) and classified information.

Extract, transform, and load data for analytic processing using AWS Glue

AWS ETL Pipleine

Terraform configuration that creates several AWS services, uploads data in S3 and starts the Glue Crawler and Glue Job.

This is a data pipeline built with the purpose of serving a business team.

Terraform module to create and manage a AWS Glue job

AWS streaming pipeline for real-time analysis

Terraform module which creates Glue Job resources on AWS.

ETL using application streaming and creating a Data Lake

This project outlines the final project requirements for DAV6100 - Information Architectures, focusing on group assignments, scoring criteria, topic selection, core requirements, and project components such as design, development, visualization, and executive presentation.

Used AWS Glue to perform ETL operations and load resultant data to AWS Redshift. In the second phase used AWS CloudWatch rules and LAMBDA to automatically run GLUE Jobs

AWS Glue & Airflow automate weekly churn analysis pipeline (S3, Redshift) for telco (1M+ customers) with actionable Tableau dashboard (user segmentation, churn reasons, geographical distribution).

O projeto foi elaborado com o objetivo de estabelecer uma arquitetura na AWS, originada a partir de uma migração de um banco de dados existente em um ambiente local (on-premise).

This Project aims to automate the process of infrastructure creation.

This project aims to analyze the popularity of YouTube content across different regions by leveraging datasets sourced from Kaggle. It employs a systematic approach to data preprocessing, cleaning, and analysis using various AWS (Amazon Web Services) services including S3, Lambda, Glue, and others, to build an automated ETL pipeline.

Glue Data Quality Example - Deploy to your AWS Account w/ Terraform to test