A Re-do of Perth City Properties project using Azure Data Engineering technologies such as Azure Data Factory (ADF), Azure Data Lake Storage Gen2, Azure Blob Storage, Azure Databricks.

Springboard Open Ended Capstone

Covid ETL Project using Azure Data Engineering Stack

The data engineering project aims to migrate a company's on-premises database to Azure, leveraging Azure Data Factory for data ingestion, transformation, and storage. The project will implement a three-stage storage strategy, consisting of bronze, silver, and gold data layers (Medalion architecture). Documentation of the project is in PDF file.

Azure-based solution for ingesting and analyzing Formula 1 data using Azure Data Lake Storage Gen2 and Databricks

Integration of Covid-19 data utilising Azure Data Factory to perform data ingestion, transformation and storage activities. The goal of this guided project was to become familiar with Microsoft Azure technologies, including; Azure Data Factory(ADF), Azure Data Lake Storage Gen2, Azure SQL Database, Azure Blob Storage, Dataflow, Databricks, etc.

Copying data from Amazon S3 bucket to Azure Blob container by using Azure Data Factory pipeline. This Data is mounted to Databricks and further analysis is done using Spark SQL.

data pipeline azure databricks