Azure Data Factory Pipeline Project

This is a guided project where I used Azure Data Factory to create a data pipeline and connected it to Azure Data Studio for data processing and visualization.

Technologies Used:

• Azure Data Factory
• Azure Data Studio
• Azure Blob Storage
• Azure SQL Database
• SQL

Project Overview:

In this project, I built a data pipeline in Azure Data Factory to automate the process of extracting data, transforming it, and loading it into Azure Data Studio. The pipeline processes data from multiple sources and loads it into a destination for further analysis.

Steps Involved:

1. Set up the Azure Data Factory environment.
2. Create a pipeline to extract data from multiple sources using linked services.
3. Use data flows to transform the extracted data.
4. Load the transformed data into an Azure SQL Database.
5. Connect the Azure SQL Database to Azure Data Studio.
6. Schedule the pipeline to run automatically (optional).

How to Run the Project:

• To replicate this project, you will need access to Azure and set up Azure Data Factory.
• You can use the provided ARM template to import the pipeline structure into your own Azure Data Factory instance.

Screenshots:

Azure Data Factory Pipeline

Connection to Azure Data Studio

This project demonstrates how to use Azure Data Factory to automate data pipelines that extract, transform and load (ETL) data from multiple sources into Azure SQL database and also connect your server to Azure data studio.