Data-Engineering-W22

This repository contains the notebooks for the Data Engineering course.

Lab #	Topic	Lab Notebook	Exercise Solutions Notebook
1	Explore Your Data	Lab 1
2	Data Visualization	Lab 2	Lab 2 Solution
3	Data Visualization II	Lab 3	Lab 3 Solution
4	Data Cleaning	Lab 4	Lab 4 Solution
5	Data Transformation	Lab 5	Lab 5 Solution
6	Data Integration	Lab 6	Lab 6 Solution
7	Data Tidying	Lab 7	Lab 7 Solution
8	Airflow	Slides

Prerequisites

This repository requires that you have:-

• Python3+
• Numpy
• Matplotlib
• Seaborn
• Jupyter Notebook

Installation of Prerequisites

Easy way (More HD space, less hassle)

Install Anaconda then just run Jupyter.

Hard way (Less HD space, more hassle)

Install Python3+

Make sure Python and pip are added to environment variables

From your Linux, Mac, or Windows terminal, verify that both are installed correctly.

$ python --version
$ pip --version

Using the same terminal install numpy, matplotlib, pillow and notebook

$ pip install numpy matplotlib pillow notebook

Alternative way (Cloud but you have to upload the data)

Click on the next to the notebook you would like to exercise.

Upload the data needed as Data.zip using the following command

from google.colab import files
uploaded = files.upload()

Extract the zipped folder into the cloud using the following command

!unzip [foldername].zip

How To Run

From your terminal, run this command then navigate to the .ipynb file you would like to exercise

jupyter notebook

License

MIT License