Data-Pipelining

http://michael-harmon.com/blog/AirflowETL.html

This repository contains the artifacts for the presentation on Data Pipelining by Team 1 on tools such as GNU Make, Luigi, DASK, Celery and Airflow.

Video recording for the presentation can be found on Blackboard under Tegrity classes

Please feel free to drop us an email for any questions or suggestions :

Vishal Satam : satam.v@husky.neu.edu

Manasi Dalvi : dalvi.m@husky.neu.edu

Videos:

The links for the videos that we have recorded to explain the demo better are given below

Airflow

Part 1

https://youtu.be/Qs02p3mh8m4

Part2

https://youtu.be/83gPOMr6NOE

Celery

https://youtu.be/XelpLUcpKLM

DASK

https://www.youtube.com/watch?v=x4HGbh8mJNQ&t

Luigi

https://www.youtube.com/watch?v=NxsVEj0NeH4&t

Make

https://www.youtube.com/watch?v=Pb7lJMeXczo&t