Weather_Difference_Between_SFO/SEA

This code is used to connect to fixer.io api, Download the currency exchange rates as keeping EUR as base currency. It also provides functions to download the historical data for any date and the average conversion date between 2 dates.

AirAware, a distributed data platform for monitoring air quality at arbitrary U.S. locations, developed during the 18B Insight Data Engineering session

Data engineering Project

Reddit trending

Pipeline validation using Great Expectations library

Compressing data using Python and compression techniques for better data storage and transfer.

ETL workflow for Udacity's Data Engineering Nanodegree project 3 on Redshift data warehousing

ETL pipeline build in Postgres with Python and SQL.

Airflow - The Forex Data Pipeline

Crosslend DE Assignment

Github star counter

Some useful stuff for Data Engineers

In the Project Workspace, I'll find a data set containing real messages that were sent during disaster events. I will be creating a machine learning pipeline to categorize these events so that you can send the messages to an appropriate disaster relief agency. This project will include a web app where an emergency worker can input a new message …

Streaming techniques to build a data pipeline which pulls data from an external API (RESTFul) that returns massive amounts of data, and auto-insert into a PostgreSQL database

Udacity DataEngineering NanoDegree First Project(Data Modeling)

Extract logs based off events from sysmon. Comes as a package, cli and ui.

Personal Apache Beam studies repository

Data engineering interviews Q&A for data community by data community

Courses and projects on Data Camp