One framework to develop, deploy and operate data workflows with Python and SQL.

Projeto com o objetivo de extrair dados da web.

Cleaning the dataframe and adding columns with the average grade and whether the student was approved based on the average grade

😎 A curated list of awesome DataOps tools

A Data Engineering project. Repository for backend infrastructure and Streamlit app files for a Premier League Dashboard.

Data Engineering Project

Automating pipelines using airflow operators

Repository for Replication of Professor Teo Calvo's Projects

Minha resolução para um teste prático de uma vaga de Analytics Engineer Júnior

Realtime Data Streaming | End-to-End Data Engineering Project

An ETL pipeline that extracts, transforms, and loads data from various sources related to electric vehicle (EV) stocks.

ETL to move data from MySQL into BigQuery using Airflow

Data Engineering Project with Hadoop HDFS and Kafka

This project Synthetic data generator plus (SDGP) is a python script that generates mock data based on given configurations. It can also edit and scale existing data to create high volume data. It is useful for testing, learning data domine and prototyping purposes.

The Real-time Ecommerce Data Collection and Processing project empowers businesses with real-time insights by efficiently extracting, processing, and storing ecommerce data from multiple sources. Combining Golang and Python, this cutting-edge solution streamlines data handling from diverse ecommerce websites.

Data Processing with PySpark: Parsing Data from MongoDB

Scraping data from Spotify Playlist URL using Python and Selenium

ETL pipeline for ten movies from TMDB API.

Resoluções das atividades das aulas de Engenharia de Dados da Alpha EdTech.