An end-to-end data pipeline which extracts divvy bikeshare data from web loads it into data lake and datawarehouse transforms it using dbt and finally , a dashboard to visualize the data using looker studio, the pipeline is orchestrated using prefect

End-to-end ELT data engineering project

Extracted the top 100 repo with highest star and keeps them in BigQuery, visualized through Looker.

A complete pipeline to pull data from Scryfall's "Magic: The Gathering"-API, via Prefect orchestration and dbt transformation.

This is a PySpark-based data pipeline that fetches weather data for a few cities, performs some basic processing and transformation on the data, and then writes the processed data to a Google Cloud Storage bucket and a BigQuery table.The data is then viewed in a looker dashboard

Data science dashboards on daily updated data (OpenData Barcelona)

Pipeline orchestration and analysis of Minneapolis 311 Service Request data.

Data Engineering Project to Extract and Process Solana Reddit Data

Analysis of 311 Service Requests for the City of NYC (from 2010 to 2023) Tech: Prefect cloud, dbt core, BigQuery, Compute Engine, CloudRun, Artifact Registry, Terraform, Docker

This is repository is related to building end to end data pipeline for football leagues analysis and predictive modeling.

OCR con comunicación a la API de BigQuery y presentación de los datos en Looker Studio.

💨 'Live' presenting air condition in Poland

This pipeline retrieves Yelp Fusion API restaurant data as JSON files, then loads it into Google Cloud Storage. The data is transformed while being moved into BigQuery and refined further with dbt. Finally, Looker Studio visualizes the processed data. Prefect orchestrates the entire workflow.

The Walmart Black Friday Real-Time Data Warehousing project aims to build an advanced solution for handling the surge of data on Black Friday, enabling strategic decision-making.

This repository contains scripts for extracting, transforming, loading, and processing Pokemon data from the PokeAPI - https://pokeapi.co/.

ETL pipeline built with Apache Airflow to prepare Uber and Lyft trip data for consumption by a Looker Studio report.