A complete pipeline to pull data from Scryfall's "Magic: The Gathering"-API, via Prefect orchestration and dbt transformation.

An end-to-end data pipeline which extracts divvy bikeshare data from web loads it into data lake and datawarehouse transforms it using dbt and finally , a dashboard to visualize the data using looker studio, the pipeline is orchestrated using prefect

Analysis of 311 Service Requests for the City of NYC (from 2010 to 2023) Tech: Prefect cloud, dbt core, BigQuery, Compute Engine, CloudRun, Artifact Registry, Terraform, Docker

Data Engineering Project to Extract and Process Solana Reddit Data

End-to-end ELT data engineering project

Pipeline orchestration and analysis of Minneapolis 311 Service Request data.

Steam and Batch pipeline to to show updates in DataTalksClub Youtube video analytics

ETL pipeline for cleaning/transforming historical Bitcoin price data, and web-scraping new data going forward in order for ChatGPT to analyze and email you some fun predictions of the future price of Bitcoin.

Python package for uploading files and folders to Google Drive

Extracted the top 100 repo with highest star and keeps them in BigQuery, visualized through Looker.

Spotify API, Airflow, Docker, AWS S3, Snowflake, dbt, localstack, Looker Studio

This is a PySpark-based data pipeline that fetches weather data for a few cities, performs some basic processing and transformation on the data, and then writes the processed data to a Google Cloud Storage bucket and a BigQuery table.The data is then viewed in a looker dashboard

Creating a robust and scalable data pipeline on Google Cloud Platform (GCP) to monitor and analyze stock performance. Leveraging the power of GCP's data processing and storage services, a comprehensive solution has been built to efficiently collect, process, and visualize stock data.

This is repository is related to building end to end data pipeline for football leagues analysis and predictive modeling.

This pipeline retrieves Yelp Fusion API restaurant data as JSON files, then loads it into Google Cloud Storage. The data is transformed while being moved into BigQuery and refined further with dbt. Finally, Looker Studio visualizes the processed data. Prefect orchestrates the entire workflow.

Varios Projetos de Engenharia de Dados usando principais ferramentas como: Airflow, Snowflake, dbt, Postrgres, Looker Studio, Power BI

ETL pipeline built with Apache Airflow to prepare Uber and Lyft trip data for consumption by a Looker Studio report.

random quote JSON API load to BigQuery and Scheduling using Airflow and make a near real-time dashboard in Looker Studio

OCR con comunicación a la API de BigQuery y presentación de los datos en Looker Studio.