End-to-end ELT data engineering project
-
Updated
Dec 24, 2022 - Python
End-to-end ELT data engineering project
This documentation is like a quick snapshot of my project in the data field, showing off my skills and know-how in this area.
An end-to-end data pipeline which extracts divvy bikeshare data from web loads it into data lake and datawarehouse transforms it using dbt and finally , a dashboard to visualize the data using looker studio, the pipeline is orchestrated using prefect
Use data from your Kentico Xperience website in Looker Studio reports.
Extracted the top 100 repo with highest star and keeps them in BigQuery, visualized through Looker.
A complete pipeline to pull data from Scryfall's "Magic: The Gathering"-API, via Prefect orchestration and dbt transformation.
This repository contains a custom connector script to fetch university data from universities.hipolabs.com and visualize it in Looker Studio using Google Apps Script.
This ETL (Extract, Transform, Load) project demonstrates the process of extracting data from a SQL Server database, transforming it using Python, orchestrating the data pipeline with Apache Airflow (running in a Docker container), loading the transformed data into Google BigQuery data warehouse, and finally creating a dashboard using Looker Studio.
This is a PySpark-based data pipeline that fetches weather data for a few cities, performs some basic processing and transformation on the data, and then writes the processed data to a Google Cloud Storage bucket and a BigQuery table.The data is then viewed in a looker dashboard
Data science dashboards on daily updated data (OpenData Barcelona)
The pipeline resides in Google BigQuery, providing dynamic insights into inventory availability and accurate demand forecasting in Vertex AI. The culmination of our efforts is a user-friendly dashboard developed on Google's Looker Studio.
Pipeline orchestration and analysis of Minneapolis 311 Service Request data.
Repositori ini berisi dokumentasi pengerjaan tugas Virtual Internship Experience Big Data Analytics di Kimia Farma yang difasilitasi oleh Rakamin Academy.
Solution for documenting & administration Events, Parameters & Annotations for Google Analytics 4 (GA4) using Google Sheet, BigQuery & Looker Studio.
Data Engineering Project to Extract and Process Solana Reddit Data
Repositório com os notebooks das atividades do curso Profissão: Analista de Dados na EBAC.
This pipeline retrieves Yelp Fusion API restaurant data as JSON files, then loads it into Google Cloud Storage. The data is transformed while being moved into BigQuery and refined further with dbt. Finally, Looker Studio visualizes the processed data. Prefect orchestrates the entire workflow.
Analysis of 311 Service Requests for the City of NYC (from 2010 to 2023) Tech: Prefect cloud, dbt core, BigQuery, Compute Engine, CloudRun, Artifact Registry, Terraform, Docker
Terraform Module for Google Cloud billing billboard.
An end-to-end data engineering project on github activities data
Add a description, image, and links to the looker-studio topic page so that developers can more easily learn about it.
To associate your repository with the looker-studio topic, visit your repo's landing page and select "manage topics."