Global Tree Cover Loss Analysis using Geotrellis and SPARK
-
Updated
Jul 15, 2024 - Scala
Global Tree Cover Loss Analysis using Geotrellis and SPARK
This course is designed to provide learners with the fundamental skills needed for data engineering using Python. The objective is to introduce anyone interested in the topic to Python's data engineering-related features.
大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台,包含数据采集,调度,权限,审批流,私域营销等模块
This is an ELT data pipeline setup to track the activities of an e-commerce website based on orders, reviews, deliveries and shipment date. This project utilized technologies like Airflow, AWS RDS-Postgres, Python etc.
An Apache Airflow data pipeline is designed to perform ELT operations, utilizing Amazon S3 and Amazon Redshift Serverless.
Automated data pipeline for cryptocurrency exchange analytics. Extracts data from multiple sources, processes it, and visualizes insights via a near real-time dashboard. Built with Python and Docker, featuring modular ETL, comprehensive logging, and automated testing.
Roadmap for Data Engineering
Simple stream processing pipeline
Ethereum client written in Go, modified for full-hierarchy data exports and block specimen production
Terraform module designed to easily backup EFS filesystems to S3 using DataPipeline
data pipeline code to download / process hydrological observation data
Tensorflow 2 Tutorials (use tensorflow and keras in a better way!)
This is End-To-End Data Engineering Project using Airflow and Python. In this project, we will extract data using Twitter API, use python to transform data, deploy the code on Airflow/EC2 and save the final result on Amazon S3
Arcpy client for GFW Forest Loss Analysis
Python event consumer boilerplate which takes advantages from combine of multiprocessing, multithreading and asynchronous programming
An End-To-End data pipeline integration from Website Source to analytical dashboard in AWS using Python flask, ML models, DynamoDB and other AWS services.
Resource for ETL & Data Ingestion program using Apache Airflow
Add a description, image, and links to the datapipeline topic page so that developers can more easily learn about it.
To associate your repository with the datapipeline topic, visit your repo's landing page and select "manage topics."