Playing around with Spark for dataset aggregation

Project to demonstrate basic data engineering skills

Trendyol Data Engineering Technical Case Study.

OpenSnowcat Relational Database Loader (Apache 2.0 License)

Optimal distributed data deduplication and supervised learning pipeline using Apache Spark

HTW Berlin | Prof. Dr. Gärtner | Content Management WS18

Data Brewery is an ETL (Extract-Transform-Load) program that connect to many data sources (cloud services, databases, ...) and manage data warehouse workflow.

Collection of various lab tests for a data engineering training

akka http service for serving spark machine learning models

This project contains the solutions to some basic exercises using Spark.

🚦 Project of Data Warehouse in star architecture based on UK traffic data

Example API implementation for Data Caterer

Common utilities used to implement hyppo integrations.

Scala library to validate tabular data loaded from CSV files

Comprehensive training program equips developers with essential skills in data engineering and data science life cycles, encompassing data processing, software development, ML/AI, and KPI visualization for real-world business problem-solving.

data engineering projects using Flink

Apache Spark With Scala - hands on with big data

Lambda architecture project

The hyppo data ingestion system worker components

A module on daily time-series ingestion from Twitter, written in Scala.