Spark Structured Streaming application transferring Avro data from Kafka with Schema Registry to Delta Lake
🚦 Project of Data Warehouse in star architecture based on UK traffic data
Lakehouse storage system benchmark
An open-source storage framework that enables building a Lakehouse architecture
Some demos of using Spark to write MySQL and Kafka data to data lake,such as Delta,Hudi,Iceberg
Shed light on your data layout in order to monitor the health of your Lakehouse tables and identify when data maintenance operations should be performed.
DeltaOMS is a solution that help build a centralized repository of Delta Transaction logs and associated operational metrics/statistics for your Delta Lakehouse. Unity Catalog supported in the v0.7.0-rc1 release.Documentation here - https://databrickslabs.github.io/delta-oms/v0.7.0-rc1/
Spark structured streaming examples with using of version 3.5.1
This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
An open protocol for secure data sharing
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Add a description, image, and links to the delta-lake topic page so that developers can more easily learn about it.
To associate your repository with the delta-lake topic, visit your repo's landing page and select "manage topics."