Home

Welcome to the delta wiki!

What is Delta

Delta Lake

Github org, major repos

open-source data management for the lakehouse

• Adding an ACID transactional storage layer on top of object storage
• data versioning, rollback
• handle both batch and streaming data in a unified way

Donoted to Linux Foundation

• LF announcement
• Databricks annoucement
• it was launch in October 2017
• from Databricks, by Oct 2019

DeltaLogs

Delta Lake transaction Log

• Table Ledger: single source of truth trackcing table changes
• guarantees atomicity of Delta Lake
• transaction details are logged in JSON format.

Delta Table

The table in Delta Lake

• Delta tables are schema abstractions over data files that are stored in Delta format

• Doc: Create a Delta table

Delta engine

A query optimizer for Delta Lake

• Proprietary to Databricks
• for performance of Spark SQL, Databricks SQL and Dataframe operations
• apply push down design

Delta Live Tables (DLT): Data pipeline framework

DLT is not a table

• It manage the data flow, task orchestration, data quality and error handling
• It offers declarative pipeline development
• It contains pipeline, the main unit of execution
• It manages many Delta tables
• proprietary to Databricks

Delta Sharing

An open protocol for secure data sharing