Skip to content

Modak-Labs/modak

Repository files navigation

Modak

Tier-aware data federation between Postgres and Apache Iceberg. Modak knows which tier holds what, and every query stays real time and consistent.

Status: beta. No stable release yet, interfaces can still change. See production deployment before running it anywhere that matters.

Recent rows live in Postgres, history lives in Iceberg, and plain SQL works against the whole timeline: SELECT, INSERT, UPDATE, and DELETE reach any row, wherever it lives. Both tiers stay real, independently usable systems: an unforked Postgres you run OLTP on, and a standard Iceberg warehouse any engine can read. Modak owns only the seam between them, and the protocol is public.

modak-explainer.mp4

Tables run tiered (Postgres keeps only the recent partitions) or mirrored (Postgres keeps everything while CDC trails it into the lake). Choosing a mode walks the decision, and the contract states exactly what each mode supports.

Installation

Run the full loop locally with Docker:

git clone --recurse-submodules https://github.com/Modak-Labs/modak && cd modak
docker compose up -d --build
./example/run.sh

That brings up Postgres with the extension, MinIO as the Iceberg warehouse, and the worker, then walks through tiering, corrections, mirroring, and lifecycle end to end. The console lives at http://localhost:9090.

modak-demo-compressed.mp4

For the guided version, start with the quickstart. For pointing the worker at your own Postgres and object store, see production deployment.

Documentation

Full docs at modak-labs.github.io/modak:

  • Concepts: table modes, tier key, cut-line, pinned snapshot, delta
  • Choosing a mode: which mode fits your workload
  • The contract: the mode-by-mode matrix of what you can read, write, and correct
  • Working with tables: registering, reading, corrections
  • Ingestion: bulk ingestion and Stream Load
  • Operations: production deployment, AWS, day-2 operations, console
  • Reference: SQL API, CLI, configuration, catalog schema, metrics
  • Architecture: how the extension, the worker, and the catalog cooperate

The explainer and demo videos, and the toolchain that builds them, live in Modak-Labs/modak-media.

Contributing

Bug reports and feature requests go through issues, questions and ideas through discussions. See CONTRIBUTING.md for the local setup and how to send changes.

License

MIT. See LICENSE.

About

Tier-aware data federation between Postgres and Apache Iceberg

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors