pg_mooncake 🥮

Real-time analytics on Postgres tables

Overview

pg_mooncake is a Postgres extension that creates a columnstore mirror of your Postgres tables in Iceberg, enabling fast analytics queries with sub-second freshness:

• Real-time ingestion powered by moonlink for streaming and batched INSERT/UPDATE/DELETE.
• Fast analytics accelerated by DuckDB, ranking top 10 on ClickBench.
• Postgres-native allowing you to query a columnstore table just like a regular Postgres table.
• Iceberg-native making your data readily accesssible by other query engines.

Installation

Option 1: Docker

For new users, we recommend using the Docker image to get started quickly:

docker run --name mooncake --rm -e POSTGRES_PASSWORD=password mooncakelabs/pg_mooncake

This will start Postgres with pg_mooncake preinstalled. You can then connect to it using psql with the default user postgres:

docker exec -it mooncake psql -U postgres

Option 2: From Source

To build pg_mooncake, first install Rust, pgrx, and the build tools for DuckDB.

Then, clone the repository:

git clone --recurse-submodules https://github.com/Mooncake-Labs/pg_mooncake.git

To build and install for Postgres versions 14-17, run:

# Replace with your Postgres version
cargo pgrx init --pg17=$(which pg_config)
make install PG_VERSION=pg17

Finally, add pg_mooncake to shared_preload_libraries in your postgresql.conf file and enable logical replication:

shared_preload_libraries = 'pg_mooncake'
wal_level = logical

For a complete walkthrough, refer to our Dockerfile.

Quick Start

First, create the pg_mooncake extension:

CREATE EXTENSION pg_mooncake;

Next, create a regular Postgres table trades:

CREATE TABLE trades(
  id bigint PRIMARY KEY,
  symbol text,
  time timestamp,
  price real
);

Then, create a columnstore mirror trades_iceberg that stays in sync with trades:

CALL mooncake.create_table('trades_iceberg', 'trades');

Now, insert some data into trades:

INSERT INTO trades VALUES
  (1,  'AMD', '2024-06-05 10:00:00', 119),
  (2, 'AMZN', '2024-06-05 10:05:00', 207),
  (3, 'AAPL', '2024-06-05 10:10:00', 203),
  (4, 'AMZN', '2024-06-05 10:15:00', 210);

Finally, query trades_iceberg to see that it reflects the up-to-date state of trades:

SELECT avg(price) FROM trades_iceberg WHERE symbol = 'AMZN';

Contributing

pg_mooncake is an open-source project maintained by Mooncake Labs and licensed under the MIT License. We'd love your help to make it even better! Join our Slack, participate in discussions, open issues to report bugs or suggest features, contribute code and documentation, or help us improve the project in any way. All contributions are welcome! 🥮