digoal
2024-02-22
PostgreSQL , PolarDB , DuckDB , pg_quack , table access method
在PostgreSQL使用duckdb_fdw获得嵌入duckdb的能力.
- 《用chatgpt排查PG
create extension duckdb_fdw异常:could not load library...undefined symbol...》 - 《PolarDB-PG | PostgreSQL + duckdb_fdw + 阿里云OSS 实现高效低价的海量数据冷热存储分离》
- 《PolarDB 开源版通过 duckdb_fdw 支持 parquet 列存数据文件以及高效OLAP》
- 《用duckdb_fdw加速PostgreSQL分析计算, 提速40倍, 真香.》
- 《PostgreSQL 牛逼的分析型功能 - 列存储、向量计算 FDW - DuckDB_fdw - 无数据库服务式本地lib库+本地存储》
PostgreSQL 12开始支持table access接口, 例如heap接口, columnor接口:
《hydra, 一款基于PostgreSQL的开源HTAP数据库. 支持列存,向量化,物化,冷热分离存储,cloud 等特性》
pg_quack则是使用table access, 将duckdb作为pg存储的方法.
https://github.com/hydradatabase/pg_quack
安装
git clone --depth 1 https://github.com/hydradatabase/pg_quack
cd pg_quack
USE_PGXS=1 make install
依赖duckdb lib. Makefile:
...
UNAME_S := $(shell uname -s)
ifeq ($(UNAME_S),Darwin)
DUCKDB_LIB = libduckdb.dylib
DUCKDB_ZIP = libduckdb-osx-universal.zip
endif
ifeq ($(UNAME_S),Linux)
DUCKDB_LIB = libduckdb.so
UNAME_M := $(shell uname -m)
ifeq ($(UNAME_M),aarch64)
DUCKDB_ZIP = libduckdb-linux-aarch64.zip
endif
ifeq ($(UNAME_M),x86_64)
DUCKDB_ZIP = libduckdb-linux-amd64.zip
endif
endif
DUCKDB_VERSION ?= v0.9.2
DUCKDB_BASE_URL ?= https://github.com/duckdb/duckdb/releases/download/$(DUCKDB_VERSION)
REGRESS =
include $(PGXS)
all: quack.so
$(DUCKDB_DIR)/$(DUCKDB_ZIP):
curl -sL $(DUCKDB_BASE_URL)/$(DUCKDB_ZIP) -o $(DUCKDB_DIR)/$(DUCKDB_ZIP)
# this really likes to run everytime, not sure why
$(DUCKDB_DIR)/$(DUCKDB_LIB): $(DUCKDB_DIR)/$(DUCKDB_ZIP)
cd $(DUCKDB_DIR) && unzip -o -q $(DUCKDB_ZIP) $(DUCKDB_LIB)
...
