pgColumnar 1.0-alpha

First tagged release of pgColumnar: columnar storage for PostgreSQL, written as a
clean-room, MIT-licensed implementation. Tables are created USING pgcolumnar after
CREATE EXTENSION pgcolumnar, and are written in the native on-disk format PGCN v1.

Supported: PostgreSQL 15, 16, 17, 18 and 19 beta.

What it does

• Column-oriented table access method with per-column compression (none, pglz,
  lz4, zstd), an adaptive per-vector encoding cascade, zone maps for row-group
  and per-vector skipping, and per-chunk bloom filters.
• Column projection, on by default: a scan reads only the columns a query
  references. Worth 6.9x on a single-column aggregate over a 100M-row, 21-column
  fixture.
• Full DML: INSERT, UPDATE, DELETE, index scans, index-only scans through a
  columnar visibility map, unique constraints, foreign keys and triggers.
• Parallel scan, and parallel COPY loading through pgcolumnar.parallel_copy.
• Arrow and Parquet import and export, including a Parquet foreign-data wrapper.
• Clustering and maintenance: vacuum_sorted, declarative sort_by, online
  recluster, sort_status, compaction and space reclaim.
• Opt-in vectorized aggregates, grouped and ungrouped, serial and parallel.

Measured

100,000,000-row TSBS cpu fixture, 21 columns, all engines on the same host with the
same (hostname, time DESC) index. Full tables and method in
docs/benchmarks.md.

Storage, the same 100M rows:

heap	pgColumnar (zstd)	TimescaleDB columnstore
22 GB	6,590 MB (3.4x smaller)	7,975 MB

Query time, stated plainly. Against heap, pgColumnar wins the wide scan-bound
aggregates (q4, q5, q6, q8) by roughly 20 to 50 percent and loses the one-host
queries. Against TimescaleDB it is slower on every query TimescaleDB completes,
by 1.6x on q4 and q5 and by 101x, 442x and 679x on q1, q2 and q3.

The one-host gap has a known cause and it is a layout one: this fixture is stored in
load order, which is sorted on neither key (measured correlation 0.013 for time,
-0.004 for hostname), so the zone maps skip nothing. Measured on q2, pgColumnar
reads 667 of 667 row groups and filters 99,995,680 rows to return 4,320. A user with
this shape would cluster on hostname, which is what TimescaleDB's segmentby does
for it. That configuration is not measured here, so the q1-q3 gap should not be
read as a property of columnar storage until it is.

Verification

• Full five-major matrix: 112 suites on each of PostgreSQL 15, 16, 17, 18 and 19.
  ALL VERSIONS PASSED, zero failures.
• Sanitizer gate: ASAN + UBSAN, violations fatal, over the write, read, encode and
  import/decode paths.
• CI builds every major and runs the suites on 17 and 18 for every pull request.

Known limitations

This is an alpha and its sharp edges are written down rather than left in the tracker.
The full list is in CHANGELOG.md
and docs/limitations.md:

• Point lookups are slower than heap, and the cost of a fetch grows with table
  width: an index fetch decodes the attribute prefix up to the highest-numbered
  column the query reads, not only the columns it reads.
• The index-fetch cost model is bounded, not modelled against the plan above it,
  so a consumer that stops early inherits more of the penalty than it should
  (#376).
• The grouped vectorized aggregate's parallel arm is declined on an expression
  grouping key, because the core Finalize is priced off a group estimate that can be
  25x to 42x wrong (#369).
  Off by default.
• The by-row-number fetch cache is bounded by 4 x (cap + retained position indexes + groupBuffer) rather than 4 x cap; a table of many wide varlena columns
  measured one 62 MB entry against a 32 MB cap
  (#364).
• Bulk load is slower than heap on most shapes, from 0.67x to 8.83x depending on
  column type and width.

A note on the extension version

VERSION and this tag read 1.0-alpha, but the extension's own default_version is
still 1.0-dev, so SELECT extversion FROM pg_extension reports 1.0-dev. That
field governs ALTER EXTENSION UPDATE and moving it needs an upgrade script that does
not exist yet; breaking installs to tidy a version string is the wrong trade. It will
move with the first release that ships one.

Installation

See docs/installation.md.
make && make install, then CREATE EXTENSION pgcolumnar;.

The clean-room record is in
PROVENANCE.md.