Skip to content

pgvfs 0.3.0

Pre-release
Pre-release

Choose a tag to compare

@adonm adonm released this 14 Oct 15:30
· 7 commits to main since this release

Native DuckDB databases in PostgreSQL: ATTACH 'pgdb://<volume>/<name>' is a real DuckDB database (indexes, keys, constraints, WAL) with one writer and any number of readers that follow new commits by themselves.

INSTALL pgvfs FROM 'https://pgvfs.adonm.dev';   -- DuckDB 1.5.6, or a 2.0 dev build
LOAD pgvfs;

What's in 0.3.0

Storage layout: v2 (unchanged). pgdb:// adds five tables
(dbschema.sql), created by the first database writer; the pgvfs:// tables
do not change.

  • pgdb://: native DuckDB databases in PostgreSQL (experimental). ATTACH 'pgdb://<volume>/<name>' is a real DuckDB database (ART indexes, primary and
    foreign keys, constraints, sequences, WAL) stored as versioned 8 KB chunks.
    One writer, refused at attach if another holds it and fenced on every write;
    any number of readers, on standbys too, each reading one published version
    and re-attached at the newest every pgdb_refresh_interval seconds (running
    transactions keep theirs). Readers cache through DuckDB's external file cache
    under keys a new version leaves valid. Experimental: its tables may still
    change, with a migration. See
    Databases for the design, limits
    and benchmarks: against a local file, warm readers run at 93% of the
    throughput, loads at 1.24x the time, commits at about 2 ms; cold scattered
    lookups are the slow case.
  • Concurrent searches of one tantivy split read in parallel, through up to
    four handles instead of one under a lock.
  • Secrets are found for database-level I/O (no client context), through
    the file opener.
  • pgvfs_stats() adds db_reads, db_read_bytes, db_pieces,
    db_cache_bytes and db_publishes.
  • Tests and benchmarks: just bench-db (pgdb:// against a local DuckDB
    file), just sqllogic (DuckDB's own SQL tests on pgdb://), and a
    pgdb:// storage contract and end-to-end test in just contract.

Upgrading from beta.6: nothing to do. Readers of pgdb:// need SELECT on the
new tables, writers INSERT, UPDATE and DELETE (see
Databases).

Docs: https://pgvfs.adonm.dev · Changelog: https://github.com/adonm/pgvfs/blob/v0.3.0/CHANGELOG.md · All builds: https://pgvfs.adonm.dev/install.html#builds