Releases: abd-ulbasit/pgoverlay
Release list
v1.0.0
Changelog
- a9e2f43: build(cow): reproducible lazyrw builds for glibc and musl on amd64 and arm64 (@abd-ulbasit)
- caf0d36: build(image): cross-compile multi-arch images and stamp the build version (@abd-ulbasit)
- 0261beb: build(make): serialize the IT targets like CI, and test both JS SDKs (@abd-ulbasit)
- bd64ff8: build: add a .dockerignore (@abd-ulbasit)
- a1507f4: ci(bench): run hack/bench-cow.sh on a GitHub-hosted runner (@abd-ulbasit)
- ebf8ba6: ci(docs): build the mkdocs site strictly and deploy it to GitHub Pages (@abd-ulbasit)
- 9265058: ci: lazyrw reproducibility and C tests, postgres import audit, arm64 job (@abd-ulbasit)
- 778187d: ci: pin actions, scope the token, and test what we ship on a schedule (@abd-ulbasit)
- 290b2ea: deps: go 1.26.6 and moby/go-archive v0.3.0 to clear the govulncheck gate (@abd-ulbasit)
- d34d4f9: deps: go 1.26.6, moby/go-archive v0.3.3 and the pending dependabot updates (@abd-ulbasit)
- 246a44b: docs(adr): ADR-11, the copy-on-write strategy and the options evaluated (@abd-ulbasit)
- ffa2a36: docs(api): list every /v1 endpoint with its role, body and errors (@abd-ulbasit)
- 12b6042: docs(benchmarks): add the #49 results and a placeholder for the pgbench run (@abd-ulbasit)
- 72c990a: docs(benchmarks): fill the #49 pgbench table and add the torture-suite results (@abd-ulbasit)
- 4949899: docs(benchmarks): publish the read-path copy-up measurement (@abd-ulbasit)
- 1298338: docs(benchmarks): the #49 gate passes on GitHub-hosted runners (@abd-ulbasit)
- e27290d: docs(changelog): record the release fixes, Alpine seeds and the gate result (@abd-ulbasit)
- 495f187: docs(code-tour): walk the shim install and cow-mode read; close deep dive 7 (@abd-ulbasit)
- 98aac9c: docs(community): contributing guide, code of conduct, issue and PR templates (@abd-ulbasit)
- 286c4f8: docs(concepts): describe the settle as it ships and the checkpoint-time copy (@abd-ulbasit)
- 0276ebf: docs(concepts): explain the lazyrw shim, seed settle and clone copy-up (@abd-ulbasit)
- cb6a676: docs(eks): stop publishing a plaintext proxy to the whole internet (@abd-ulbasit)
- 28cdde1: docs(ghook): document repository-keyed names, delivery handling and every GHOOK_ variable (@abd-ulbasit)
- d5d5c60: docs(ha): explain how clients reach the leader and what failover guarantees (@abd-ulbasit)
- 73b2e45: docs(ha): explain why /readyz ignores leadership and the pods patch scope (@abd-ulbasit)
- fd0a58a: docs(hack): make the README tapes reproducible and honest about branch-from-branch (@abd-ulbasit)
- 466a086: docs(helm): note that the leader's pods patch grant is namespace-wide (@abd-ulbasit)
- 33ae87c: docs(kubernetes): document helper Secrets, node pinning, Pod Security and egress (@abd-ulbasit)
- 31e8f3d: docs(kubernetes): hostpath branches get the shim; csi and zfs are unchanged (@abd-ulbasit)
- b846594: docs(pgproxy): say what the uniform route refusal does not hide (@abd-ulbasit)
- 2538e25: docs(readme): quote the #49 benchmark and say where the first-write wait lands (@abd-ulbasit)
- 83c2d6a: docs(readme): reads copy nothing; a write copies the file it touches once (@abd-ulbasit)
- e4dee58: docs(reference): document lazyrw, the settle heartbeat and eager branches (@abd-ulbasit)
- 616c831: docs(security): add a threat model and hardening checklist (@abd-ulbasit)
- c966afc: docs(security): private reporting, version-agnostic policy, accurate advisories (@abd-ulbasit)
- 6e520fd: docs(security): say branch containers share the source's Docker network (@abd-ulbasit)
- f95ba5e: docs(security): scope the shim, name the probe helper, add the XFS CVE (@abd-ulbasit)
- 318dbbe: docs(testing): lead with the router endpoint and match the shipped behaviour (@abd-ulbasit)
- 2e0fe50: docs(upgrading): earlier branches stay eager until reset, recover or fork (@abd-ulbasit)
- 39907f7: docs: --lazyrw, --wal-recycle, cow-mode and pgoverlay_branch_cow_mode (@abd-ulbasit)
- 7cb3749: docs: add CHANGELOG.md with the v1.0.0 entry and link it from the site (@abd-ulbasit)
- 536823b: docs: add reference, troubleshooting and upgrade pages (@abd-ulbasit)
- 89f1a42: docs: bring concepts, architecture, code tour and ADRs up to date (@abd-ulbasit)
- 519f548: docs: copy-up mode, the volume root and pgoverlay_cow_copyup_mode (@abd-ulbasit)
- bb87904: docs: describe the dedicated at-rest key and safe token rotation (ADR-10, usage) (@abd-ulbasit)
- 9a9010d: docs: document seed settle and --seed-settle (@abd-ulbasit)
- c55cab2: docs: drop the screenshot placeholder and flag the images as amd64 only (@abd-ulbasit)
- 2ff50de: docs: fix the operational guides for v1 behaviour (@abd-ulbasit)
- 310d550: docs: point the demo links at pgoverlay-demo (@abd-ulbasit)
- d221e7b: docs: re-record demo.gif and features.gif against v1-hardening (@abd-ulbasit)
- a2dbd9e: docs: restart an eager branch with docker restart; note diff after settle (@abd-ulbasit)
- 8cd28a9: docs: rewrite the README for a first-time visitor (@abd-ulbasit)
- 2abd104: docs: the pgbench release gate passed; point at the bench-cow workflow (@abd-ulbasit)
- f77b3c3: feat(api): add POST /v1/branches/{name}/recover and pgb branch recover (@abd-ulbasit)
- c4f84c4: feat(api): advertise the router address in branch responses (@abd-ulbasit)
- 65c2330: feat(api): report password_unavailable on branches whose password cannot be decrypted (@abd-ulbasit)
- 1a7b23c: feat(branchd): --volume-root, --xfs-cowextsize and the copy-up probe (@abd-ulbasit)
- 5aba125: feat(branchd): wire --kube-helper-image, the instance id and the owner pod (@abd-ulbasit)
- 3d6410e: feat(cli): add source clear-mask and password-less sources (@abd-ulbasit)
- 72a492d: feat(config): --lazyrw and --wal-recycle for branchd, pgb and Helm (@abd-ulbasit)
- f0c36bd: feat(config): load or generate a dedicated at-rest key (@abd-ulbasit)
- 3b97a35: feat(cow): embed the lazyrw builds for the branch install helper (@abd-ulbasit)
- b10e14a: feat(cow): install the lazyrw builds into a branch's rw volume (@abd-ulbasit)
- ea5eacd: feat(cow): lazyrw shim that copies a relation up on first write, not first read (@abd-ulbasit)
- 132c2e2: feat(cow): pgoverlay-du, usage that counts reflinked extents once (@abd-ulbasit)
- 34cc9ca: feat(cow): preload lazyrw in the overlay entrypoint when it is safe (@abd-ulbasit)
- 838a816: feat(cow): probe whether overlay copy-up copies or clones (@abd-ulbasit)
- dece826: feat(engine): detect copy-up at startup and count usage by it (@abd-ulbasit)
- e5cc6f1: feat(engine): lazyrw in overlay branches, and the mode they run in (@abd-ulbasit)
- 1b187e9: feat(engine): recover failed branches on their existing data (@abd-ulbasit)
- bd61827: feat(engine): settle every new seed; --seed-settle=freeze|recover|off (@abd-ulbasit)
- 074b6dc: feat(ha): label the leader pod so the API Service can route to it (@abd-ulbasit)
- eacc6c6: feat(metrics): add pgoverlay_leader and pgoverlay_leader_transitions_total (@abd-ulbasit)
- eb09b82: feat(metrics): pgoverlay_branch_cow_mode (@abd-ulbasit)
- 7d1f6e0: feat(metrics): pgoverlay_cow_copyup_mode (@abd-ulbasit)
- 230f985: feat(pgctl): freeze and analyze a dump seed before its clean stop (@abd-ulbasit)
- e8f7aee: feat(pgctl): settle a pg_basebackup seed before branches start from it (@abd-ulbasit)
- 5d19dbe: feat(pgoverlaytest): let ProxyDSN target a separate router host (@abd-ulbasit)
- 096961a: feat(pgproxy): re-read a branch...
v1.0.0-rc.4 — pgbranch is now pgoverlay
pgbranch is now pgoverlay. The name states the mechanism: branches are OverlayFS
copy-on-write mounts over PGDATA. Every branch shares one read-only copy of the seeded
source and stores only the blocks it changes, so creating one is a mount rather than a
copy — 5 GiB branched in 1.89 s behind a 33.1 MiB writable layer, and the same 1.89 s at
1 GiB. Each branch is its own live Postgres container, so branches run concurrently, and a
branch can itself be branched.
The repository moved to github.com/abd-ulbasit/pgoverlay. GitHub redirects the old URL,
so existing clones and links keep working, but update your remote when convenient.
Breaking
This release renames every user-visible identifier. There is no automatic migration: a
deployment created by v1.0.0-rc.3 or earlier is invisible to this release rather than
broken by it — different state directory, registry filename, labels and resource names.
Destroy branches with the old version before upgrading.
| old | new | |
|---|---|---|
| Go module | github.com/abd-ulbasit/pgbranch |
github.com/abd-ulbasit/pgoverlay |
| env vars | PGBRANCH_*, GHOOK_PGBRANCH_* |
PGOVERLAY_*, GHOOK_PGOVERLAY_* |
| state dir | ~/.pgbranch |
~/.pgoverlay |
| registry db | pgbranch.db |
pgoverlay.db |
| labels | pgbranch.managed, pgbranch.branch.id, … |
pgoverlay.* |
| metrics | pgbranch_* (14 series) |
pgoverlay_* |
| in-container paths | /pgbranch/{merged,rw,lowerN} |
/pgoverlay/… |
| host paths | /var/lib/pgbranch, /etc/pgbranch |
/var/lib/pgoverlay, /etc/pgoverlay |
| volumes/containers | pgbranch-src-*, pgbranch-br-* |
pgoverlay-* |
| k8s namespace | pgbranch |
pgoverlay |
| k8s Lease | pgbranch-branchd |
pgoverlay-branchd |
| Helm chart | pgbranch |
pgoverlay |
| images | ghcr.io/abd-ulbasit/pgbranch-{branchd,ghook} |
…/pgoverlay-{branchd,ghook} |
| commit status | pgbranch/branch |
pgoverlay/branch |
| Go SDK packages | pgbranchconnect, pgbranchtest |
pgoverlayconnect, pgoverlaytest |
| npm packages | pgbranch-test, pgbranch-connect |
pgoverlay-test, pgoverlay-connect |
Unchanged: the pgb CLI, the branchd daemon, and the /v1 REST paths — those carry
no project name and are byte-identical.
Watch out for the commit status. pgbranch/branch → pgoverlay/branch is the one
rename that fails silently. A branch protection rule listing pgbranch/branch as a
required check will never be satisfied again — the new context is a different check, so
pull requests wait forever instead of erroring. Update the rule when you upgrade ghook.
GitHub Action
The composite actions now have a floating major tag:
- uses: abd-ulbasit/pgoverlay/action@v1
- uses: abd-ulbasit/pgoverlay/action/destroy@v1v1 is the Actions-convention moving major pointer, not a stable v1.0.0 of the project —
there still isn't one. Pin @v1.0.0-rc.4 if you want an immutable reference.
v1 is the Actions-convention moving major pointer, and this is the first tag of this
repository that carries the pgoverlay action at all — every earlier tag has the old
pgbranch one and does not work under the new name. It is not a stable v1.0.0 of the
project; there still isn't one. Pin @v1.0.0-rc.4 or a commit SHA if you want an
immutable reference.
Also since v1.0.0-rc.3
- The image name is no longer written down three times. The Helm ITs render the chart to
learn which image to build, and a new no-cluster test fails theunitjob when the
Makefile andvalues.yamldisagree — drift there used to surface only as a three-minute
Helm timeout with no mention of an image. - The Helm ITs dump pod state, events and branchd logs when an install never goes Ready,
instead of reportingAvailable: 0/1and nothing else. - Dependabot alerts on, weekly
gomod/github-actions/dockerupdates; security updates
deliberately off, with the reasoning written down in SECURITY.md. golang.org/x/crypto0.51.0 → 0.52.0 (13 advisories, 6 critical).- The helm-deploy and HA-failover suites gate CI again instead of
continue-on-error. - The govulncheck gate moved into
hack/vulncheck.shsomake vulnreproduces CI exactly.
Helm chart 0.2.0, appVersion 1.0.0-rc.4.
v1.0.0-rc.3 — supply-chain + CI hardening
Builds on rc.2 with the final pre-v1 hardening (evidence from a govulncheck scan + the race detector). All 5 CI jobs green (unit+race, vuln, integration, helm, kube).
- go1.26.4 toolchain — clears 5 reachable Go stdlib CVEs (net/http, net, crypto/x509, mime, net/textproto). Verified gone via govulncheck.
- CI
vulnjob — govulncheck (binary mode) on branchd + pgbranch-github, gating on any reachable vuln except a documented allowlist for the two unfixable Moby CVEs (GO-2026-4887/4883; no plugins → unreachable). go test -racenow runs in the unit job; fixed a test-only data race ininternal/actiontest.- SECURITY.md — reporting, scanning policy, accepted-advisory table.
Everything in rc.2 is included. Known: 2 upstream docker/docker CVEs with no fixed release (tracked; allowlist drops when Moby patches).
Reference the Action for testing as abd-ulbasit/pgbranch/action@v1.0.0-rc.3. The moving v1 pointer still tracks stable until final v1.0.0.
v1.0.0-rc.2 — pre-v1 hardening
Release candidate hardening pgbranch for v1.0, from a multi-agent security/correctness review (built-in reviewer + an independent pass using installed security-auditor / threat-modeling-expert / bughunter methodologies). All four CI jobs (unit, integration, helm, kube) green.
🔴 Correctness
- Data-loss fix: an interrupted branch-from-branch freeze can no longer let the reconcile reaper delete the parent's live volume. Root cause was a stuck-timer never reset mid-saga and an unconditional volume delete — both fixed (
updated_atbump on saga progress +CountLiveBranchesReferencingRWguard). - Atomic state machine:
TransitionBranchis a conditional-UPDATE CAS — closes concurrent destroy/reset double-execution (verified under-race). - Force-destroy branches stuck in
creating/resetting(no longer wait the 10-min stuck-timeout). - Swallowed compensation errors now log + increment
pgbranch_compensation_failures_total.
Lifecycle
--max-branchesquota → HTTP 403 (stops a CI/webhook storm filling the storage node).pgbranch_disk_bytes_free/_totalgauge + ENOSPC docs/alert.--default-ttl/--max-ttl(branches, incl. ghook, no longer leak forever).
Security
- Source-name validation (closes a ZFS dataset-path injection).
- Generic 500s (no internal leak); uniform pgproxy refusal (no branch enumeration).
- Webhook 1 MiB body cap; pgproxy startup read-deadline + connection cap + idle timeout.
- apiclient cleartext-token warning,
PGBRANCH_CA_CERT, skip-verify warning. - Branch passwords AES-256-GCM encrypted at rest (key from
PGBRANCH_TOKEN). - Hardened branchd/ghook securityContexts, gated NetworkPolicy, tightened RBAC, digest-pinned images,
AutomountServiceAccountToken=falseon branch/helper pods. - ghook branch namespacing (
gh-pr-N) prevents cross-PR branch reset; ghook startup guards; proxy plaintext warning.
Audit
- Every branch transition now records the acting token (migration v11).
GET /v1/branches/{name}/history+pgb history.
Migrations
Registry schema v9 → v11 (v10 token-hash index, v11 audit actor). Forward-only; applied automatically on open.
Note: the moving v1 Action tag still points at stable — reference this rc as abd-ulbasit/pgbranch/action@v1.0.0-rc.2 when testing.
v1.0.0-rc.1 — operational trust (road to v1)
First release candidate for 1.0. v0.3 was feature-complete; this RC adds the operational-trust bar that makes pgbranch something you run for a team.
Phase 7 — the trust bar
- Observability — Prometheus
/metrics(branch/source state, op-latency histograms, reconcile/reaper counters, in-flight gauge) and a real/readyz(registry + driver health). See docs/observability.md. - Reconcile loop + leak-proof GC — one periodic authoritative loop converges state: reaps TTLs, fails stuck rows, and GCs orphaned containers/volumes and dangling frozen layers — instance-scoped so it only ever reclaims its own resources. Surfaced as
pgb doctor(read-only drift report) andpgb gc. - Authz model — scoped API tokens with roles (admin/operator/viewer, stored only as hashes),
pgb token create/ls/revoke, first-class proxy wire-TLS in the chart, and a namespaced deployer Role that replaces the cluster-admin recipe. - HA via leader election — run
branchdwith--leader-elect/replicaCount > 1: a coordination Lease elects one active writer; non-leaders serve reads + probes + metrics and 503 writes. See docs/ha.md.
Hardening shaken out by the new reconcile loop (all real concurrency fixes)
- Instance-scoped resource ownership (multi-instance safe).
- Branch container id recorded before the readiness wait — reconcile never reaps a branch mid-create.
- Idempotent container removal — a reconcile pass racing an explicit destroy no longer errors.
Also since v0.2 (Phase 6)
GitHub commit statuses + App auth + live PR comment + git-ref branch naming; Go and JS test-suite SDKs and a reusable Action; --via dump seeding for managed Postgres (Supabase/Neon/RDS); per-branch credential rotation; registry on a PVC; and pgb diff (schema + row-delta review).
No operator/CRDs (by design). Merge-back and multi-writer branches remain non-goals. CI green (unit + real-Docker integration + helm).
v0.3.0 — GitHub App, test-suite SDK, durability, branch diff
Highlights
GitHub integration, finished
pgbranch/branchcommit status on the PR head (pending → success/failure) — gate CI on branch readiness instead of polling- GitHub App authentication: ghook mints its own short-lived installation tokens (RS256, zero new dependencies); PAT remains the quick path
- The PR comment is now a live status panel, updated in place: creating → ready (connect string + expiry) → reset @ sha → destroyed
- Branch naming can follow the git ref (
GHOOK_BRANCH_NAMING=git-branch) so preview platforms (e.g. Vercel viaVERCEL_GIT_COMMIT_REF) derive the database branch from the first build — no PR-association race
A real database for every test
- Go:
pgbranchtest.Acquire(t)— an isolated, prod-shaped branch per test, auto-destroyed (stdlib-only public package) - JS: zero-dependency
pgbranch-testpackage (sdk/js) - Reusable GitHub Actions:
abd-ulbasit/pgbranch/action@main(+/destroy) - New guide: docs/testing.md
Durability
- Registry on a PVC (
persistence.*, auto-enabled in CSI mode) - Opt-in per-branch credentials (
--rotate-branch-credentials): every branch gets its own password via in-branch ALTER ROLE; reset re-rotates - Kubernetes guide is now CSI-first; hostpath documented as single-node/dev with its node-loss trade-off
Review tooling
pgb diff BRANCH/GET /v1/branches/{name}/diff— unified schema diff plus per-table row deltas vs the branch's own base snapshot (via an internal throwaway clone, ~5–10s)
Hardening from the EKS deployment (also in this release)
- Engine waits for a routable address before marking branches ready (kubelet status-sync race)
- Webhook deliveries are acked immediately; branch operations run detached (GitHub's 10s delivery timeout no longer cancels sagas)
--via dumpseeding for managed Postgres (Supabase/Neon/RDS) — no REPLICATION privilege needed- Full EKS walkthrough: docs/eks.md
pgbranch v0.2.0 — instant Postgres branches
First public release. pgbranch gives you git-branch semantics for Postgres: instant copy-on-write branches of any-size databases, self-hosted, on Docker or Kubernetes.
Headline numbers (measured — see docs/benchmarks.md)
- Branch creation: ~1.9 s p50, size-independent (identical at 1 GiB and 5 GiB)
- Disk overhead per branch: ~33 MiB
- Branch-from-a-running-branch: ~4.8 s p50
What's inside
- Core engine — OverlayFS copy-on-write assembled inside disposable Postgres containers; saga-based lifecycle (no orphans); seeding from any live Postgres via pg_basebackup; SQLite registry with journaled state machine
- branchd — REST API (bearer auth, optional TLS), embedded wire-protocol router (connect to
dbname@branchon one port; SCRAM relays transparently; optional TLS), TTL reaper, branch reset, source refresh with generations - Branch-from-branch — frozen-layer DAG: snapshot a running branch in seconds; layers are refcounted and GC'd
- Kubernetes — runtime driver with two storage modes: hostPath storage-node, or CSI PVC-clone (no SYS_ADMIN, no node pinning — branches schedule anywhere); Helm chart included
- Branch per PR — GitHub webhook service: PR opened → branch + connection-info comment; merged/closed → destroyed
- Data masking — per-source SQL hooks applied before a branch is marked ready
- Web UI — embedded, zero external assets
- Postgres 14–18 support matrix (validated), experimental ZFS backend, mkdocs docs
Scope (honest)
Dev/test data tool: branches are disposable, single-writer, point-in-time. Not an HA system, no merge-back.
🤖 Built with Claude Code