Skip to content

Releases: abd-ulbasit/pgoverlay

v1.0.0

Choose a tag to compare

@github-actions github-actions released this 29 Sep 10:37
v1.0.0
a245fa5

Changelog

Read more

v1.0.0-rc.4 — pgbranch is now pgoverlay

Choose a tag to compare

@abd-ulbasit abd-ulbasit released this 27 Jul 22:10
ab337d1

pgbranch is now pgoverlay. The name states the mechanism: branches are OverlayFS
copy-on-write mounts over PGDATA. Every branch shares one read-only copy of the seeded
source and stores only the blocks it changes, so creating one is a mount rather than a
copy — 5 GiB branched in 1.89 s behind a 33.1 MiB writable layer, and the same 1.89 s at
1 GiB. Each branch is its own live Postgres container, so branches run concurrently, and a
branch can itself be branched.

The repository moved to github.com/abd-ulbasit/pgoverlay. GitHub redirects the old URL,
so existing clones and links keep working, but update your remote when convenient.

Breaking

This release renames every user-visible identifier. There is no automatic migration: a
deployment created by v1.0.0-rc.3 or earlier is invisible to this release rather than
broken by it — different state directory, registry filename, labels and resource names.
Destroy branches with the old version before upgrading.

old new
Go module github.com/abd-ulbasit/pgbranch github.com/abd-ulbasit/pgoverlay
env vars PGBRANCH_*, GHOOK_PGBRANCH_* PGOVERLAY_*, GHOOK_PGOVERLAY_*
state dir ~/.pgbranch ~/.pgoverlay
registry db pgbranch.db pgoverlay.db
labels pgbranch.managed, pgbranch.branch.id, … pgoverlay.*
metrics pgbranch_* (14 series) pgoverlay_*
in-container paths /pgbranch/{merged,rw,lowerN} /pgoverlay/…
host paths /var/lib/pgbranch, /etc/pgbranch /var/lib/pgoverlay, /etc/pgoverlay
volumes/containers pgbranch-src-*, pgbranch-br-* pgoverlay-*
k8s namespace pgbranch pgoverlay
k8s Lease pgbranch-branchd pgoverlay-branchd
Helm chart pgbranch pgoverlay
images ghcr.io/abd-ulbasit/pgbranch-{branchd,ghook} …/pgoverlay-{branchd,ghook}
commit status pgbranch/branch pgoverlay/branch
Go SDK packages pgbranchconnect, pgbranchtest pgoverlayconnect, pgoverlaytest
npm packages pgbranch-test, pgbranch-connect pgoverlay-test, pgoverlay-connect

Unchanged: the pgb CLI, the branchd daemon, and the /v1 REST paths — those carry
no project name and are byte-identical.

Watch out for the commit status. pgbranch/branch → pgoverlay/branch is the one
rename that fails silently. A branch protection rule listing pgbranch/branch as a
required check will never be satisfied again — the new context is a different check, so
pull requests wait forever instead of erroring. Update the rule when you upgrade ghook.

GitHub Action

The composite actions now have a floating major tag:

- uses: abd-ulbasit/pgoverlay/action@v1
- uses: abd-ulbasit/pgoverlay/action/destroy@v1

v1 is the Actions-convention moving major pointer, not a stable v1.0.0 of the project —
there still isn't one. Pin @v1.0.0-rc.4 if you want an immutable reference.

v1 is the Actions-convention moving major pointer, and this is the first tag of this
repository that carries the pgoverlay action at all — every earlier tag has the old
pgbranch one and does not work under the new name. It is not a stable v1.0.0 of the
project; there still isn't one. Pin @v1.0.0-rc.4 or a commit SHA if you want an
immutable reference.

Also since v1.0.0-rc.3

  • The image name is no longer written down three times. The Helm ITs render the chart to
    learn which image to build, and a new no-cluster test fails the unit job when the
    Makefile and values.yaml disagree — drift there used to surface only as a three-minute
    Helm timeout with no mention of an image.
  • The Helm ITs dump pod state, events and branchd logs when an install never goes Ready,
    instead of reporting Available: 0/1 and nothing else.
  • Dependabot alerts on, weekly gomod/github-actions/docker updates; security updates
    deliberately off, with the reasoning written down in SECURITY.md.
  • golang.org/x/crypto 0.51.0 → 0.52.0 (13 advisories, 6 critical).
  • The helm-deploy and HA-failover suites gate CI again instead of continue-on-error.
  • The govulncheck gate moved into hack/vulncheck.sh so make vuln reproduces CI exactly.

Helm chart 0.2.0, appVersion 1.0.0-rc.4.

v1.0.0-rc.3 — supply-chain + CI hardening

Choose a tag to compare

@abd-ulbasit abd-ulbasit released this 27 Jul 22:10

Builds on rc.2 with the final pre-v1 hardening (evidence from a govulncheck scan + the race detector). All 5 CI jobs green (unit+race, vuln, integration, helm, kube).

  • go1.26.4 toolchain — clears 5 reachable Go stdlib CVEs (net/http, net, crypto/x509, mime, net/textproto). Verified gone via govulncheck.
  • CI vuln job — govulncheck (binary mode) on branchd + pgbranch-github, gating on any reachable vuln except a documented allowlist for the two unfixable Moby CVEs (GO-2026-4887/4883; no plugins → unreachable).
  • go test -race now runs in the unit job; fixed a test-only data race in internal/actiontest.
  • SECURITY.md — reporting, scanning policy, accepted-advisory table.

Everything in rc.2 is included. Known: 2 upstream docker/docker CVEs with no fixed release (tracked; allowlist drops when Moby patches).

Reference the Action for testing as abd-ulbasit/pgbranch/action@v1.0.0-rc.3. The moving v1 pointer still tracks stable until final v1.0.0.

v1.0.0-rc.2 — pre-v1 hardening

Pre-release

Choose a tag to compare

@abd-ulbasit abd-ulbasit released this 27 Jul 22:10

Release candidate hardening pgbranch for v1.0, from a multi-agent security/correctness review (built-in reviewer + an independent pass using installed security-auditor / threat-modeling-expert / bughunter methodologies). All four CI jobs (unit, integration, helm, kube) green.

🔴 Correctness

  • Data-loss fix: an interrupted branch-from-branch freeze can no longer let the reconcile reaper delete the parent's live volume. Root cause was a stuck-timer never reset mid-saga and an unconditional volume delete — both fixed (updated_at bump on saga progress + CountLiveBranchesReferencingRW guard).
  • Atomic state machine: TransitionBranch is a conditional-UPDATE CAS — closes concurrent destroy/reset double-execution (verified under -race).
  • Force-destroy branches stuck in creating/resetting (no longer wait the 10-min stuck-timeout).
  • Swallowed compensation errors now log + increment pgbranch_compensation_failures_total.

Lifecycle

  • --max-branches quota → HTTP 403 (stops a CI/webhook storm filling the storage node).
  • pgbranch_disk_bytes_free/_total gauge + ENOSPC docs/alert.
  • --default-ttl / --max-ttl (branches, incl. ghook, no longer leak forever).

Security

  • Source-name validation (closes a ZFS dataset-path injection).
  • Generic 500s (no internal leak); uniform pgproxy refusal (no branch enumeration).
  • Webhook 1 MiB body cap; pgproxy startup read-deadline + connection cap + idle timeout.
  • apiclient cleartext-token warning, PGBRANCH_CA_CERT, skip-verify warning.
  • Branch passwords AES-256-GCM encrypted at rest (key from PGBRANCH_TOKEN).
  • Hardened branchd/ghook securityContexts, gated NetworkPolicy, tightened RBAC, digest-pinned images, AutomountServiceAccountToken=false on branch/helper pods.
  • ghook branch namespacing (gh-pr-N) prevents cross-PR branch reset; ghook startup guards; proxy plaintext warning.

Audit

  • Every branch transition now records the acting token (migration v11). GET /v1/branches/{name}/history + pgb history.

Migrations

Registry schema v9 → v11 (v10 token-hash index, v11 audit actor). Forward-only; applied automatically on open.

Note: the moving v1 Action tag still points at stable — reference this rc as abd-ulbasit/pgbranch/action@v1.0.0-rc.2 when testing.

v1.0.0-rc.1 — operational trust (road to v1)

Choose a tag to compare

@abd-ulbasit abd-ulbasit released this 27 Jul 22:10

First release candidate for 1.0. v0.3 was feature-complete; this RC adds the operational-trust bar that makes pgbranch something you run for a team.

Phase 7 — the trust bar

  • Observability — Prometheus /metrics (branch/source state, op-latency histograms, reconcile/reaper counters, in-flight gauge) and a real /readyz (registry + driver health). See docs/observability.md.
  • Reconcile loop + leak-proof GC — one periodic authoritative loop converges state: reaps TTLs, fails stuck rows, and GCs orphaned containers/volumes and dangling frozen layers — instance-scoped so it only ever reclaims its own resources. Surfaced as pgb doctor (read-only drift report) and pgb gc.
  • Authz model — scoped API tokens with roles (admin/operator/viewer, stored only as hashes), pgb token create/ls/revoke, first-class proxy wire-TLS in the chart, and a namespaced deployer Role that replaces the cluster-admin recipe.
  • HA via leader election — run branchd with --leader-elect / replicaCount > 1: a coordination Lease elects one active writer; non-leaders serve reads + probes + metrics and 503 writes. See docs/ha.md.

Hardening shaken out by the new reconcile loop (all real concurrency fixes)

  • Instance-scoped resource ownership (multi-instance safe).
  • Branch container id recorded before the readiness wait — reconcile never reaps a branch mid-create.
  • Idempotent container removal — a reconcile pass racing an explicit destroy no longer errors.

Also since v0.2 (Phase 6)

GitHub commit statuses + App auth + live PR comment + git-ref branch naming; Go and JS test-suite SDKs and a reusable Action; --via dump seeding for managed Postgres (Supabase/Neon/RDS); per-branch credential rotation; registry on a PVC; and pgb diff (schema + row-delta review).

No operator/CRDs (by design). Merge-back and multi-writer branches remain non-goals. CI green (unit + real-Docker integration + helm).

v0.3.0 — GitHub App, test-suite SDK, durability, branch diff

Choose a tag to compare

@abd-ulbasit abd-ulbasit released this 27 Jul 22:10

Highlights

GitHub integration, finished

  • pgbranch/branch commit status on the PR head (pending → success/failure) — gate CI on branch readiness instead of polling
  • GitHub App authentication: ghook mints its own short-lived installation tokens (RS256, zero new dependencies); PAT remains the quick path
  • The PR comment is now a live status panel, updated in place: creating → ready (connect string + expiry) → reset @ sha → destroyed
  • Branch naming can follow the git ref (GHOOK_BRANCH_NAMING=git-branch) so preview platforms (e.g. Vercel via VERCEL_GIT_COMMIT_REF) derive the database branch from the first build — no PR-association race

A real database for every test

  • Go: pgbranchtest.Acquire(t) — an isolated, prod-shaped branch per test, auto-destroyed (stdlib-only public package)
  • JS: zero-dependency pgbranch-test package (sdk/js)
  • Reusable GitHub Actions: abd-ulbasit/pgbranch/action@main (+ /destroy)
  • New guide: docs/testing.md

Durability

  • Registry on a PVC (persistence.*, auto-enabled in CSI mode)
  • Opt-in per-branch credentials (--rotate-branch-credentials): every branch gets its own password via in-branch ALTER ROLE; reset re-rotates
  • Kubernetes guide is now CSI-first; hostpath documented as single-node/dev with its node-loss trade-off

Review tooling

  • pgb diff BRANCH / GET /v1/branches/{name}/diff — unified schema diff plus per-table row deltas vs the branch's own base snapshot (via an internal throwaway clone, ~5–10s)

Hardening from the EKS deployment (also in this release)

  • Engine waits for a routable address before marking branches ready (kubelet status-sync race)
  • Webhook deliveries are acked immediately; branch operations run detached (GitHub's 10s delivery timeout no longer cancels sagas)
  • --via dump seeding for managed Postgres (Supabase/Neon/RDS) — no REPLICATION privilege needed
  • Full EKS walkthrough: docs/eks.md

pgbranch v0.2.0 — instant Postgres branches

Choose a tag to compare

@abd-ulbasit abd-ulbasit released this 27 Jul 22:10

First public release. pgbranch gives you git-branch semantics for Postgres: instant copy-on-write branches of any-size databases, self-hosted, on Docker or Kubernetes.

Headline numbers (measured — see docs/benchmarks.md)

  • Branch creation: ~1.9 s p50, size-independent (identical at 1 GiB and 5 GiB)
  • Disk overhead per branch: ~33 MiB
  • Branch-from-a-running-branch: ~4.8 s p50

What's inside

  • Core engine — OverlayFS copy-on-write assembled inside disposable Postgres containers; saga-based lifecycle (no orphans); seeding from any live Postgres via pg_basebackup; SQLite registry with journaled state machine
  • branchd — REST API (bearer auth, optional TLS), embedded wire-protocol router (connect to dbname@branch on one port; SCRAM relays transparently; optional TLS), TTL reaper, branch reset, source refresh with generations
  • Branch-from-branch — frozen-layer DAG: snapshot a running branch in seconds; layers are refcounted and GC'd
  • Kubernetes — runtime driver with two storage modes: hostPath storage-node, or CSI PVC-clone (no SYS_ADMIN, no node pinning — branches schedule anywhere); Helm chart included
  • Branch per PR — GitHub webhook service: PR opened → branch + connection-info comment; merged/closed → destroyed
  • Data masking — per-source SQL hooks applied before a branch is marked ready
  • Web UI — embedded, zero external assets
  • Postgres 14–18 support matrix (validated), experimental ZFS backend, mkdocs docs

Scope (honest)

Dev/test data tool: branches are disposable, single-writer, point-in-time. Not an HA system, no merge-back.

🤖 Built with Claude Code