Skip to content

fix(phlower): reclaim freed pages so DROP actually shrinks the file#27

Merged
webjunkie merged 1 commit into
mainfrom
fix/sqlite-incremental-vacuum
May 4, 2026
Merged

fix(phlower): reclaim freed pages so DROP actually shrinks the file#27
webjunkie merged 1 commit into
mainfrom
fix/sqlite-incremental-vacuum

Conversation

@webjunkie
Copy link
Copy Markdown
Contributor

Why

prod-us PVC filled at 49 GB on May 4 ~04:47 UTC during the hourly purge. Emergency purge dropped invocations_legacy (26 GB), but the file didn't shrink — SQLite's DROP TABLE marks pages as free in an internal freelist without returning them to the OS, and the DROP transaction's WAL temporarily ballooned to fill what little headroom was left. Result: disk full, INSERTs failing with sqlite3.OperationalError, ~14h of records silently dropped through the buffer-cap safety net.

What

  • PRAGMA auto_vacuum=INCREMENTAL set in _connect() before any tables are created. Tracks freed pages in a separate list so they can be reclaimed via PRAGMA incremental_vacuum. SQLite requires this PRAGMA on a fresh DB — on existing DBs it's a no-op without a full VACUUM, which is exactly why this PR pairs with a PVC drop and fresh-start deploy.
  • _reclaim_free_pages() called after DROP TABLE in purge_old_partitions. Uses PRAGMA incremental_vacuum to truncate freed pages off the file. The PRAGMA emits one result row per freed page so the cursor MUST be drained with fetchall() — without that, only one page gets reclaimed.
  • SQLITE_DISK_USAGE_PCT_CAP default lowered from 85 → 75, giving headroom for WAL spikes during DROP. With incremental_vacuum keeping the file lean this should rarely fire.

Verified locally

after writes: 20.34 MB
after purge:   0.05 MB

20 MB of records → DROP TABLE → file truncates to 50 KB. Without fetchall() the same test only reclaimed 1 page out of 5188.

Deploy plan

  1. Merge → image builds
  2. Drop the PVC
  3. Restart phlower → fresh DB picks up auto_vacuum=INCREMENTAL at first connect
  4. From here on, daily partition drops keep the file size tracking live data instead of high-water mark

Test plan

  • auto_vacuum verified as INCREMENTAL on fresh DB
  • purge_old_partitions shrinks file from 20 MB → 50 KB
  • Deploy to prod-us, verify file size stays bounded over the next purge cycle

…file

prod-us PVC filled at 49 GB on 2026-05-04 04:47 UTC during the hourly
purge. Emergency purge dropped invocations_legacy (26 GB), but the
file didn't shrink — SQLite's DROP TABLE marks pages as free in the
file's internal freelist without returning them to the OS, and the
DROP transaction's WAL temporarily ballooned to fill what little
headroom was left. Result: disk full, INSERTs failing with
sqlite3.OperationalError, ~14h of records silently dropped through
the buffer-cap safety net.

Three changes:

- PRAGMA auto_vacuum=INCREMENTAL set in _connect() before any tables
  are created. Tracks freed pages in a separate list so they can be
  reclaimed by PRAGMA incremental_vacuum. Must be set on a fresh DB —
  on existing DBs it requires a full VACUUM to apply, which is exactly
  why this PR pairs with a PVC drop / fresh-start deploy.

- _reclaim_free_pages() called after DROP TABLE in
  purge_old_partitions(). Uses PRAGMA incremental_vacuum to truncate
  the freed pages off the file. The PRAGMA emits one result row per
  freed page so the cursor MUST be drained with fetchall() — without
  that, only one page gets reclaimed and the file barely shrinks.

- SQLITE_DISK_USAGE_PCT_CAP default lowered from 85 → 75, giving more
  headroom for WAL spikes during a DROP. With incremental_vacuum
  keeping the file lean this should rarely fire, but the cushion
  matters when it does.
@webjunkie webjunkie merged commit 6fe6b4e into main May 4, 2026
10 checks passed
@webjunkie webjunkie deleted the fix/sqlite-incremental-vacuum branch May 4, 2026 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant