Patch release. If you are running v1.0.0, upgrade — it ships an archive that
grows without bound.
The fix
v1.0.0's replay bar asked the backend for archive coverage every 5 seconds, and
each request scanned the whole archive. That scan takes longer than the gap
between polls, so a read transaction was always open — and SQLite can only
checkpoint its write-ahead log past the oldest live reader. The recorder never
stops writing, so the log grew forever.
It compounds, which is why it runs away rather than settling: every page read has
to search the log for that page's newest version, so a bigger log makes the scan
slower, which holds the reader open longer, which grows the log faster.
On the development box that produced a 49.6 GB write-ahead log against 15.2 GB
of actual data — 71 GB of disk for 15 GB of archive. A checkpoint reclaimed
48.6 GB of it, so 98% was redundant page versions that could never be written
back. Left recording on a smaller disk, it fills it.
Coverage is now cached behind a lock, so the scan runs at most once every two
minutes and never twice at once, and the log is bounded by journal_size_limit
so it is truncated back whatever the read pattern does.
Replaying the exact workload that caused it, against a real 16 GB archive:
| v1.0.0 | v1.0.1 | |
|---|---|---|
| write-ahead log | grew to 49.6 GB | 0 MB, flat |
| memory | +4-5 GB/min, peaked at 83 GB | flat, ~660 MB |
| coverage request | 73 s, every call | 14.8 s once, then cached |
Upgrading
Nothing to migrate. Pull and restart; the oversized log from a v1.0.0 run is
checkpointed back into the database on the next clean start. If you want the
disk back immediately, stop the API and run:
sqlite3 data/history.db "PRAGMA wal_checkpoint(TRUNCATE);"That took 37.7 s against a 49.6 GB log here. It is lossless — the log's contents
are written into the database, which is what a clean shutdown does anyway.
Everything else
Unchanged from v1.0.0. 1,721 backend tests passing. AGPL-3.0.