My free project's disk filled up with WAL, database won't start, and pause won't stick — ticket SU-442291 #48904
Replies: 1 comment
|
You have already discovered the core problem, so let me at least confirm it so you stop burning hours on workarounds: nothing you can do from the dashboard or API will fix this. Dropping a replication slot, freeing WAL, or nudging the volume all require either a running Postgres or platform-level shell access, and you have neither while the disk is full. The pause loop you are seeing is also expected behavior, the platform verifies project health before completing a pause and flips it back when the health check fails, which it always will here. Your diagnosis is very likely right. A WAL folder larger than the database itself on a single-node free project is the classic stuck-slot signature, almost always a leftover logical slot from an old realtime/bootstrap process that stopped consuming. Postgres cannot recycle WAL past the slot's restart_lsn, so the folder only grows. Given that, the ticket is genuinely the only path. Things that help it move faster:
One small correction to your last point: the 500 MB read-only guard measures the database size, not disk usage, so WAL growth on top of an 806 MB database never triggers it. Whether that should change is a fair product question, but it explains why the guard stayed silent. Also, do not delete anything in the project's storage from the dashboard while you wait, it will not free database volume anyway and you do not want to compound the incident. |
Uh oh!
There was an error while loading. Please reload this page.
Hey folks — hoping someone from the Supabase team can grab this, because I've hit a wall I literally cannot get out of on my own.
My project (
ptfozazzfbtsrpuavdmb, free plan) went down two days ago. The disk is 2 GB and it's completely full — but here's the kicker: the WAL folder alone is 992 MB, which is bigger than my entire database (806 MB). From what I've learned, that means a dead replication slot is holding onto all that WAL so it never gets cleaned up.The database won't start because there's no room left. The logs just repeat this forever, every 5 seconds:
The frustrating part: recovery actually finishes fine (redo completes in 0.00s), so my data is okay. It just needs room for one 16 MB file and it can't get it.
What I've already tried:
pg_drop_replication_slotfix myself, ever.So I'm stuck in a loop: the fix is a 30-second command, but the database has to be running to accept the command, and it can't run because of the very thing the command would fix.
I have support ticket SU-442291 filed with all the logs. If anyone from Supabase sees this, I'd be beyond grateful if you could drop the stuck slot server-side (or give the volume a temporary nudge of space so it can boot).
One more thing worth flagging for the team: the docs say free projects go read-only at 500 MB of database size. Mine sailed right past that to 806 MB and never got put in read-only mode — if that guard had kicked in, the WAL never could have filled the disk in the first place.
Thanks for reading. Two days down and counting — any help means a lot. 🙏
All reactions