logs_2.sqlite never reclaims freed pages: auto_vacuum=INCREMENTAL is set but never run, so the file grows monotonically despite working 10-day retention

[ignaciomontenegroc]

What version of the Codex App are you using (From “About Codex” dialog)?

1.2026.190.0 (MSIX OpenAI.ChatGPT-Desktop_1.2026.190.0_x64__2p2nqsd0c76g0)

What subscription do you have?

Pro $100

What platform is your computer?

Microsoft Windows NT 10.0.26100.0 x64 (Windows 11 Home Single Language)

What issue are you seeing?

~/.codex/logs_2.sqlite grows without bound on a normal desktop profile even though row retention is working correctly. The row count is at steady state (~10 days of rows), but the file on disk keeps climbing, because pages freed by the retention DELETE go to the SQLite freelist and are never returned to the OS.

The DB is created with auto_vacuum = 2 (INCREMENTAL), but nothing ever runs PRAGMA incremental_vacuum. run_logs_startup_maintenance() deletes rows older than the retention window and then runs PRAGMA wal_checkpoint(PASSIVE) — neither of which shrinks the main DB file. So retention bounds rows but not bytes.

Measured on this machine (Windows 11, Codex Desktop 1.2026.190.0), on an offline copy of the DB:

metric	value
logs_2.sqlite	579,276,800 bytes (552 MiB)
logs_2.sqlite-wal	4.6 MiB
page_count	141,425 (4096-byte pages)
freelist_count	37,128 — 26% of the file is dead space
auto_vacuum	2 (INCREMENTAL — never invoked)
rows in logs	253,043
SUM(estimated_bytes)	~267 MB of actual log content
ts span	1784402440 → 1785266499 (exactly 10.0 days — retention is working)

~267 MB of live content is occupying a 552 MiB file.

Growth rate. I compacted this same DB with VACUUM INTO on 2026-07-17: 1.38 GB → 270 MB, zero rows lost. Eleven days later it is back to 552 MiB — roughly 25 MB/day of permanent, unreclaimed growth, with the row count flat. Left alone it returns to >1 GB in about a month, every month, on a machine that is only running the app normally.

Level distribution over that same 10-day window (the volume amplifier, already tracked in #29674 / #31542 / #31111 — this issue is about the reclamation half):

level	rows	est. bytes
TRACE	136,468	130.7 MB
INFO	56,841	69.3 MB
DEBUG	54,048	63.8 MB
WARN	5,511	3.2 MB
ERROR	175	0.1 MB

TRACE is 54% of rows and 49% of logged bytes, at default settings with no debug flag enabled.

What steps can reproduce the bug?

1. Run Codex Desktop normally for several weeks on Windows — default config, no debug flag, RUST_LOG unset.
2. Wait until age-based retention is actively deleting (i.e. the oldest row is ~10 days old).
3. Copy ~/.codex/logs_2.sqlite, -wal and -shm to a scratch directory. Do not open the live DB — it is in WAL mode under a running app; inspect the copy only.
4. Inspect the copy:

import sqlite3
c = sqlite3.connect("logs_2.sqlite")   # the COPY
q = lambda s: c.execute(s).fetchall()
print("page_count  ", q("PRAGMA page_count"))
print("freelist    ", q("PRAGMA freelist_count"))
print("auto_vacuum ", q("PRAGMA auto_vacuum"))
print("rows        ", q("SELECT COUNT(*) FROM logs"))
print("live bytes  ", q("SELECT SUM(estimated_bytes) FROM logs"))
print("ts span     ", q("SELECT MIN(ts), MAX(ts) FROM logs"))
print("levels      ", q("SELECT level, COUNT(*) FROM logs GROUP BY level"))

5. Repeat over days/weeks. COUNT(*) and the ts span stay flat at the retention window, but page_count and freelist_count only ever go up. The file never shrinks, at any point in the app lifecycle — launch, quit, idle, or restart.

To see the reclaimable amount directly: VACUUM INTO 'compact.sqlite' on the copy. Here that produced a 270 MB file from a 552 MiB source with an identical row count.

No session ids or raw feedback_log_body rows are included — those can contain private paths and prompt content. All figures above are aggregates.

What is the expected behavior?

Retention should bound the log DB on disk, not just the row count. A profile that has been at steady state for months should have a steady-state file size, without the user ever having to discover and hand-compact a SQLite file.

Any one of these would fix it, in increasing order of effort:

1. Call PRAGMA incremental_vacuum after the retention DELETE in run_logs_startup_maintenance(). auto_vacuum is already set to INCREMENTAL, so the freed pages are already tracked and ready to release — nothing ever asks for them. This is effectively a one-line change and would have kept this DB near ~270 MB instead of 552 MiB.
2. Add a byte cap alongside the age cap, and run a full VACUUM when the file exceeds it — off the hot startup path, since a large-DB VACUUM at launch would just trade this bug for Desktop launch can fail when logs_2.sqlite grows large: app-server SQLite pool times out during startup #27741 / Current CLI startup can stall before the TUI prompt when logs_2.sqlite/WAL is large or busy #30517.
3. Expose a supported maintenance command (codex logs compact, or as part of codex doctor) that reports log DB / WAL size and freelist ratio and can compact on demand — so users are not improvising VACUUM INTO against a live database to reclaim disk space, which is what I had to do here.

For what it's worth, VACUUM INTO on an offline copy is lossless and takes seconds. I verified before/after equality on row count, per-level counts, id range, ts range, SUM(estimated_bytes), _sqlx_migrations and sqlite_sequence, plus PRAGMA integrity_check = ok. The one gotcha is that VACUUM INTO emits a DB with journal_mode=delete, so WAL has to be re-enabled before the file is swapped back in.

Additional information

Scope. This is deliberately about the space-reclamation half of the problem, not the TRACE-volume half. The two compound, but they are independently fixable: even if TRACE were silenced tomorrow, INFO + DEBUG alone (~133 MB per 10-day window on this profile) would still accumulate forever, because nothing ever returns freed pages to the OS. Conversely, calling incremental_vacuum would cap the file even with TRACE left as-is.

Related / overlapping issues (I searched before filing):

• Desktop should bound and compact logs_2.sqlite before it becomes recovery pressure #30431 — umbrella: "bound and compact logs_2.sqlite". That issue asks for the policy (byte caps, a compact command); this one names the specific mechanism — auto_vacuum=INCREMENTAL set but incremental_vacuum never called — and the minimal fix. Happy to have this folded in as a comment there if maintainers prefer.
• state_5.sqlite and logs_1.sqlite grow unboundedly on long-running installs, causing write lock contention and app restart loop #16270 — the same unbounded-growth shape on the older logs_1.sqlite / state_5.sqlite, closed as completed; the reclamation behaviour appears to have regressed or was never applied to logs_2.sqlite.
• TRACE volume: Excessive TRACE logging causes logs_2.sqlite/WAL growth and SSD write pressure #29674, logs_2.sqlite TRACE churn: root cause (payload callsites bypass the sink filter) and minimal fix #31542, Codex desktop writes high-frequency TRACE logs to ~/.codex/logs_2.sqlite despite RUST_LOG=warn #31111, Codex App writes high-volume TRACE logs to logs_2.sqlite despite RUST_LOG=warn #30236, High-frequency TRACE rows are still written to logs_2.sqlite with RUST_LOG=warn #30780.
• Large-DB symptoms: logs_2.sqlite-wal grows without bound into tens of GB #28997 (WAL growth), Desktop launch can fail when logs_2.sqlite grows large: app-server SQLite pool times out during startup #27741 (app-server SQLite pool times out at startup), Current CLI startup can stall before the TUI prompt when logs_2.sqlite/WAL is large or busy #30517 (CLI stalls before the TUI prompt), Bug: CLI crashes with SIGTRAP (trace trap) when logs_2.sqlite exceeds ~200MB #29237 (SIGTRAP above ~200 MB).

Environment

• Codex Desktop 1.2026.190.0, MSIX OpenAI.ChatGPT-Desktop_1.2026.190.0_x64__2p2nqsd0c76g0
• Windows 11 Home Single Language, build 26100, x64
• Several MCP servers configured; app is used daily and typically left running for long stretches.

Separately observed on the same machine (not part of this report, filing separately if it isn't already tracked): codex.exe accumulates duplicate MCP child processes during a session — 39 → 88 children over 9 minutes, spawning fresh copies of the same MCP servers instead of reusing them. On this profile that is a larger performance drain than the log DB.

[github-actions]

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• [macOS Desktop 26.721.41059] Persistent SQLite TRACE churn remains in bundled 0.146.0-alpha.3.1 #35308

Powered by Codex Action

[Totoro-debug]

Additional Windows Codex Desktop reproduction that connects the unreclaimed database pages to a user-visible startup/new-thread performance regression:

• Codex Desktop package: OpenAI.Codex_26.707.3563.0_x64
• Platform: Windows x64
• User-visible symptom: opening Codex or starting a new conversation intermittently stalled for several seconds; normal CPU, memory, disk capacity, and network availability did not explain it.
• Before recovery, logs_2.sqlite was 2,682,204,160 bytes (~2.68 GB).
• Read-only SQLite metrics: page_count=654835, page_size=4096, freelist_count=265739 (>1 GB of unreclaimed pages), auto_vacuum=2, and quick_check=ok.
• The database contained 132,936 rows and about 1.47 GB of log bodies; codex_http_client::transport accounted for about 1.36 GB.
• A thread-specific query used idx_logs_thread_id_ts, so this did not appear to be corruption or an unavoidable full-table scan.

After the app was fully stopped and only the logs_2.sqlite* diagnostic files were rotated, Codex recreated logs_2.sqlite at about 5.8 MB and the startup/new-conversation stalling stopped. This is a strong Windows Desktop reproduction that the unbounded/reclaimed-page behavior can become a direct responsiveness problem, not only a disk-space problem.

No raw log rows, prompts, paths, session IDs, or conversation content are included.

[ariccio]

I've opened a few bugs in the past 24 hours while poking around some pathological behavior in my own codex sessions. I had codex look into them, and it produced this:

While investigating unusual disk usage from an exceptionally large and long-running Codex session lineage, I asked Codex to separate rollout-file storage from SQLite reclamation concerns. It produced the following:

One architecture/safety note from a separate macOS session-storage audit—not an additional logs_2.sqlite reproduction:

logs_2.sqlite, logs_2.sqlite-wal, and logs_2.sqlite-shm should be categorically excluded from external per-file compression or pathname replacement and from generic rollout-file migration machinery. Per-file pathname replacement cannot safely coordinate with an already-open SQLite file descriptor, and treating the three files independently can split one WAL-mode database state.

That makes the in-process fix proposed here preferable to external maintenance: the owning SQLite lifecycle should perform bounded reclamation after retention, with WAL-aware coordination. I would add regression coverage that:

• reaches a steady live-row/byte set while DELETE creates a material freelist;
• runs bounded incremental_vacuum work outside the critical startup/new-thread path;
• handles BUSY by backing off rather than blocking startup or forcing replacement;
• covers active WAL readers/writers, crash/restart, and low free space;
• proves row/content invariants and integrity_check remain valid; and
• asserts the main DB file plateaus or shrinks over repeated retention cycles.

A supported diagnostic/maintenance command should operate through SQLite, report DB/WAL/SHM sizes and freelist ratio, and refuse unsafe replacement while Codex owns the database. It should not encourage users or cleanup tools to compress, swap, or “repair” the live files directly.