Skip to content

fix(peewee): enable WAL journal mode for SQLite#132

Merged
ErikBjare merged 2 commits intomasterfrom
fix/peewee-wal-mode
Apr 13, 2026
Merged

fix(peewee): enable WAL journal mode for SQLite#132
ErikBjare merged 2 commits intomasterfrom
fix/peewee-wal-mode

Conversation

@ErikBjare
Copy link
Copy Markdown
Member

Summary

  • Enable WAL (Write-Ahead Logging) journal mode for PeeweeStorage's SQLite database
  • SqliteStorage already had WAL enabled, but PeeweeStorage (the default backend) did not

Problem

Without WAL mode, SQLite uses DELETE journal mode which requires exclusive locks for writes. When aw-server's threaded Flask handles concurrent heartbeats from multiple watchers, this causes frequent sqlite3.OperationalError: database is locked errors.

On one system, this resulted in:

  • 3,984 "database is locked" errors in a 21-day session
  • 1,992 HTTP 500 responses to watcher heartbeats
  • Watchers retrying failed heartbeats, creating a cascade of load
  • 2060 CPU-minutes accumulated by the aw-server process

Fix

WAL mode allows concurrent readers and a single writer without blocking, dramatically reducing contention. This is a one-line change to pass pragmas={"journal_mode": "wal"} to SqliteExtDatabase.init().

Test plan

  • Existing tests pass
  • Verify WAL mode is active: sqlite3 <db-path> "PRAGMA journal_mode;" should return wal
  • Monitor for "database is locked" errors with multiple active watchers

Related: ActivityWatch/aw-server#151

PeeweeStorage was using SQLite's default DELETE journal mode, while
SqliteStorage already had WAL enabled. Without WAL, concurrent read/write
access from threaded Flask causes frequent "database is locked" errors,
leading to 500 responses and watcher retry storms.
@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Apr 13, 2026

Greptile Summary

This PR enables WAL journal mode for PeeweeStorage's SQLite database by passing pragmas={"journal_mode": "wal", "wal_autocheckpoint": 100} to SqliteExtDatabase.init(), bringing it in line with SqliteStorage which already set WAL via PRAGMA journal_mode=WAL. The fix directly addresses the database is locked errors under concurrent Flask-threaded writes.

Confidence Score: 5/5

Safe to merge — minimal, well-targeted change with no correctness risks.

The only finding is a P2 style suggestion: auto_migrate opens its own connection and won't inherit the per-connection wal_autocheckpoint setting (though journal_mode=WAL persists at the file level, so correctness is unaffected). No P0/P1 issues exist.

No files require special attention.

Important Files Changed

Filename Overview
aw_datastore/storages/peewee.py Adds journal_mode=WAL and wal_autocheckpoint=100 pragmas to the SqliteExtDatabase.init() call, matching the WAL configuration already present in SqliteStorage; one minor note that auto_migrate's separate connection doesn't inherit the per-connection wal_autocheckpoint setting.

Sequence Diagram

sequenceDiagram
    participant Flask Thread 1
    participant Flask Thread 2
    participant PeeweeStorage
    participant SQLite (WAL)

    Flask Thread 1->>PeeweeStorage: heartbeat (insert event)
    Flask Thread 2->>PeeweeStorage: heartbeat (insert event)
    PeeweeStorage->>SQLite (WAL): BEGIN WRITE (Thread 1)
    PeeweeStorage->>SQLite (WAL): BEGIN READ (Thread 2)
    Note over SQLite (WAL): WAL allows concurrent reader + writer
    SQLite (WAL)-->>PeeweeStorage: OK (Thread 2 reads)
    SQLite (WAL)-->>PeeweeStorage: OK (Thread 1 writes)
    Note over SQLite (WAL): Checkpoint every 100 pages
Loading

Comments Outside Diff (1)

  1. aw_datastore/storages/peewee.py, line 47-60 (link)

    P2 auto_migrate connection bypasses wal_autocheckpoint

    auto_migrate opens its own SqliteExtDatabase(path) without any pragmas. journal_mode=WAL is a persistent database property and will still be in effect, but wal_autocheckpoint is a per-connection setting — it resets to the SQLite default of 1000 during every migration run. If checkpointing behaviour during migration matters, the pragmas should be passed here too.

Reviews (1): Last reviewed commit: "fix(peewee): enable WAL journal mode for..." | Re-trigger Greptile

SQLite's default of 1000 pages is fine; setting 100 would cause more
frequent checkpointing and unnecessary IO.
@ErikBjare ErikBjare merged commit 40fccd0 into master Apr 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant