Skip to content

Refactor: Add schema versioning and migrations for the SQLite runtime store #142

@techmore

Description

@techmore

Title: Add schema versioning and migrations for the SQLite runtime store

Type: refactor
Severity: high
Area: runtime database / persistence architecture

Description:
The SQLite runtime store now underpins jobs, logs, reports, history, customer history, maintenance status, and export. Despite that central role, the store has no explicit schema versioning or migration framework. It initializes tables opportunistically with CREATE TABLE IF NOT EXISTS, but there is no PRAGMA user_version, migration registry, or startup compatibility check. Future schema changes will be forced into ad hoc conditional SQL and risk silent incompatibilities for installed databases.

Evidence:

  • /Users/techmore/projects/NmapUI/nmapui/runtime_db.py defines SCHEMA_STATEMENTS and runs them in initialize() but does not track a runtime DB schema version
  • the runtime DB is now used widely across /Users/techmore/projects/NmapUI/nmapui/handlers/routes.py, /Users/techmore/projects/NmapUI/nmapui/handlers/connections.py, and report/history flows, which raises the cost of incompatible schema drift
  • migration/export behavior in /Users/techmore/projects/NmapUI/build.sh and /Users/techmore/projects/NmapUI/scripts/backfill_runtime_store.py assumes the DB can be copied forward without explicit version negotiation

Proposed Fix:
Introduce versioned runtime DB migrations with explicit startup enforcement.

Implementation Notes:

  • Use PRAGMA user_version or a dedicated metadata table as the single schema version source
  • Add ordered migration functions and tests for upgrade paths
  • Fail clearly on unsupported future versions instead of implicitly operating on unknown schemas
  • Add an admin command to inspect and repair runtime DB versions

Related Issues:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions