Skip to content

Add lightweight database migration system #44

@Aaronontheweb

Description

@Aaronontheweb

Problem

The skill-server currently uses CREATE TABLE IF NOT EXISTS / CREATE VIRTUAL TABLE IF NOT EXISTS for schema setup in DatabaseInitializer.cs. This works for initial creation but cannot handle schema changes to existing tables (e.g., altering FTS5 tokenizer settings, adding columns, changing constraints).

As the server evolves, we need a way to apply incremental schema changes safely.

Proposal

Add a simple numbered-migration system, following the pattern used in Memorizer's SchemaMigrator:

1. Schema version table

CREATE TABLE IF NOT EXISTS schema_version (
    version INT PRIMARY KEY,
    name TEXT NOT NULL,
    applied_at TEXT NOT NULL
);

2. Migration files

Numbered SQL files in a migrations/ directory (embedded or on disk):

migrations/
  001_initial_schema.sql
  002_fts5_porter_stemmer.sql
  ...

3. Migrator behavior (on startup)

  1. Create schema_version table if it doesn't exist
  2. Read which migrations have already been applied
  3. Apply any new migrations in order
  4. Record each applied migration with timestamp

4. Retrofit existing schema

Move the current CREATE TABLE IF NOT EXISTS block from DatabaseInitializer.cs into 001_initial_schema.sql. Existing databases with no schema_version table get migration 001 applied (idempotent due to IF NOT EXISTS), then future migrations run normally.

Why

This is a prerequisite for #43 (FTS5 Porter stemmer) and any future schema changes. Without it, changes to existing tables require manual intervention or risky drop/recreate logic.

Reference

  • Memorizer's implementation: SchemaMigrator.cs
  • SQLite-compatible (uses TEXT for timestamps instead of TIMESTAMPTZ)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions