Schema Migrations

Schema Migrations

What this page covers: evolving a backend's schema with the optional SchemaAwareStorage capability — register(...).migrate(), the forward-only model, the _schema_migrations ledger, and the three backend base classes (SqlMigration, MongoMigration, LocalFileMigration) plus getNativeClient for full control. It also explains how migrations relate to the automatic table creation SQL does on its own.

---

The 30-second version

Migrations are a capability, expressed as the interface schema.SchemaAwareStorage. Register your migrations and call migrate():

import br.com.finalcraft.everydatabase.*;
import br.com.finalcraft.everydatabase.schema.SchemaAwareStorage;
import br.com.finalcraft.everydatabase.modules.sql.SqlMigration;

// A migration: a version, a description, and what to do.
public final class V001_CreatePlayers extends SqlMigration {
    public static final V001_CreatePlayers INSTANCE = new V001_CreatePlayers();
    private V001_CreatePlayers() {}

    public String version()     { return "001"; }
    public String description() { return "Create players table"; }
    public String upScript() {
        return "CREATE TABLE IF NOT EXISTS players ("
             + "  uuid VARCHAR(36) NOT NULL,"
             + "  name VARCHAR(64) NOT NULL,"
             + "  PRIMARY KEY (uuid))";
    }
}

// Register + apply (idempotent: already-applied migrations are skipped).
storage.init().join();
if (storage instanceof SchemaAwareStorage schema) {
    schema.register(V001_CreatePlayers.INSTANCE, V002_AddLastSeen.INSTANCE)
          .migrate()
          .join();
}

migrate() applies the pending migrations in version order, records each one in a reserved _schema_migrations ledger, and skips anything already applied. Running it again is a no-op.

📌 Note — migrate() returns a CompletableFuture<Void>; the example uses .join() for brevity. If a migration fails, the future completes exceptionally and the sequence aborts — migrations after the failing one are not applied. See The Async API.

---

Capabilities are interfaces, not flags

There's no storage.supportsMigrations() boolean. A backend that can track and apply migrations implements SchemaAwareStorage; narrow to it with instanceof before calling register/migrate:

if (storage instanceof SchemaAwareStorage schema) {
    schema.register(V001.INSTANCE, V002.INSTANCE).migrate().join();
}

When you already hold a concrete type from a typed factory (e.g. Storages.createSQL(...) returns a SqlStorage, which is a SchemaAwareStorage), you can skip the check:

SqlStorage sql = Storages.createSQL(new SqlConfig("jdbc:mariadb://localhost/mc", "root", "pass"));
sql.init().join();
sql.register(V001_CreatePlayers.INSTANCE).migrate().join();

The interface surface:

public interface SchemaAwareStorage extends Storage {
    SchemaAwareStorage register(List<Migration> migrations);
    default SchemaAwareStorage register(Migration... migrations);   // varargs shorthand
    CompletableFuture<SchemaVersion>     currentVersion();          // latest applied, or none()
    CompletableFuture<List<Migration>>   pending();                 // registered-but-not-applied, in order
    CompletableFuture<Void>              migrate();                  // apply all pending, in order
}

📌 Note — register(...) must be called before migrate(), returns this for chaining, and accumulates across calls. Migrations are sorted by version() automatically, so registration order doesn't matter.

---

Forward-only, by design

There is intentionally no downScript() / rollback migration. Rollback-by-reverse-migration is an anti-pattern in production; if you need to undo something, write a compensating forward migration (V005_DropDeprecatedColumn) instead. Each migration applies exactly once and is recorded.

A Migration is identified by a lexicographically sortable version() string — "001", "002", or dates like "2024-01-15". The string must be unique per storage; ordering is natural string order, so zero-pad numeric versions ("010", not "10") to keep them sorting correctly.

---

The _schema_migrations ledger

Each backend records which versions it has applied in its own reserved location:

Backend	Where applied versions are tracked
MySQL / MariaDB · PostgreSQL · H2	a _schema_migrations table
MongoDB	a _schema_migrations collection
LocalFile	a metadata file

This is what makes migrate() idempotent and safe to call on every startup: it consults the ledger, applies only the pending entries, and records each success. Query the state without applying anything:

SchemaVersion v = schema.currentVersion().join();   // e.g. SchemaVersion{002}, or none() == "0"
List<Migration> todo = schema.pending().join();     // empty == up to date

⚠️ Gotcha — if a migration throws, the runner wraps the exception, aborts the sequence, and the returned future completes exceptionally. Migrations after the failing one are not applied; ones before it stay recorded as applied. Fix the cause and re-run migrate() — it resumes from the first unapplied version.

---

Writing a migration: the three base classes

Implement the right base class for your backend; each unwraps the native client for you. For full control (multi-statement SQL, mixed operations, your own Mongo session), implement Migration.execute(MigrationContext) directly and pull the native client yourself.

SQL — extend SqlMigration, implement upScript()

One DDL/DML statement, no trailing semicolon. The base class opens a Statement on the transaction's Connection and runs it:

public final class V002_AddLastSeen extends SqlMigration {
    public static final V002_AddLastSeen INSTANCE = new V002_AddLastSeen();
    private V002_AddLastSeen() {}

    public String version()     { return "002"; }
    public String description() { return "Add last_seen column"; }
    public String upScript() {
        return "ALTER TABLE players ADD COLUMN last_seen BIGINT";
    }
}

💡 Tip — for multiple statements or procedural logic, override execute(MigrationContext) and call context.getNativeClient(Connection.class) yourself rather than cramming several statements into one upScript().

MongoDB — extend MongoMigration, implement executeOnDatabase(MongoDatabase)

Mongo migrations are always code-based (no script concept) — you write Java against the MongoDatabase:

import com.mongodb.client.MongoDatabase;
import com.mongodb.client.model.Indexes;

public final class V002_AddNameIndex extends MongoMigration {
    public static final V002_AddNameIndex INSTANCE = new V002_AddNameIndex();
    private V002_AddNameIndex() {}

    public String version()     { return "002"; }
    public String description() { return "Add ascending index on name in player_data"; }

    protected void executeOnDatabase(MongoDatabase db) {
        db.getCollection("player_data")
          .createIndex(Indexes.ascending(COL_DATA + ".name"));   // COL_DATA == "storage_data"
    }
}

MongoMigration exposes the document field names the repository uses, so you can reach into stored blobs without the package-private class: COL_KEY ("storage_key", the serialized key) and COL_DATA ("storage_data", the JSON entity blob).

⚠️ Gotcha — executeOnDatabase runs outside a client session, and the runner does not wrap migrations in a transaction. Multi-document transactions need a replica set anyway. If you need transactional safety inside a migration, override execute(MigrationContext) and start your own session.

LocalFile — extend LocalFileMigration, implement executeOnStorage(LocalFileStorage)

You get the LocalFileStorage itself — use storage.repository(descriptor) for high-level CRUD, or storage.baseDirectory() for raw file work:

import java.nio.file.*;

public final class V001_FixCorruptedFiles extends LocalFileMigration {
    public static final V001_FixCorruptedFiles INSTANCE = new V001_FixCorruptedFiles();
    private V001_FixCorruptedFiles() {}

    public String version()     { return "001"; }
    public String description() { return "Delete files with corrupted JSON"; }

    protected void executeOnStorage(LocalFileStorage storage) throws Exception {
        Path dir = storage.baseDirectory().resolve("playerdata");
        Files.walk(dir, 1)
             .filter(p -> p.toString().endsWith(".json") && isCorrupted(p))
             .forEach(p -> { try { Files.delete(p); } catch (IOException ignored) {} });
    }
}

---

Full control: MigrationContext.getNativeClient

Every base class is sugar over Migration.execute(MigrationContext). Implement it directly and pull the typed native handle when you need more than the base class offers:

<T> T getNativeClient(Class<T> type);

Connection    conn = context.getNativeClient(Connection.class);       // SQL
MongoDatabase db   = context.getNativeClient(MongoDatabase.class);     // Mongo
LocalFileStorage s = context.getNativeClient(LocalFileStorage.class);  // LocalFile

⚠️ Gotcha — requesting a type the backend doesn't provide throws IllegalArgumentException, which the runner surfaces as a migration failure. Ask each backend for its native type only.

---

Migrations vs. automatic table creation (SQL)

These are complementary, not competing. SqlStorage auto-creates the entity table on the first repository(descriptor) call (createTableIfAbsent — idempotent, also reconciles declared indexes), so for a brand-new collection you often need no migration at all. Migrations are for the changes auto-create can't infer: backfilling data, renaming columns, adding non-index constraints, custom DDL, data fixes.

🧭 Decision — let auto-create handle the initial table + declared indexes; reach for a migration when you must transform existing data or apply schema changes the descriptor doesn't express. The two run independently — migrate() doesn't replace auto-create, and auto-create doesn't record anything in _schema_migrations.

📌 Note — In-Memory does not implement SchemaAwareStorage (there's no schema to migrate). SQL (all dialects), Mongo, and LocalFile do. See Choosing a Backend.

---

See also

• The Async API — the CompletableFuture<Void> migrate() returns and how failures surface.
• Defining Entities — descriptors and the auto-created tables migrations complement.
• Indexing & Queries — declared indexes are reconciled by auto-create, not migrations.
• Transactions — the other opt-in capability checked via instanceof.
• Choosing a Backend — which backends are SchemaAwareStorage.
• MongoDB — the replica-set note that also applies to transactional Mongo migrations.
• Moving Data Between Backends — StorageTransfer can apply target migrations during a copy.