Skip to content
This repository

ORM-agnostic schema upgrade tools in Python

branch: master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 examples
Octocat-spinner-32 schemup
Octocat-spinner-32 README.md
Octocat-spinner-32 setup.py
README.md

Some Design Notes

Versioning:

  • ORM:

    • Each model has a __version__ attribute
    • When incrementing the version, supply an upgrader function
    • Global dict of upgraders: (tablename, fromVersion, toVersion) -> function
    • Upgrader makes direct DDL calls to alter its table / data.
  • DB:

    • "migrations" table with table name, version, is_current, schema dump (text)

On webserver startup:

  • For each model:

    • Check __version__ against is_current version in DB migrations table
    • If different, complain and bail out -- need to run a migration
    • If same, generate new schema dump (via DB) and compare that to schema in is_current.
    • If different, complain and bail out -- DB was changed without corresponding migration, deployment is considered broken.

Running a migration:

  • Clone DB as DB_migration_temp
  • For each model:

    • Get current DB version from migrations table
    • Get current code version from __version__
    • Attempt to find an upgrader path to get there
    • Run each upgrader in turn
    • When finished, generate new schema dump and compare to dump in migrations table
    • If different, upgrade is broken. Bail out.
  • When finished:

    • Move DB to DB_old_datetime
    • Move DB_migration_temp to DB

Creating a migration:

  • Bump __version__ in ORM definition
  • Write upgrader
  • Run migration "create" tool

    • Checks current schema against expected, as above
    • Runs upgrader on current DB table
    • Creates new schema dump and writes migration table entry
  • Manually verify, manually roll back if broken.

Adding a table:

  • As for creating a migration, above
  • Upgrader does a "CREATE TABLE"
  • Non-existing table is implicitly version 0

Issues:

  • How do we handle multiple developers writing version upgrades at once?

    • Merge tree, like git? (terrifying)
    • Don't Do That
  • Upgraders need to be run in order

    • e.g. adding a foreign key to a table being created in the same migration
    • Probably impossible to solve (databases are cyclic graphs, topological sort not enough)
    • Should be possible to manually specify an ordering between specific upgraders
    • But not required all the time
Something went wrong with that request. Please try again.