Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(db) new migrations framework #3802

merged 30 commits into from Sep 26, 2018


None yet
3 participants
Copy link

thibaultcha commented Sep 24, 2018

Implements a new migrations framework with several goals in mind. Among the improvements in this new framework:

  • First and foremost, support for the new DAO (kong.db module).
  • Ability to run migrations concurrently from any number of nodes, thanks to leader-election through the cluster_mutex() utility.
  • Ability to implement 2 step migrations (non-breaking, then destructive steps). This allows for smoother upgrades between major versions, as it grants a period of time for a database schema to be compatible with both running versions (e.g. "blue" and "green"). Currently, "translation functions" are implemented as a PoC, but the nature of 1.0 (non breaking) does not require us to complete this work at the current time. Locking writes to tables has also been discussed and is NYI as well.
  • PostgreSQL migrations are executed within transactions.
  • Scaling up and down the Cassandra cluster at runtime does not prevent starting new Kong nodes anymore.
  • Made Cassandra migrations more resilient and re-entrant in case of unexpected failures/retries.
  • Implemented a kong migrations list command. This command also returns separate exit codes for various states, allowing for better script-ability.
  • Ability to customize database timeouts for migrations with CLI arguments.
  • Improved CLI logging during migrations.
  • Migrations do not expose the underlying DAO anymore, but only the connector object, therefore, avoiding issues related to schema validations in migrations.
  • Migrations are consolidated (squashed) into a single one for each subsystem, as per the state of the schema in 0.14.

Subsequent work

In a subsequent PR, the following additions should be contributed:

  • A migration from 0.14 (and the old migrations framework) to the new framework.
  • Any missing migrations that was contributed between the release of 0.14 and the present day, in the next branch.

Additionally, documentation will be updated with the new CLI commands and new recommended upgrade path.

New workflow:

Starting a fresh node:

$ kong migrations bootstrap
$ kong start

Listing the state of the schema and its migrations:

$ kong migrations list

Upgrading from a previous version:

$ kong migrations up
 # Start new nodes nodes
$ kong start
# Decommission previous version nodes
# Finish pending migrations
$ kong migrations finish


# Run migrations *and* start new nodes once done;
# can be called from any number of nodes
$ kong start --run-migrations
# Decommission previous version nodes
# Finish pending migrations 
$ kong migrations finish

thibaultcha and others added some commits Aug 11, 2018

feat(db) groundwork for new migrations framework
* entirely built into new DAO (`kong.db`)
* only Cassandra support for now
* better design for schema_meta table (storing past executed migrations)
* concept of "subsystem": core, plugins, and "EE" are all possible
  subsystem, each with their own set of migrations
* migrations are in separate files, indexed in `kong.db.migrations` for
  the core ones
* migrations are usually split in two steps: non-destructive, and
  destructive steps
* new CLI commands: check, bootstrap, finish (+ up and reset as
* migrations run within cluster_mutex() (i.e. leader-election), which
  means they can be launched simultaneously from any number of nodes, and
  not a single one anymore
* migrations can be "executed" or "pending", when their second
  (destructive) step is still pending
* hypothetical support for "translation functions", for breaking changes
  on-the-fly without downtime or incompatibility time windows
* squashed core Cassandra migrations into one
feat(db) framework to provision subsystems migrations
* core migrations moved under `kong.db.migrations.core`
* to load other subsystems migrations, we retrieve the path to
  `kong/db/migrations` on the file system, and iterate over the
* plugins migrations are retrieved from `kong.plugins.<name>.migrations`
* each subsystem (from core or plugins) maintains:
  1. an index of migrations in an `init.lua` file at the base of their
     migrations namespace
  2. a file per migration, with the `postgres` and `cassandra` root
feat(schema) support for fields with 'function' type
Also add tests for the Entity schema subclass.
feat(db) validate migrations with schema
Produces errors such as:

$ kong migrations up
Error: migration 'kong.db.migrations.core.000-base' of 'core' subsystem
is invalid: schema violation (postgres.up: required field missing)
feat(cmd) polishing new 'migrations' command
* revisited stdout/stderr output to be closer to
  previous migrations framework
* early exit from cluster mutexes when possible
* new --db-timeout and --lock-timeout arguments
* no prompt when not a tty
* prefix error messages from kong.db
* revisited schema consensus waiting
feat(db) remove 'check' in favor of a better 'list'
* list all executed migrations
* custom exit codes on actionable statuses
* --quiet flag
feat(*) convert plugins migrations to new DAO
* squash of all plugins migrations as of 0.14
* Cassandra only
feat(db) better CLI logging during migrations
* new infos() method from connector to retrieve user-facing info + DB
* move `KONG_TEST_PG_*` env vars to be global since unit tests also
  connect to PostgreSQL, but now actually make a query (for
* distinguish pending and executed migrations in CLI logs
feat(cli) make `migrations reset` return 0 only when it actually rese…
…ts the db

This allows tests to tell apart if the command actually performed a reset
or not.

@thibaultcha thibaultcha merged commit 16f81cf into next Sep 26, 2018

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
continuous-integration/travis-ci/push The Travis CI build passed

@thibaultcha thibaultcha deleted the feat/new-migrations branch Sep 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.