We've documented our learned best practices for applying schema changes without downtime in the post PostgreSQL at Scale: Database Schema Changes Without Downtime on the Braintree Product and Technology Blog. Many of the approaches we take and choices we've made are explained in much greater depth there than in this README.
Internally we apply those best practices to our Rails applications through this gem which updates ActiveRecord migrations to clearly delineate safe and unsafe DDL as well as provide safe alternatives where possible.
Some projects attempt to hide complexity by having code determine the intent and magically do the right series of operations. But we (and by extension this gem) take the approach that it's better to understand exactly what the database is doing so that (particularly long running) operations are not a surprise during your deploy cycle.
Add this line to your application's Gemfile:
And then execute:
Or install it yourself as:
$ gem install pg_ha_migrations
Because we require that "Rollback strategies do not involve reverting the database schema to its previous version", PgHaMigrations does not support ActiveRecord's automatic migration rollback capability.
Instead we write all of our migrations with only an
def up method like:
def up safe_add_column :table, :column end
and never use
def change. We believe that this is the only safe approach in production environments. For development environments we iterate by recreating the database from scratch every time we make a change.
There are two major classes of concerns we try to handle in the API:
- Database safety (e.g., long-held locks)
- Application safety (e.g., dropping columns the app uses)
We rename migration methods with prefixes denoting their safety level:
safe_*: These methods check for both application and database safety concerns prefer concurrent operations where available, set low lock timeouts where appropriate, and decompose operations into multiple safe steps.
unsafe_*: These methods are generally a direct dispatch to the native ActiveRecord migration method.
Calling the original migration methods without a prefix will raise an error.
The API is designed to be explicit yet remain flexible. There may be situations where invoking the
unsafe_* method is preferred (or the only option available for definitionally unsafe operations).
unsafe_* methods were historically (through 1.0) pure wrappers for invoking the native ActiveRecord migration method, there is a class of problems that we can't handle easily without breaking that design rule a bit. For example, dropping a column is unsafe from an application perspective, so we make the application safety concerns explicit by using an
unsafe_ prefix. Using
unsafe_remove_column calls out the need to audit the application to confirm the migration won't break the application. Because there are no safe alternatives we don't define a
safe_remove_column analogue. However there are still conditions we'd like to assert before dropping a column. For example, dropping an unused column that's used in one or more indexes may be safe from an application perspective, but the cascading drop of the index won't use a
CONCURRENT operation to drop the dependent indexes and is therefore unsafe from a database perspective.
unsafe_* migration methods support checks of this type you can bypass the checks by passing an
:allow_dependent_objects key in the method's
options hash containing an array of dependent object types you'd like to allow. Until 2.0 none of these checks will run by default, but you can opt-in by setting
config.check_for_dependent_objects = true in your configuration initializer.
Running multiple DDL statements inside a transaction acquires exclusive locks on all of the modified objects. For that reason, this gem disables DDL transactions by default. You can change this by resetting
ActiveRecord::Migration.disable_ddl_transaction in your application.
The following functionality is currently unsupported:
Safely creates a new table.
safe_create_table :table do |t| t.type :column end
Safely create a new enum without values.
Or, safely create the enum with values.
safe_create_enum_type :enum, ["value1", "value2"]
Safely add a new enum value.
safe_add_enum_value :enum, "value"
Safely add a column.
safe_add_column :table, :column, :type
Unsafely add a column, but do so with a lock that is safely acquired.
unsafe_add_column :table, :column, :type
Safely change the default value for a column.
safe_change_column_default :table, :column, "value"
Safely make the column nullable.
safe_make_column_nullable :table, :column
Unsafely make a column not nullable.
unsafe_make_column_not_nullable :table, :column
Add an index concurrently.
safe_add_concurrent_index :table, :column
Add a composite btree index.
safe_add_concurrent_index :table, [:column1, :column2], name: "index_name", using: :btree
Safely remove an index. Migrations that contain this statement must also include
safe_remove_concurrent_index :table, :name => :index_name
Safely acquire a lock for a table.
safely_acquire_lock_for_table(:table) do ... end
Adjust lock timeout.
adjust_lock_timeout(seconds) do ... end
Adjust statement timeout.
adjust_statement_timeout(seconds) do ... end
Set maintenance work mem.
The gem can be configured in an initializer.
PgHaMigrations.configure do |config| # ... end
disable_default_migration_methods: If true, the default implementations of DDL changes in
ActiveRecord::Migrationand the PostgreSQL adapter will be overridden by implementations that raise a
check_for_dependent_objects: If true, some
unsafe_*migration methods will raise a
PgHaMigrations::UnsafeMigrationErrorif any dependent objects exist. Default:
Use this to check for blocking transactions before migrating.
$ bundle exec rake pg_ha_migrations:check_blocking_database_transactions
This rake task expects that you already have a connection open to your database. We suggest that you add another rake task to open the connection and then add that as a prerequisite for
namespace :db do desc "Establish a database connection" task :establish_connection do ActiveRecord::Base.establish_connection end end Rake::Task["pg_ha_migrations:check_blocking_database_transactions"].enhance ["db:establish_connection"]
After checking out the repo, run
bin/setup to install dependencies and start a postgres docker container. Then, run
rake spec to run the tests. You can also run
bin/console for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run
bundle exec rake install. To release a new version, update the version number in
version.rb, and then run
bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the
.gem file to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/braintreeps/pg_ha_migrations. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
The gem is available as open source under the terms of the MIT License.
Code of Conduct
Everyone interacting in the PgHaMigrations project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.