Database migrations. CLI and Golang library.
Clone or download
dhui Merge pull request #128 from andrei-m/fix-redshift
Fix Redshift migrations driver
Latest commit 7f00868 Nov 14, 2018
Permalink
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Add Go version to but report template Sep 19, 2018
cli Use `git describe --tags` to set the version number when building the… Nov 12, 2018
database Merge pull request #128 from andrei-m/fix-redshift Nov 14, 2018
source Use strings.NewReader() instead of bytes.NewReader() when source is a… Nov 6, 2018
testing Remove unused invalid formatter Jan 21, 2018
.dockerignore add dockerization Fix #23 May 9, 2018
.gitignore Ignore vendor dir Jan 19, 2018
.travis.yml Revert "Remove support for Go modules" Oct 10, 2018
CONTRIBUTING.md Remove references to `dep` from docs Oct 29, 2018
Dockerfile Revert "Docker builds need to use dep to manage dependencies" Oct 10, 2018
FAQ.md FAQ: add required extension Jul 28, 2017
Gopkg.lock Update dependencies Oct 15, 2018
Gopkg.toml github.com/docker/docker package updated to api v1.37 Oct 15, 2018
LICENSE Update copyright in LICENSE Nov 8, 2018
MIGRATIONS.md Link to comparison of transactional DDLs Jul 10, 2018
Makefile Replace 'redshift' with 'redshift2' Nov 3, 2018
README.md Remove references to `dep` from docs Oct 29, 2018
docker-deploy.sh Add latest tag to docker image during build Jul 3, 2018
go.mod Update dependencies Oct 15, 2018
go.sum Update dependencies Oct 15, 2018
log.go add some more comments Feb 10, 2017
migrate.go Use v4 for Go module support Oct 10, 2018
migrate_test.go Use strings.NewReader() instead of bytes.NewReader() when source is a… Nov 6, 2018
migration.go fix old ref to LogString Feb 10, 2017
migration_test.go rename Migration.LongString to LogString Feb 10, 2017
util.go Improve error output for missing or malformed '-database' and '-sourc… Aug 8, 2018
util_test.go Improve error output for missing or malformed '-database' and '-sourc… Aug 8, 2018

README.md

Build Status GoDoc Coverage Status packagecloud.io Docker Pulls Supported Go Versions GitHub Release

migrate

Database migrations written in Go. Use as CLI or import as library.

  • Migrate reads migrations from sources and applies them in correct order to a database.
  • Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
  • Database drivers don't assume things or try to correct user input. When in doubt, fail.

Looking for v1?

Forked from mattes/migrate

Databases

Database drivers run migrations. Add a new database?

Database URLs

Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?option1=true&option2=false

Any reserved URL characters need to be escaped. Note, the % character also needs to be escaped

Explicitly, the following characters need to be escaped: !, #, $, %, &, ', (, ), *, +, ,, /, :, ;, =, ?, @, [, ]

It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python helpers below:

$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$

Migration Sources

Source drivers read migrations from local or remote sources. Add a new source?

CLI usage

  • Simple wrapper around this library.
  • Handles ctrl+c (SIGINT) gracefully.
  • No config search paths, no config files, no magic ENV var injections.

CLI Documentation

Basic usage:

$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2

Docker usage

$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate 
    -path=/migrations/ -database postgres://localhost:5432/database up 2

Use in your Go project

  • API is stable and frozen for this release (v3.x).
  • Uses Go modules to manage dependencies
  • To help prevent database corruptions, it supports graceful stops via GracefulStop chan bool.
  • Bring your own logger.
  • Uses io.Reader streams internally for low memory overhead.
  • Thread-safe and no goroutine leaks.

Go Documentation

import (
    "github.com/golang-migrate/migrate/v4"
    _ "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/github"
)

func main() {
    m, err := migrate.New(
        "github://mattes:personal-access-token@mattes/migrate_test",
        "postgres://localhost:5432/database?sslmode=enable")
    m.Steps(2)
}

Want to use an existing database client?

import (
    "database/sql"
    _ "github.com/lib/pq"
    "github.com/golang-migrate/migrate/v4"
    "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/file"
)

func main() {
    db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable")
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    m, err := migrate.NewWithDatabaseInstance(
        "file:///migrations",
        "postgres", driver)
    m.Steps(2)
}

Migration files

Each migration has an up and down migration. Why?

1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql

Best practices: How to write migrations.

Development and Contributing

Yes, please! Makefile is your friend, read the development guide.

Also have a look at the FAQ.


Looking for alternatives? https://awesome-go.com/#database.