An abstract task migration tool written in Go for Go services. Database and non database migrations management brought to your CLI.
- Use case
- Installation
- Usage
- Features
The motivation behind creating this tool, is to abstract away the database part. If your task can be completed with Pure Go or via a Go driver of your service, then this is for you. Since it makes use of the Go runtime, you can even perform database migrations like PostgreSQL, Mongo, Redis, Elasticsearch, GCP Buckets etc. You just need to be able to interact with your data store or complete your task using Go.
The main use case is when you won't be able to do everything with SQL or No-SQL syntax. There might be some tasks where you need to aggregate data, iterate over them, and do business related stuff with the retrieved data. All you need to know is Go syntax and write a Go program.
go get github.com/g14a/metana
homebrew tap g14a/homebrew-metana
homebrew install metana
Checkout the releases page and download your platform's binaries to install them.
Prerequisites:
- Git
- Go 1.13 or newer. Go modules are needed. Better if its the latest version.
git clone https://github.com/g14a/metana
cd metana
go install
docker pull g14a/metana
docker run g14a/metana
After installation, let's just hit metana on the terminal.
$ metana
An abstract migration tool for all types of migrations
Usage:
metana [command]
Available Commands:
config Manage your local metana config in .metana.yml
create Create a migration in Go
down Run downward migrations
help Help about any command
init Initialize a migrations directory
list List existing migrations
up Run upward migrations
wipe Wipe out old stale migration files and track in your store
Flags:
--config string config gen (default is $HOME/.metana.yaml)
-h, --help help for metana
-t, --toggle Help message for toggle
Use "metana [command] --help" for more information about a command.
init
initializes a boilerplate migrations directory in your current path.
$ metana init
✓ Created /Users/g14a/metana/migrations/main.go
By default it will create a migrations
folder if no such folder exists. If it does, it adds the main.go
file into the same.
If you want to initialize migrations in a different directory, you can do so with the --dir | -d
flag:
metana init --dir /path/to/folder
create
creates a migration script with two functions Up()
and Down()
denoting the upward and downward migration of the same.
$ metana create initSchema
✓ Created /Users/g14a/metana/migrations/1614532908-Sample.go
✓ Updated /Users/g14a/metana/migrations/main.go
Head over to your 1614532908-InitSchema.go
to edit your script. Remember to not change any function signature.
up
runs all the upward migrations in the migrations directory in order of their creation time.
$ metana up
>>> Migrating up: 1619942687-InitSchema.go
InitSchema up
>>> Migrating up: 1619942704-AddIndexes.go
AddIndexes up
>>> migration : complete
down
runs the downward migrations in the reverse order of creation time because we're trying to undo the upward migrations.
$ metana down
>>> Migrating down: 1619942704-AddIndexes.go
AddIndexes down
>>> Migrating down: 1619942687-InitSchema.go
InitSchema down
>>> migration : complete
wipe
removes stale(already executed) migration files and track in your store to keep things clean.
$ metana wipe
? Wiping will delete stale migration files. Continue? Yes
list
lists all the migrations present in your migrations folder along with the last modified time.
$ metana list
+----------------------------------+------------------+
| MIGRATION | LAST MODIFIED |
+----------------------------------+------------------+
| 1619943670-InitSchema.go | 02-05-2021 13:51 |
| 1619943677-AddIndexes.go | 02-05-2021 13:51 |
| 1619943874-AddFKeys.go | 02-05-2021 13:54 |
| 1619943888-AddBoilerPlateRows.go | 02-05-2021 13:54 |
+----------------------------------+------------------+
Specify a custom directory when creating and running upward or downward migrations using the --dir
flag. Be default it is set to "migrations"
$ metana init --dir schema-mig
✓ Created /Users/g14a/metana/schema-mig/main.go
$ metana create initSchema --dir schema-mig
✓ Created /Users/g14a/metana/schema-mig/scripts/1619943164-InitSchema.go
✓ Updated /Users/g14a/metana/schema-mig/main.go
$ metana up --dir schema-mig
>>> Migrating up: 1619943670-InitSchema.go
InitSchema up
>>> migration : complete
Run upward and downward migrations until(and including) a certain migration with the --until
flag.
$ metana create initSchema
✓ Created /Users/g14a/metana/migrations/scripts/1619942687-InitSchema.go
✓ Updated /Users/g14a/metana/migrations/main.go
$ Create more migration scripts...
$ metana list
+----------------------------------+------------------+
| MIGRATION | LAST MODIFIED |
+----------------------------------+------------------+
| 1619943670-InitSchema.go | 02-05-2021 13:51 |
| 1619943677-AddIndexes.go | 02-05-2021 13:51 |
| 1619943874-AddFKeys.go | 02-05-2021 13:54 |
| 1619943888-AddBoilerPlateRows.go | 02-05-2021 13:54 |
+----------------------------------+------------------+
$ metana up --until AddFkeys
>>> Migrating up: 1619942687-InitSchema.go
InitSchema up
>>> Migrating up: 1619942704-AddIndexes.go
AddIndexes up
>>> Migrated up until: 1619942704-AddIndexes.go
>>> migration : complete
$ metana down --until AddIndexes
>>> Migrating down: 1619943888-AddBoilerPlateRows.go
AddBoilerPlateRows down
>>> Migrating down: 1619943874-AddFKeys.go
AddFKeys down
>>> Migrating down: 1619943677-AddIndexes.go
AddIndexes down
>>> Migrated down until: 1619943677-AddIndexes.go
>>> migration : complete
Notice the capitalized format when passing to --until
.
Store and track your migrations in your favourite database by passing the --store
flag.
metana up --store <db-connection-url>
If your connection URL is store in an environment variable you can pass it as --store @MONGO_URL
and it will automatically be picked up from your environment.
Right now, PostgreSQL(which means even CockroachDB URLs) and MongoDB are supported to store migrations.
If no --store
flag is passed, migrations will be stored in a default migrate.json
file in the migrations directory.
Dry run your migrations using the --dry
flag.
You can dry run your migrations using the explicit --dry
option. This option doesn't track any migrations, doesn't create a default migrate.json
file. It literally just dry runs. However your tasks are run. This helps when you're incrementally writing, testing and running your functions instead of manually deleting states in your store.
$ metana up --dry
>>> Migrating up: 1619942687-InitSchema.go
InitSchema up
>>> Migrating up: 1619942704-AddIndexes.go
AddIndexes up
>>> dry run migration : complete
$ metana down --dry
>>> Migrating down: 1619942704-AddIndexes.go
AddIndexes down
>>> Migrating down: 1619942687-InitSchema.go
InitSchema down
>>> dry run migration : complete
All the other options like --dir
and --until
work along with --dry
.
Set your custom config in your .metana.yml
file. As of now it supports dir
and store
keys.
For eg:
dir: schema-mig
store: '@MONGO_URL'
Remember to add it to your git unless you want to miss migrations on deployments.
If your store has a remote database URL you can specify it via '@' syntax and it will automatically be picked up from your environment variables (Remember the single quotes).You don't want to hardcode API Keys and connection URLs in your codebase.
.metana.yml
is created automatically when you run metana init
which can be used for subsequent migration operations.
You can either manually add the config on to the .metana.yml
file or do it via
metana config set --store @MONGO_URL
$ metana config set --help
Set your metana config
Usage:
metana config set [flags]
Flags:
-d, --dir string Set your migrations directory (default "migrations")
-h, --help help for set
-s, --store string Set your store
CAUTION:
If you change the dir flag in your .metana.yml
after running metana init
, don't forget to rename your migrations directory to the new directory. Otherwise running migrations would result in failure.
Priority order of config:
- Flags passed explicitly
.metana.yml
if it exists.- Default values of flags.
Wipe out stale(already executed) migration files and update your store with the wipe
command.
$ metana wipe
Wipe out old stale migration files and track in your store
Usage:
metana wipe [flags]
Flags:
-d, --dir string Specify custom migrations directory
-h, --help help for wipe
-s, --store string Specify a connection url to track migrations
-y, --yes Proceed at all costs i.e by pass the prompt
Global Flags:
--config string config gen (default is $HOME/.metana.yaml)
Pass the -y
flag to bypass the prompt during script.
Even the wipe
command takes configuration from your .metana.yml
file one exists.
Otherwise the priority order is considered while wiping.
With the latest version of metana you can add create a custom template Go program and create migration scripts with that.
Your template should be a valid Go program(atleast syntactically) and needs
to have an Up()
and a Down()
function returning an error.
Spaces between the function name and the returning error are taken care of.
The contents of your Up()
and Down()
will be copied into the migration script.
So the following signatures are valid for now:
func Up() error {}
func Up() (err error) {}
func Down() error {}
func Down() (err error) {}
Create a migration with a template with the --template
flag:
metana create addIndexes --template ../path/to/template.go
You can get enable shell completions to metana by doing
metana completion --help
and selecting your shell.
$ metana completion --help
To load completions:
Bash:
$ source <(metana completion bash)
# To load completions for each session, execute once:
# Linux:
$ metana completion bash > /etc/bash_completion.d/metana
# macOS:
$ metana completion bash > /usr/local/etc/bash_completion.d/metana
Zsh:
# If shell completion is not already enabled in your environment,
# you will need to enable it. You can execute the following once:
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
# To load completions for each session, execute once:
$ metana completion zsh > "${fpath[1]}/_metana"
# You will need to start a new shell for this setup to take effect.
fish:
$ metana completion fish | source
# To load completions for each session, execute once:
$ metana completion fish > ~/.config/fish/completions/metana.fish
PowerShell:
PS> metana completion powershell | Out-String | Invoke-Expression
# To load completions for every new session, run:
PS> metana completion powershell > metana.ps1
# and source this file from your PowerShell profile.
Usage:
metana completion [bash|zsh|fish|powershell]
Flags:
-h, --help help for completion
Global Flags:
--config string config gen (default is $HOME/.metana.yaml)
Now you can run upward and downward migrations and specify your .env
files.
Use the --env-file
flag to do so. By default the value is set to .env
.
$ metana up --env-file dev.env
✓ .metana.yml found
>>> Migrating up: 1623502023-InitSchema.go
InitSchema up
>>> migration : complete
With metana, you can manage your multiple environments in your migrations setup.
You can manage your deployments like dev
, staging
, and production
etc.
It is recommended that you either manage environments, or the traditional metana init
way.
To initialize a environment, run metana init
with the --env
flag.
All environments need to be unique.
$ metana init --env dev
✓ Created /Users/g14a/metana/migrations/environments/dev/main.go
$ metana init --env dev
Environment `dev` already exists
$ metana create initSchema --env dev
✓ .metana.yml found
✓ Created /Users/g14a/metana/migrations/environments/dev/scripts/1623502023-InitSchema.go
✓ Updated /Users/g14a/metana/migrations/environments/dev/main.go
$ metana create initSchema --env staging
✓ .metana.yml found
✓ Created /Users/g14a/metana/migrations/environments/staging/scripts/1623502023-InitSchema.go
✓ Updated /Users/g14a/metana/migrations/environments/staging/main.go
$ metana up --env dev
✓ .metana.yml found
>>> Migrating up: 1623502023-InitSchema.go
InitSchema up
>>> migration : complete
All the other flags like --dry
, --until
, --dir
work with environments seamlessly.
You can specify your environment configuration in the same .metana.yml
file.
dir: migrations
store: ""
environments:
- name: dev
store: "dev store"
- name: staging
store: "staging store"
- name: prod
store: "prod store"
The name
field of the config should be the same as the environment directory inside /environments/
.