Skip to content

Using migrate with Go 1.24 "go tool" command #1232

Open
@alexedwards

Description

@alexedwards

I'm trying to use migrate in conjuction with the new go tool command (https://tip.golang.org/doc/go1.24#go-command), but not having much luck. I wondered if anyone knows knows a workaround or the right commands to run?

I've tried:

$ go get -tool  github.com/golang-migrate/migrate/v4/cmd/migrate@latest
$ go tool migrate -path ./migrations -database postgres://user:pa55word@localhost/db_name up

But this fails with the following error.

error: failed to open database: database driver: unknown driver postgres (forgotten import?)

I tried specifying the build tag postgres during the go get, but get the same error.

$ go get -tool -tags 'postgres'  github.com/golang-migrate/migrate/v4/cmd/migrate@latest
$ go tool migrate -path ./migrations -database postgres://user:pa55word@localhost/db_name up
error: failed to open database: database driver: unknown driver postgres (forgotten import?)

In contrast, go run with the build tag postgres works as expected:

$ go run -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest -path ./migrations -database postgres://user:pa55word@localhost/db_name
up
no change

Activity

eberkund

eberkund commented on Mar 4, 2025

@eberkund

It would be nice to be able to manage all my tool dependencies using the same method.

I guess the problem is that go get -tool does not support tags. Maybe migrate could include all the drivers by default and have another tag to exclude drivers?

bcomnes

bcomnes commented on Mar 21, 2025

@bcomnes
Contributor

fwiw, when you use go run -tags 'postgres', you can have migrate defined in your tool sections and it will use that version. Stashing this in a makefile seems to be a decent solution.

skandragon

skandragon commented on Mar 23, 2025

@skandragon

fwiw, when you use go run -tags 'postgres', you can have migrate defined in your tool sections and it will use that version. Stashing this in a makefile seems to be a decent solution.

I'm not sure this is true...

I have it added to my go.mod:

tool (
	github.com/apache/skywalking-eyes/cmd/license-eye
	github.com/golang-migrate/migrate/v4/cmd/migrate
	github.com/golangci/golangci-lint/cmd/golangci-lint
	github.com/goreleaser/goreleaser/v2
	github.com/sqlc-dev/sqlc/cmd/sqlc
)

However:

$  go run -tags 'postgres' migrate -path internal/dbase/metadatadb/migrations -database postgres://localhost/local_metadata up
package migrate is not in std (/opt/homebrew/Cellar/go/1.24.1/libexec/src/migrate)
bcomnes

bcomnes commented on Mar 23, 2025

@bcomnes
Contributor

With the following makefile commands:

migrate-up: ## Run database migrations up (loads .env if present)
	@set -o allexport; \
	if [ -f .env ]; then source .env; fi; \
	go run -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate \
		-database "$$DATABASE_URL" \
		-path ./internal/database/migrations \
		up

migrate-down: ## Rollback the last migration (loads .env if present)
	@set -o allexport; \
	if [ -f .env ]; then source .env; fi; \
	go run -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate \
		-database "$$DATABASE_URL" \
		-path ./internal/database/migrations \
		down 1

Without migrate in my tools directive I get:

 go-todo % make migrate-up
no required module provides package github.com/golang-migrate/migrate/v4/cmd/migrate; to add it:
	go get github.com/golang-migrate/migrate/v4/cmd/migrate
make: *** [migrate-up] Error 1

With migrate in my tools directive, I get:

√ go-todo % make migrate-up
1/u init_schema (25.483542ms)
tool (
	github.com/golang-migrate/migrate/v4/cmd/migrate
)

I don't have a firm grasp on the nuances of the tool directive, so if this is working in some kind of unexpected way, maybe someone can clarify.

?2 go-todo % go version
go version go1.24.1 darwin/arm64

Example repo: https://github.com/bcomnes/go-todo

skandragon

skandragon commented on Apr 1, 2025

@skandragon

@bcomnes I think what you describe is a side-effect.

When you have no tools included, do you still have a reference to github.com/golang-migrate/migrate/v4/cmd/migrate or the package it is included in within your go.mod?

My guess here is the tool definition is creating it, but since you are explicitly running it based on the go run and not go tool command, it is taking what it sees inside your mod file and running that, and that can use the tags.

bcomnes

bcomnes commented on Apr 10, 2025

@bcomnes
Contributor

When you have no tools included, do you still have a reference to github.com/golang-migrate/migrate/v4/cmd/migrate or the package it is included in within your go.mod?

No, nothing else pulls it in when the tools directive is removed.

My guess here is the tool definition is creating it, but since you are explicitly running it based on the go run and not go tool command, it is taking what it sees inside your mod file and running that, and that can use the tags.

It's not quite the desired outcome (using go tool) but it is close (defining the dep and version as a tool, and running it at the correct version, controlled by the mod file, afaict).

yshngg

yshngg commented on Apr 11, 2025

@yshngg

I propose adding a new file internal/cli/build.go with the following build constraints to control database/source driver imports:

//go:build !(aws_s3 || bitbucket || cassandra || clickhouse || cockroachdb || firebird || github || gitlab || go_bindata || godoc_vfs || google_cloud_storage || mongodb || mysql || neo4j || pgx || pgx5 || postgres || ql || redshift || rqlite || snowflake || spanner || sqlcipher || sqlite || sqlite3 || sqlserver || yugabytedb)
// +build !aws_s3,!bitbucket,!cassandra,!clickhouse,!cockroachdb,!firebird,!github,!gitlab,!go_bindata,!godoc_vfs,!google_cloud_storage,!mongodb,!mysql,!neo4j,!pgx,!pgx5,!postgres,!ql,!redshift,!rqlite,!snowflake,!spanner,!sqlcipher,!sqlite,!sqlite3,!sqlserver,!yugabytedb

package cli

import (
	_ "github.com/ClickHouse/clickhouse-go"
	_ "github.com/golang-migrate/migrate/v4/database/cassandra"
	_ "github.com/golang-migrate/migrate/v4/database/clickhouse"
	_ "github.com/golang-migrate/migrate/v4/database/cockroachdb"
	_ "github.com/golang-migrate/migrate/v4/database/firebird"
	_ "github.com/golang-migrate/migrate/v4/database/mongodb"
	_ "github.com/golang-migrate/migrate/v4/database/mysql"
	_ "github.com/golang-migrate/migrate/v4/database/neo4j"
	_ "github.com/golang-migrate/migrate/v4/database/pgx"
	_ "github.com/golang-migrate/migrate/v4/database/pgx/v5"
	_ "github.com/golang-migrate/migrate/v4/database/postgres"
	_ "github.com/golang-migrate/migrate/v4/database/ql"
	_ "github.com/golang-migrate/migrate/v4/database/redshift"
	_ "github.com/golang-migrate/migrate/v4/database/rqlite"
	_ "github.com/golang-migrate/migrate/v4/database/snowflake"
	_ "github.com/golang-migrate/migrate/v4/database/spanner"
	_ "github.com/golang-migrate/migrate/v4/database/sqlcipher"
	_ "github.com/golang-migrate/migrate/v4/database/sqlite"
	_ "github.com/golang-migrate/migrate/v4/database/sqlite3"
	_ "github.com/golang-migrate/migrate/v4/database/sqlserver"
	_ "github.com/golang-migrate/migrate/v4/database/yugabytedb"
	_ "github.com/golang-migrate/migrate/v4/source/aws_s3"
	_ "github.com/golang-migrate/migrate/v4/source/bitbucket"
	_ "github.com/golang-migrate/migrate/v4/source/github_ee"
	_ "github.com/golang-migrate/migrate/v4/source/gitlab"
	_ "github.com/golang-migrate/migrate/v4/source/go_bindata"
	_ "github.com/golang-migrate/migrate/v4/source/godoc_vfs"
	_ "github.com/golang-migrate/migrate/v4/source/google_cloud_storage"
)

This setup ensures that all database/source drivers are built by default when running go build without any -tags flags. The build constraints work as follows:

  1. The //go:build directive (Go 1.17+ syntax) uses negation to exclude specific drivers
  2. The legacy // +build format maintains compatibility with older Go versions
  3. Drivers are only excluded when their specific tag is provided (e.g., -tags 'aws_s3' would exclude AWS S3 support)
$ CGO_ENABLED=0 go run -tags='aws_s3' ./cmd/migrate/
Usage: migrate OPTIONS COMMAND [arg...]
       migrate [ -version | -help ]
...
Source drivers: s3, file
Database drivers: stub
exit status 2

$ CGO_ENABLED=0 go run ./cmd/migrate/
Usage: migrate OPTIONS COMMAND [arg...]
       migrate [ -version | -help ]
...
Source drivers: go-bindata, github-ee, gitlab, bitbucket, s3, gcs, file, github, godoc-vfs
Database drivers: cockroach, redshift, cockroachdb, yugabytedb, firebird, firebirdsql, mysql, cassandra, rqlite, spanner, mongodb, snowflake, stub, postgresql, yugabyte, clickhouse, sqlite, ysql, neo4j, sqlserver, mongodb+srv, ql, sqlcipher, sqlite3, pgx, postgres, pgx5, crdb-postgres, pgx4
exit status 2

Further validation needed.

dhui

dhui commented on Apr 17, 2025

@dhui
Member

Thanks for the discussion! I'd rather wait for go tool to support build tags and not complicate our builds/codebase. Currently, build tags are all managed in the Makefile and I don't want another place to update when adding a new db or source driver.

yshngg

yshngg commented on Apr 18, 2025

@yshngg

Hi @dhui , thx for the response! 🙏 The existing Go community issue golang/go#71503 hasn't seen progress yet. Hoping to provide a temporary workaround here so other projects can manage tool dependencies in go.mod via go tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @skandragon@bcomnes@alexedwards@dhui@eberkund

      Issue actions

        Using migrate with Go 1.24 "go tool" command · Issue #1232 · golang-migrate/migrate