feat(migrations): expose the SQL migrations as an embed.FS - #138
Merged
Conversation
These .sql files are plain files in the module, and Go's embed cannot reach into a dependency - a consumer that imports restcol can see the package but not the files beside it. So a downstream service wanting to apply these migrations had only one option: copy them. FootprintAI/grandturks#987 did exactly that, into deploy/demo/restcol-migrations/ behind a drift test. It was the right stopgap for a broken deployment, but a copy of a file versioned elsewhere drifts - and silently in the direction that matters, since the copy keeps working while the pinned version moves underneath it. Exporting the FS makes "the version pinned" and "the migrations applied" the same fact by construction. Consumers read it with golang-migrate's iofs source driver. The tests are about coverage rather than SQL: a migration on disk but not in the embed is invisible to every consumer, and nothing else in the build would say so. TestEmbeddedMigrationsCoverEveryFileOnDisk walks both and compares, so a .sql file that lands outside the embed pattern - in a subdirectory, say - fails instead of being silently skipped. Verified it does fail that way before trusting it. Asked for by FootprintAI/grandturks#1004.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Asked for by FootprintAI/grandturks#1004, which is Phase 1 of that repo's
docs/architecture/database-schema-provisioning.md.Why
These
.sqlfiles are plain files in the module, and Go'sembedcannot reach into a dependency — a consumer that imports restcol can see the package but not the files beside it. So a downstream service wanting to apply these migrations had exactly one option: copy them.FootprintAI/grandturks#987 did that, into
deploy/demo/restcol-migrations/behind a drift test. It was the right stopgap for a deployment that was broken, but a copy of a file versioned elsewhere drifts — and silently in the direction that matters, because the copy keeps working while the pinned version moves underneath it.Exporting the FS makes "the version pinned" and "the migrations applied" the same fact by construction.
What changed
migrations/embed.go— one//go:embed *.sqland an exportedEmbeddedMigrations, the shape grandturks' owncomponents/notification/broker/migrationsalready uses. Consumers read it with golang-migrate'siofssource driver:No new dependencies —
embedis stdlib, and nothing here imports golang-migrate.Tests
They are about coverage, not SQL. A migration on disk but not in the embed is invisible to every consumer, and nothing else in the build would say so:
TestEmbeddedMigrationsCoverEveryFileOnDisk— walks the directory and the embed FS and compares both the file set and the bytes. Verified it actually fails: dropping a.sqlinto a subdirectory producesextra/0003_x.up.sql is on disk but not in EmbeddedMigrationsrather than passing quietly.TestEveryMigrationHasBothDirections— an up without a down is a migration that cannot be rolled back, discovered when someone needs to roll it back.TestNoEmbeddedMigrationIsEmpty— an empty file advances the version table without changing the schema, which is worse than a failure.None of them needs a database.
Note on the existing suite
go test ./...is red on this branch — and equally red on a cleanmain, which I checked by stashing this change:integrationtest,pkg/app, and the threepkg/storage/*suites panic inAutoMigrateon a nilPostgresDbwhen no Postgres is configured. Pre-existing and unrelated to this PR, but worth stating plainly rather than reporting "tests pass": they don't, they just don't fail because of this.go build ./...,go vet ./migrations/,gofmtandgo test ./migrations/are all clean.Once this merges, grandturks bumps its pin and deletes
deploy/demo/restcol-migrations/plus its drift test (that half is grandturks#1005).