Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the ability to apply missing (out-of-order) migrations #280

Merged
merged 16 commits into from Oct 24, 2021

Conversation

mfridman
Copy link
Collaborator

@mfridman mfridman commented Oct 16, 2021

Closes #249 #233 #262 #279 #172 and probably a bunch of other issues.

At a high, the library adds functional options to support applying missing or out-of-order migrations and the pre-built binary will have a new flag: -allow-missing=true or -out-of-order=true .. not sure which name folks like better? (suggestions welcome)

goose.Up(
	db, 
	migrationsDir, 
	goose.WithAllowMissing(), // <- apply missing migrations
)

Note. There was an existing bug reported previously, described in detail here #279. In this PR goose will detect "pending" migrations and return and error unless you enable the flags (CLI) or options (library) mentioned earlier. This should have always been the behaviour, but alas this might be a breaking change to some.

Those that follow hybrid-versioning should not have pending migrations and thus will not be affected.

  • Add a dry-run options that display what migrations will be applied without applying them. This will be a separate feature.
  • Add tests for down out-of-order migrations. Need to decide how to handle them, i.e., what order do you migrate down in? Probably based on the applied order instead of the version_id order.

Down migrations are applied in the reverse order they were applied. So up: 1, 4, 2, 3, 5 means down will be the reverse: 5, 3, 2, 4, 1.

version = flags.Bool("version", false, "print version")
certfile = flags.String("certfile", "", "file path to root CA's certificates in pem format (only support on mysql)")
sequential = flags.Bool("s", false, "use sequential numbering for new migrations")
allowMissing = flags.Bool("allow-missing", false, "applies missing (out-of-order) migrations")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I keep flip flopping on the name, I've narrowed it down to:

  • out-of-order
  • allow-missing

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going with "allow-missing" terminology, which is analogous to "out-of-order"

@mfridman
Copy link
Collaborator Author

mfridman commented Oct 16, 2021

One behaviour I don't like is that running Status displays migrations in the file order, instead of the applied database order. Which looks odd now that out-of-order migrations are supported.

This was always the default in goose .. but I think this should list migrations based on time applied instead of the file order. 🤔

Here I didn't apply migration 6 and 8. And the current Status returns:

2021/10/16 16:45:47 OK    00001_a.sql
2021/10/16 16:45:47 OK    00002_b.sql
2021/10/16 16:45:47 OK    00003_c.sql
2021/10/16 16:45:47 OK    00004_d.sql
2021/10/16 16:45:47 OK    00005_e.sql
2021/10/16 16:45:47 OK    00007_g.sql
2021/10/16 16:45:47     Applied At                  Migration
2021/10/16 16:45:47     =======================================
2021/10/16 16:45:47     Sat Oct 16 20:45:47 2021 -- 00001_a.sql
2021/10/16 16:45:47     Sat Oct 16 20:45:47 2021 -- 00002_b.sql
2021/10/16 16:45:47     Sat Oct 16 20:45:47 2021 -- 00003_c.sql
2021/10/16 16:45:47     Sat Oct 16 20:45:47 2021 -- 00004_d.sql
2021/10/16 16:45:47     Sat Oct 16 20:45:47 2021 -- 00005_e.sql
2021/10/16 16:45:47     Pending                  -- 00006_f.sql
2021/10/16 16:45:47     Sat Oct 16 20:45:47 2021 -- 00007_g.sql
2021/10/16 16:45:47     Pending                  -- 00008_h.sql

But then I ran Up allowing out-of-order migrations and then ran Status:

2021/10/16 16:45:52 OK    00006_f.sql
2021/10/16 16:45:52 OK    00008_h.sql
2021/10/16 16:45:52 goose: no migrations to run. current version: 8
2021/10/16 16:45:52     Applied At                  Migration
2021/10/16 16:45:52     =======================================
2021/10/16 16:45:52     Sat Oct 16 20:45:47 2021 -- 00001_a.sql
2021/10/16 16:45:52     Sat Oct 16 20:45:47 2021 -- 00002_b.sql
2021/10/16 16:45:52     Sat Oct 16 20:45:47 2021 -- 00003_c.sql
2021/10/16 16:45:52     Sat Oct 16 20:45:47 2021 -- 00004_d.sql
2021/10/16 16:45:52     Sat Oct 16 20:45:47 2021 -- 00005_e.sql
2021/10/16 16:45:52     Sat Oct 16 20:45:52 2021 -- 00006_f.sql <----- applied 5s after 
2021/10/16 16:45:52     Sat Oct 16 20:45:47 2021 -- 00007_g.sql
2021/10/16 16:45:52     Sat Oct 16 20:45:52 2021 -- 00008_h.sql <----- applied 5s after 

@mfridman
Copy link
Collaborator Author

Missing migrations now raise an error when attempting to apply migrations.

2021/10/24 11:00:38 OK    00001_a.sql
2021/10/24 11:00:38 OK    00002_b.sql
2021/10/24 11:00:38 OK    00003_c.sql
2021/10/24 11:00:38 OK    00004_d.sql
2021/10/24 11:00:38 OK    00005_e.sql
2021/10/24 11:00:38 OK    00007_g.sql
error: found 1 missing migrations:
        version 6: testdata/postgres/migrations/00006_f.sql

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant