Custom MigrationResolver support

[flyway]

Original author: jt...@okta.com (March 28, 2013 21:19:58)

Howdy,

I know there's been at least one request for custom MigrationResolver support in the past (https://groups.google.com/forum/#!topic/flyway-dev/yLxuUIwK8uI) but I'd like to bring up a new use case which requires the ability to plugin a custom MigrationResolver.

For migrations that involve complex business logic we've built a meta-data migration system which allows complex migrations to be defined declaratively in XML and then parsed by a migrator which enforces the business logic and manages the state change.

In our use case we're fine keeping the exiting default MigrationResolvers but would also like the ability to add additional resolvers. One thought would be to extend the Flyway and CompositeMigrationResolvers to support add(MigrationResolver resolver). If we're going this far to make MigrationResolver part of the public API I could also see just supporting a fully overridable MigrationResolver for more flexibility by instead adding Flyway#setMigrationResolver.

Do you think this use case justifies adding this to the project? Any suggestions for the best approach?

I'm going to code up a solution now and would be happy to work with you on getting something suitable for adding to the core project.

Original issue: http://code.google.com/p/flyway/issues/detail?id=469

[flyway]

From jt...@okta.com on March 29, 2013 07:07:45
Pull request for the more generic solution is posted here: #18

[flyway]

From axel.fontaine.business@gmail.com on April 01, 2013 17:39:59
Hi Jon,

thanks for explaining your use case. I currently don't want to make this part of the API public. I want to keep it deliberately small and easy. This doesn't pull its own conceptual weight when you look at the actual number of users it would serve.
I would recommend a common Java migration base class to support the loading and parsing of your xml dialect.

Can you tell a bit more about what kind of functionality your xml solution is trying to provide so I can keep it in mind for future consideration?

Cheers
Axel

[flyway]

From jt...@okta.com on April 01, 2013 21:29:37
Hi Axel,

I can appreciate your desire to keep the API surface area to a minimum and to that end you've done a great job of keeping the code simple and easy to understand and it'd obviously add overhead to maintain the lifecycle of the additional classes. FWIW these were the classes that I needed to depend on to make the use case work:

import com.googlecode.flyway.core.dbsupport.DbSupport;
import com.googlecode.flyway.core.dbsupport.JdbcTemplate;
import com.googlecode.flyway.core.resolver.MigrationExecutor;
import com.googlecode.flyway.core.api.MigrationType;
import com.googlecode.flyway.core.api.MigrationVersion;
import com.googlecode.flyway.core.resolver.MigrationInfoHelper;
import com.googlecode.flyway.core.resolver.MigrationResolver;
import com.googlecode.flyway.core.resolver.ResolvedMigration;

If I understand your suggestion properly, that'd require a separate java file per migration. Sure the code that reads the xml could be in a common base class but we'd still need a separate java migration per version to allowing the versioning to work. Is my understanding correct?

If that's the case there are two reasons that doesn't work for us:

1. That require additional boilerplate code per migration
2. We want to limit the scope of what can be migrated by what's possible in the xml, having the java migration will make it too easy to expand scope

Here's more info on the use case: We have a number of tables in our DB which have such complex business logic rules: e.g. validation, downstream dependencies, etc. that this data can only be changed by code. This is pretty common and we have many distinct object graphs that fall into this category. To support this we could have gone with writing java migrations but we instead opted for a data driven model as this limits the scope of what's possible in these complex migrations. And tools for creating these migrations, simply dump the declarative changes.

I realize this exact problem might not be common but perhaps there is a compromise. Rather than allowing for customer MigrationResolvers would you be willing to add a generic FileBaseMigrationResolver? Given a set of locations this would read either from the file system or class path any files which match some configured prefix and suffix following the flyway versioning conventions. Then it'd delegate to a handler interface like:

public interface FileBaseMigrationHandler
{
/**

• Method to perform a migration

• ...

• @return True if the migration succeeded
  */
  public boolean migrate(String filename, InputStream fileContents) {

  }
  }

[axelfontaine]

Small update on this: I put a lot of thought into this and I believe I now have a solution that would minimize the amount of API I would need to expose for this.

It would involve exposing MigrationResolver, ResolvedMigration and MigrationExecutor. Expect to see this in the next major release.

[thomasgillet]

Hello Jon and Axel,

I'd like to report another use case which I think would benefit from a custom migration resolver.

I'm currently trying to integrate Flyway with OSGi (using indifferently Felix, Knopflerfish or Virgo).

My first attempt was to deploy the Flyway bundle as is in my framework, and then write a little bundle containing SQL migration files and some boilerplate code to call Flyway.migrate() at start time.

I didn't manage to do it, and after looking at the code and the OSGi sample it seems to me that the only two ways to use Flyway with OSGi are:

• embed Flyway into the bundle in charge of the migrations ;
• package migration files in a fragment attached to the Flyway bundle (sample).

The problem seems to be the line 42 in OsgiClassPathLocationScanner.java which prevents the migration file list to be retrieved from another bundle:

Bundle bundle = FrameworkUtil.getBundle(getClass());
Enumeration<URL> entries = bundle.findEntries(locationUrl.getPath(), "*", true);

A custom migration resolver would resolve my problem and allow me to put my migration files in any bundle. (And therefore in more than one bundle, since my goal is to deploy one bundle per database schema.)

Looking forward to the 3.0 release,
Thomas

[nafg]

FWIW another use case: I'd like to keep my migrations, written in scala, within a "cake", so they won't be top-level classes discoverable by reflection. Instead I can now pass the migrations to flyway explicitly, with a teeny bit of helpers.
Anyway thanks for implementing it.