API: Make it possible to use pre-instantiated Java-based migrations

[guylabs]

Hi,

as we are using Spring with Flyway we want to use other beans inside the SpringJdbcMigrations. Therefore we created a new resolver which retrieves the SpringJdbcMigrations from the Spring application context. This way we are able to define the migrations inside the application context and reference other beans we would need for this migration inside the SpringJdbcMigraiton with the dependency injection mechanisms of Spring (@Autowired). Please have a look at the linked pull request and the tests how this is done.

If this pull request is accepted and integrated, then I would also add some documentation to the website project of Flyway on how to use this new resolver.

Thanks and regards,

Guy

[kapil-malik]

Hi,
This is very useful for us as well. The PR looks pretty cool. Any updates on how soon can it be part of flyway builds?
Till then we will override SpringJbdcMigrationResolver (like your code) as a workaround.

[guylabs]

@kapil-malik Thanks and yes this would be really useful.

@axelfontaine Thanks a lot for taking it up as a feature!

[wimdeblauwe]

+1 for this!

For those that cannot wait until this gets done: I have added a workaround on stackoverflow: http://stackoverflow.com/a/36474970/40064

BTW: If this is released, will there be an easy way to still have the SQL resolver as well to avoid the hacky workaround I had to do now?

[crstalli]

@wimdeblauwe Thank you very much! That was extremely helpful!

[marbot]

+1, that would be a very useful feature.

[axelfontaine]

See also #869

[geirgp]

+1

[tpischke]

+1

[paolodenti]

+1

[k-yosuke]

+1

[zinefer]

When I use Spring JPA flyway migrations seem to run before the application context has been instantiated ... Does anyone know of a way around this, or how I can instantiate my repositories inside a flyway migration?

[OldIMP]

We use a workaround from https://github.com/avehlies/spring-beans-flyway2 But I really don't recommend injecting data repo in migrations since the domain could have been changed.

[zinefer]

Yes, I've tried that technique... But in my project, flyway migrations are being run before the application context - I get Null exceptions.

[zinefer]

Infact, if you take that project (spring-beans-flyway2) and add compile("org.springframework.boot:spring-boot-starter-data-jpa") to the dependencies in build.gradle it will fail with a null pointer exception

[OldIMP]

Sry I forgot to mention our configuration:

@Configuration
class FlywayConfiguration {
    @Primary
    @Bean(name = "flywayInitializer")
    @DependsOn("springUtility") //or whatever your ApplicationContextAware is named
    FlywayMigrationInitializer flywayMigrationInitializer(){
      //your flyway init configuration...
    }
}

The trick here is to force FlywayMigrationInitializer to get instantiated after the ApplicationContextAware workaround

[zinefer]

I will try that when I get home. Thanks so much.

EDIT: I did eventually get this working. See my SO answer here. Thanks everyone.

[purocean]

+1

[fladulima]

+1

[idallas456]

+1, this should really get fixed at some point.

[purocean]

Now I use my simple Migration for app, very simple. I leave Flayway for long time.

https://github.com/purocean/kotlin-db-migration

1. simple & light
2. name based version
3. support dry run
4. support code migration and sql migration

Only support MySQL :(

[axelfontaine]

This has now been implemented generically using a new javaMigrations property in the API.

Spring users can use this to automatically use all JavaMigration Spring beans with Flyway:

import org.flywaydb.core.Flyway;
import org.flywaydb.core.api.migration.JavaMigration;
import org.springframework.context.ApplicationContext;

...
ApplicationContext applicationContext = ...; // obtain a reference to Spring's ApplicationContext.

Flyway flyway = Flyway.configure()
    .dataSource(url, user, password)
    // Add all Spring-instantiated JavaMigration beans
    .javaMigrations(applicationContext.getBeansOfType(JavaMigration.class).values().toArray(new JavaMigration[0]))
    .load();
flyway.migrate();

[guylabs]

@axelfontaine Thanks for incorporating the feedback! I will have a look at it when I'll work again with a Spring project. Maybe someones else in the thread is able to use it the way you implemented it. Always nice to see that feedback gets integrated!

[simonzhong1985]

Sry I forgot to mention our configuration:

@Configuration
class FlywayConfiguration {
    @Primary
    @Bean(name = "flywayInitializer")
    @DependsOn("springUtility") //or whatever your ApplicationContextAware is named
    FlywayMigrationInitializer flywayMigrationInitializer(){
      //your flyway init configuration...
    }
}

The trick here is to force FlywayMigrationInitializer to get instantiated after the ApplicationContextAware workaround

This has now been implemented generically using a new javaMigrations property in the API.

Spring users can use this to automatically use all JavaMigration Spring beans with Flyway:

import org.flywaydb.core.Flyway;
import org.flywaydb.core.api.migration.JavaMigration;
import org.springframework.context.ApplicationContext;

...
ApplicationContext applicationContext = ...; // obtain a reference to Spring's ApplicationContext.

Flyway flyway = Flyway.configure()
    .dataSource(url, user, password)
    // Add all Spring-instantiated JavaMigration beans
    .javaMigrations(applicationContext.getBeansOfType(JavaMigration.class).values().toArray(new JavaMigration[0]))
    .load();
flyway.migrate();

@axelfontaine
Could you help to provide one example to show how to inject spring bean into class implementation of JavaMigration interface?

Thanks,
Simon

[fitzyjoe]

@simonzhong1985 Right now all of my Flyway config is handled through the application.yml. Can I still configure it that way when using your code - I'm referring to this comment:

//your flyway init configuration...

Also, I don't know what class the second snippet is in. This is the snippet where you get a reference to Spring's ApplicationContext and call flyway.migrate(). What calls this code?

[jamesdh]

Has something changed since @axelfontaine's comment? It appears that having a JavaMigration as a Spring Component (and making use of dependency injection, etc) is automatically loaded and migrated without needing to define a FlywayConfigurationCustomizer that performs the example code he provided. Simply having the Bean-based JavaMigration in your regular source code (so Spring can scan it) as opposed to the typical Flyway migration location appears to be sufficient now.

[nbabb]

As @jamesdh pointed out, we are able to use BaseJavaMigration as spring @Component's, along with dependency injection, without the code cited above. The migrations just need to be component scanned.

Once we upgraded to spring boot 2.7.4 however, we did have to set allow-circular-references=true to avoid circular dependency errors. I've been trying to find a way to preserve what we have, without setting allow-circular-references=true, as that seems to be framed as an option of last resort. I've yet to find a satisfactory solution.

The command line runner strategy mentioned here allows the spring boot app to become accessible before the migrations are finished running, and has the added disadvantage of not being configurable via application.yml. I've been searching for a way to trigger the migration at a point where we'd avoid circular dependency issues, but prior to the application becoming available, but haven't found a solution yet.

And the SpringUtility strategy mentioned here which would allow us to keep the application.yml based configuration, didn't resolve the circular dependency issues for us.

We recognize that having migrations use repository/entity code is an issue when our object model changes. The strategy we employ is to simply remove the migration body once this becomes a problem, and to create a new migration, if it is behavior we need to preserve. We typically use java/spring based migrations for data seeding/migration tasks and not to create db structure.

It'd be nice if the ability to use spring/java based migrations (making full use of the repository layer) was documented and fully supported in the future. Using JDBCTemplate is not a satisfactory solution for use, when we have a nice repository layer to work with.

EDIT
The strategy that seems to work is to disable the automatic running of flyway (spring.flyway.enabled=false) and to then launch flyway manually using an InitializingBean. The InitializingBean afterPropertiesSet appears to block Spring Boot from starting before the migrations are completed, which is what we want.

@Component
class FlywayInit(
    private val runner: FlywayMigrationRunner
) : InitializingBean {

    override fun afterPropertiesSet() {
        runner.runMigrations()
    }
}

@Component
class FlywayMigrationRunner(
    private val migrations: List<BaseJavaMigration>,
    private val dataSource: DataSource
) {

    fun runMigrations(): MigrateResult =
        Flyway.configure()
            .dataSource(dataSource)
            .baselineVersion("1")
            .baselineDescription("initial_script")
            .baselineOnMigrate(true)
            .table("my_schema_history")
            .javaMigrations(*migrations.toTypedArray())
            .load()
            .migrate()
}

[la3rence]

If we set up a configuration bean by ourselves, we might exclude the FlywayAutoConfiguration.class with the annotation @SpringBootApplication, which will help to avoid the circular dependency error. See my blog post for details that how to integrate Flyway and do DI with Springboot.