Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add native-image support for Flyway #31999

Closed
mhalbritter opened this issue Aug 8, 2022 · 16 comments
Closed

Add native-image support for Flyway #31999

mhalbritter opened this issue Aug 8, 2022 · 16 comments
Assignees
@mhalbritter
Labels
theme: aot An issue related to Ahead-of-time processing type: enhancement A general enhancement
Milestone
3.0.0-RC1

Comments

@mhalbritter
Copy link
Contributor

mhalbritter commented Aug 8, 2022 •
edited

The Flyway smoke tests currently fail with GraalVM complaining about proxy generation, which could be related to @FlywayDataSource:

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dataSource': Unsatisfied dependency expressed through method 'dataSource' parameter 0: Error creating bean with name 'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties': Proxy class defined by interfaces [interface org.springframework.beans.factory.annotation.Qualifier, interface org.springframework.core.annotation.SynthesizedAnnotation] not found. Generating proxy classes at runtime is not supported. Proxy classes need to be defined at image build time by specifying the list of interfaces that they implement. To define proxy classes use -H:DynamicProxyConfigurationFiles=<comma-separated-config-files> and -H:DynamicProxyConfigurationResources=<comma-separated-config-resources> options.
	at org.springframework.beans.factory.aot.BeanInstanceSupplier.resolveArgument(BeanInstanceSupplier.java:349) ~[na:na]
	at org.springframework.beans.factory.aot.BeanInstanceSupplier.resolveArguments(BeanInstanceSupplier.java:265) ~[na:na]
	at org.springframework.beans.factory.aot.BeanInstanceSupplier.get(BeanInstanceSupplier.java:208) ~[na:na]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.obtainInstanceFromSupplier(AbstractAutowireCapableBeanFactory.java:1224) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.obtainFromSupplier(AbstractAutowireCapableBeanFactory.java:1209) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1156) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:566) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:526) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:930) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:926) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:592) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731) ~[flyway:3.0.0-SNAPSHOT]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:430) ~[flyway:3.0.0-SNAPSHOT]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) ~[flyway:3.0.0-SNAPSHOT]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[flyway:3.0.0-SNAPSHOT]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[flyway:3.0.0-SNAPSHOT]
	at com.example.flyway.FlywayApplication.main(FlywayApplication.java:13) ~[flyway:na]
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties': Proxy class defined by interfaces [interface org.springframework.beans.factory.annotation.Qualifier, interface org.springframework.core.annotation.SynthesizedAnnotation] not found. Generating proxy classes at runtime is not supported. Proxy classes need to be defined at image build time by specifying the list of interfaces that they implement. To define proxy classes use -H:DynamicProxyConfigurationFiles=<comma-separated-config-files> and -H:DynamicProxyConfigurationResources=<comma-separated-config-resources> options.
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:611) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:526) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:254) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1374) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1294) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.aot.BeanInstanceSupplier.resolveArgument(BeanInstanceSupplier.java:332) ~[na:na]
	... 20 common frames omitted
Caused by: com.oracle.svm.core.jdk.UnsupportedFeatureError: Proxy class defined by interfaces [interface org.springframework.beans.factory.annotation.Qualifier, interface org.springframework.core.annotation.SynthesizedAnnotation] not found. Generating proxy classes at runtime is not supported. Proxy classes need to be defined at image build time by specifying the list of interfaces that they implement. To define proxy classes use -H:DynamicProxyConfigurationFiles=<comma-separated-config-files> and -H:DynamicProxyConfigurationResources=<comma-separated-config-resources> options.
	at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:89) ~[na:na]
	at com.oracle.svm.reflect.proxy.DynamicProxySupport.getProxyClass(DynamicProxySupport.java:158) ~[na:na]
	at java.lang.reflect.Proxy.getProxyConstructor(Proxy.java:48) ~[flyway:na]
	at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:1037) ~[flyway:na]
	at org.springframework.core.annotation.SynthesizedMergedAnnotationInvocationHandler.createProxy(SynthesizedMergedAnnotationInvocationHandler.java:305) ~[na:na]
	at org.springframework.core.annotation.TypeMappedAnnotation.createSynthesizedAnnotation(TypeMappedAnnotation.java:333) ~[na:na]
	at org.springframework.core.annotation.AbstractMergedAnnotation.synthesize(AbstractMergedAnnotation.java:210) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.core.annotation.AbstractMergedAnnotation.synthesize(AbstractMergedAnnotation.java:200) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.core.annotation.AnnotationUtils.getAnnotation(AnnotationUtils.java:227) ~[na:na]
	at org.springframework.core.annotation.AnnotationUtils.getAnnotation(AnnotationUtils.java:252) ~[na:na]
	at org.springframework.beans.factory.annotation.BeanFactoryAnnotationUtils.isQualifierMatch(BeanFactoryAnnotationUtils.java:182) ~[na:na]
	at org.springframework.beans.factory.annotation.BeanFactoryAnnotationUtils.qualifiedBeansOfType(BeanFactoryAnnotationUtils.java:68) ~[na:na]
	at org.springframework.boot.context.properties.ConversionServiceDeducer$ConverterBeans.beans(ConversionServiceDeducer.java:99) ~[na:na]
	at org.springframework.boot.context.properties.ConversionServiceDeducer$ConverterBeans.<init>(ConversionServiceDeducer.java:93) ~[na:na]
	at org.springframework.boot.context.properties.ConversionServiceDeducer.getConversionServices(ConversionServiceDeducer.java:65) ~[na:na]
	at org.springframework.boot.context.properties.ConversionServiceDeducer.getConversionServices(ConversionServiceDeducer.java:55) ~[na:na]
	at org.springframework.boot.context.properties.ConfigurationPropertiesBinder.getConversionServices(ConfigurationPropertiesBinder.java:183) ~[flyway:na]
	at org.springframework.boot.context.properties.ConfigurationPropertiesBinder.getBinder(ConfigurationPropertiesBinder.java:168) ~[flyway:na]
	at org.springframework.boot.context.properties.ConfigurationPropertiesBinder.bind(ConfigurationPropertiesBinder.java:95) ~[flyway:na]
	at org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor.bind(ConfigurationPropertiesBindingPostProcessor.java:89) ~[flyway:na]
	at org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor.postProcessBeforeInitialization(ConfigurationPropertiesBindingPostProcessor.java:78) ~[flyway:na]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:425) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1745) ~[flyway:6.0.0-SNAPSHOT]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:604) ~[flyway:6.0.0-SNAPSHOT]
	... 29 common frames omitted
@mhalbritter mhalbritter added type: enhancement A general enhancement theme: aot An issue related to Ahead-of-time processing labels Aug 8, 2022
@mhalbritter mhalbritter added this to the 3.0.x milestone Aug 8, 2022
@mhalbritter
Copy link
Contributor Author

When this is fixed, i expect more flyway problems. Flyway enumerates migration files from the classpath, which is not supported on GraalVM.

@mhalbritter mhalbritter mentioned this issue Aug 10, 2022
Closed
@mhalbritter
Copy link
Contributor Author

Stephane will add the proxy hints to Spring Framework, as this proxy hint is not specific to @FlywayDataSource but for all annotations meta-annotated with @Qualifier.

@snicoll snicoll mentioned this issue Aug 23, 2022
Closed
@mhalbritter
Copy link
Contributor Author

I've created a PR to the reachability repo for Flyway. In the tests there is a custom ResourceProvider which is needed because GraalVM doesn't support enumerating classpath resources. I guess we have to add something to Boot which gathers the migration files in the AOT phase, and then either generates code which contains the list of migration files, or writes a file, which is then read by a custom ResourceProvider.

@mhalbritter mhalbritter self-assigned this Aug 24, 2022
@mhalbritter
Copy link
Contributor Author

Even with the PR merged, Flyway still fails because it thinks there is no SLF4J available and tries to use Log4J2 as a logging system. I created another PR on the reachability repo to fix that for logback.

Besides that, i played around with migration finding in a native image (because classpath enumeration doesn't work :/), you can view the code here. I've implemented two approaches:

  1. find migrations in AOT phase and write them to an index file, which is later used to find the migrations
  2. find migrations in AOT phase and generate code for a bean which then knows how to find the migrations

@mhalbritter
Copy link
Contributor Author

mhalbritter commented Aug 24, 2022 •
edited

After a review from Stephane:

Problems with the PoC: it uses a BeanFactoryInitializationAotProcessor instead of a BeanRegistrationAotProcessor. Using a BeanRegistrationAotProcessor would allow to react on the presence of the default Flyway ResourceProvider bean, and then replace that bean with code, which news up a ResourceProvider with a fixed list of migrations.

Another problem is that the code re-implements how Flyway finds the migrations (using classpath enumeration). If would be much better if Flyway would provide an API, which we could call in AOT phase to get the list of migrations, which we then feed into our custom ResourceProvider. Right now we risk that Flyway does something subtle different and we don't include all migrations, or include too much, etc.

Generally speaking, solution 2 (code generation) is the way to go, solution 1 (index file) can be dropped.

Problems with the new approach:

  • The default ResourceProvider in Flyway seems to be Scanner, which is really hard to new. We can workaround this by introducing another signalling type, like PersistenceManagedTypes done in JPA support.

I'm going to investigate how to refactor the PoC to accomodate the changes.

This was referenced Aug 24, 2022
Closed
Open
@mhalbritter
Copy link
Contributor Author

I've opened / commented on Flyway and GraalVM issues to get Flyway working without us having to write a bunch of workarounds.

@mhalbritter
Copy link
Contributor Author

mhalbritter commented Aug 25, 2022 •
edited

Even if GraalVM team / Flyway team doesn't want to support it by themselves, I found a better workaround than to look for the migrations at AOT phase: Implement a Flyway custom ResourceProvider which uses the resource: scheme from GraalVM to find the migrations. The PoC is here: https://github.com/mhalbritter/flyway-native-image

All that is left with this ResourceProvider in place is resource hints on db/migration/*.sql (or whatever the user has configured).

Drawback of that approach is that, when the custom ResourceProvider is in place, it get's called for all migration locations, maybe even one with filesystem: or s3: prefixes. It would be nice to delegate this back to the default ResourceProvider, but it doesn't seem like there is an easy way to do that.

@mhalbritter
Copy link
Contributor Author

I'll put this on hold until either Flyway or GraalVM reacts on the issue.

@mhalbritter mhalbritter added the status: on-hold We can't start working on this issue yet label Aug 25, 2022
@mhalbritter
Copy link
Contributor Author

So far I got no reaction from the Flyway team regarding GraalVM.

I'm currently playing around with Flyway in native-image and may have found a way to use their Scanner to get support for AWS, file, etc. and only fall back to our own implementation when running in a native-image and reading classpath: locations, using the new functionality in PathMatchingResourcePatternResolver to list resources in a native-image.

@mhalbritter
Copy link
Contributor Author

mhalbritter commented Oct 17, 2022 •
edited

I've got something working here: https://github.com/mhalbritter/sb3-native-flyway-poc-2

There's a ResourceProviderCustomizer which is used when not running in AOT / native image. It's a FlywayConfigurationCustomizer and is called by the Spring Boot FlywayAutoconfiguration when Flyway is set up. This ResourceProviderCustomizer is a noop implementation and does nothing. Its only purpose is to be a signal for the FlywayBeanRegistrationAotProcessor to kick in when running in AOT mode. This FlywayBeanRegistrationAotProcessor replaces the no-op ResourceProviderCustomizer with NativeImageResourceProviderCustomizer. When Spring Boot executes this customizer, it checks if the user has provided their own ResourceProvider, and if not, installs NativeImageResourceProvider.

The NativeImageResourceProvider uses the default Flyway Scanner to support S3, Google Cloud etc. and additionally uses the PathMatchingResourcePatternResolver to find migration files when running in a native image. When running in AOT mode on the JVM, it essentially delegates all work to the Flyway Scanner.

@mhalbritter
Copy link
Contributor Author

One caveat: Flyway logs this:

2022-10-17T12:04:43.155+02:00  WARN 182256 --- [           main] o.f.core.internal.util.FeatureDetector   : Unable to scan location: /db/migration (unsupported protocol: resource)
2022-10-17T12:04:43.155+02:00  WARN 182256 --- [           main] o.f.core.internal.util.FeatureDetector   : Unable to scan location: /db/migration2 (unsupported protocol: resource)
2022-10-17T12:04:43.155+02:00  WARN 182256 --- [           main] o.f.c.i.s.classpath.ClassPathScanner     : Unable to scan location: /db/migration (unsupported protocol: resource)
2022-10-17T12:04:43.155+02:00  WARN 182256 --- [           main] o.f.c.i.s.classpath.ClassPathScanner     : Unable to scan location: /db/migration2 (unsupported protocol: resource)

this is due to the fact that the NativeImageResourceProviderCustomizer passes all locations to Flyways Scanner. We could remove the classpath locations from the locations passed to the Scanner when running in a native image to get rid of those warnings.

@bclozel
Copy link
Member

bclozel commented Oct 17, 2022

Besides the WARN logs, this looks indeed like a viable solution for Flyway support.
I'm wondering if the FlywayBeanRegistrationAotProcessor step is needed and if always adding the NativeImageResourceProviderCustomizer with a native image check would work - I guess doing this has drawbacks that I'm currently missing?
Pinging @snicoll to know if there are better patterns we could use for this.

@mhalbritter
Copy link
Contributor Author

mhalbritter commented Oct 17, 2022 •
edited

I'm not 100% convinced that the scanner created by NativeImageResourceProviderCustomizer behaves exactly the same as the default one from Flyway. That's the reason I was hesitant to always use the NativeImageResourceProviderCustomizer on JVM and on native image.

@mhalbritter
Copy link
Contributor Author

We could remove the classpath locations from the locations passed to the Scanner when running in a native image to get rid of those warnings.

This, unfortunately, doesn't remove the warnings from ClassPathScanner, only the one from FeatureDetector.

@philwebb philwebb removed the status: on-hold We can't start working on this issue yet label Oct 17, 2022
mhalbritter added a commit to mhalbritter/spring-boot that referenced this issue Oct 18, 2022
@mhalbritter
Add Flyway native-image support
f0e72f0 
The ResourceProviderCustomizer, which is used by FlywayAutoConfiguration
gets replaced with NativeImageResourceProviderCustomizer when running
in AOT mode. The NativeImageResourceProvider does the heavy lifting when
running in a native image: it uses PathMatchingResourcePatternResolver
to find the migration files.

Closes spring-projectsgh-31999
@mhalbritter mhalbritter modified the milestones: 3.0.x, 3.0.0-RC1 Oct 18, 2022
@s4l4r
Copy link

s4l4r commented Sep 8, 2023

Came along this warning too, tho all migrations are scanned and applied successfully. Is this just a warning or can it cause issues at later stages?

@mhalbritter
Copy link
Contributor Author

I'm not aware of any problems related to the warnings. As I said in #31999 (comment), this is due to the fact that we don't disable the built-in flyway scanner, which doesn't support native-image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mhalbritter mhalbritter

Labels
theme: aot An issue related to Ahead-of-time processing type: enhancement A general enhancement
Projects
None yet
Milestone
3.0.0-RC1
Development

No branches or pull requests
4 participants
@bclozel @philwebb @s4l4r @mhalbritter