Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

classpath scanner does not find db.migration package or resources #1261

Closed
pabl0rg opened this issue Mar 29, 2016 · 18 comments
Closed

classpath scanner does not find db.migration package or resources #1261

pabl0rg opened this issue Mar 29, 2016 · 18 comments
Labels
Milestone

Comments

@pabl0rg
Copy link

@pabl0rg pabl0rg commented Mar 29, 2016

What version of Flyway are you using?

4.0

What operating system are you using?

MacOS

What did you do?

(Please include the content causing the issue, any relevant configuration settings, and the command you ran)
If i run the following (kotlin) class from a jar file (the db.migration package & resources are part of the same jar):

            val flyway = Flyway()
            flyway.baselineDescription = "Initialize Flyway (${Version.getVersion()})"
            flyway.setBaselineVersionAsString("1.9.30")
            val (url, user, password) = DBUser.getConnectionInfo("web")
            flyway.setDataSource(url, user, password)
            flyway.target = MigrationVersion.fromVersion(Version.getShortVersion())
            flyway.isBaselineOnMigrate = true
            flyway.migrate()

I get the following, confusing log messages:

INFO [org.flywaydb.core.internal.util.VersionPrinter] Flyway 4.0 by Boxfuse
INFO [org.flywaydb.core.internal.dbsupport.DbSupportFactory] Database: jdbc:postgresql://127.0.0.1:5432/fms (PostgreSQL 9.3)
DEBUG [org.flywaydb.core.Flyway] DDL Transactions Supported: true
DEBUG [org.flywaydb.core.Flyway] Schema: public
DEBUG [org.flywaydb.core.internal.util.FeatureDetector] Spring Jdbc available: false
DEBUG [org.flywaydb.core.internal.callback.SqlScriptFlywayCallback] Scanning for SQL callbacks ...
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning for classpath resources at 'classpath:db/migration' (Prefix: '', Suffix: '.sql')
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Determining location urls for classpath:db/migration using ClassLoader sun.misc.Launcher$AppClassLoader@4e0e2f2a ...
WARN [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Unable to resolve location classpath:db/migration
DEBUG [org.flywaydb.core.internal.command.DbValidate] Validating migrations ...
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning for classpath resources at 'classpath:db/migration' (Prefix: 'V', Suffix: '.sql')
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning for classpath resources at 'classpath:db/migration' (Prefix: 'R', Suffix: '.sql')
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning for classes at 'classpath:db/migration' (Implementing: 'org.flywaydb.core.api.migration.jdbc.JdbcMigration')
INFO [org.flywaydb.core.internal.command.DbValidate] Successfully validated 1 migration (execution time 00:00.042s)
DEBUG [org.flywaydb.core.internal.command.DbSchemas] Schema "public" already exists. Skipping schema creation.
DEBUG [org.flywaydb.core.internal.dbsupport.Table] Locking table "public"."schema_version"...
DEBUG [org.flywaydb.core.internal.dbsupport.Table] Lock acquired for table "public"."schema_version"
INFO [org.flywaydb.core.internal.command.DbMigrate] Current version of schema "public": 1.9.30
INFO [org.flywaydb.core.internal.command.DbMigrate] Schema "public" is up to date. No migration necessary.

The messages are very confusing because first there's a WARN saying the location of db/migration could not be found, and later an INFO message saying 2 migrations were validated. Either way, no migrations run.

If i unzip the jar file and then run the same class, the log messages are different and the migrations actually get run

INFO [org.flywaydb.core.internal.util.VersionPrinter] Flyway 4.0 by Boxfuse
INFO [org.flywaydb.core.internal.dbsupport.DbSupportFactory] Database: jdbc:postgresql://127.0.0.1:5432/fms (PostgreSQL 9.3)
DEBUG [org.flywaydb.core.Flyway] DDL Transactions Supported: true
DEBUG [org.flywaydb.core.Flyway] Schema: public
DEBUG [org.flywaydb.core.internal.util.FeatureDetector] Spring Jdbc available: false
DEBUG [org.flywaydb.core.internal.callback.SqlScriptFlywayCallback] Scanning for SQL callbacks ...
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning for classpath resources at 'classpath:db/migration' (Prefix: '', Suffix: '.sql')
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Determining location urls for classpath:db/migration using ClassLoader sun.misc.Launcher$AppClassLoader@73d16e93 ...
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning URL: file:/Users/juanliska/Documents/startrack/test/tmp/db/migration
DEBUG [org.flywaydb.core.internal.util.FeatureDetector] JBoss VFS v2 available: false
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.FileSystemClassPathLocationScanner] Scanning starting at classpath root in filesystem: /Users/juanliska/Documents/startrack/test/tmp/
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.FileSystemClassPathLocationScanner] Scanning for resources in path: /Users/juanliska/Documents/startrack/test/tmp/db/migration (db/migration)
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Filtering out resource: db/migration/R__New_perm_refpoint_groups.class (filename: R__New_perm_refpoint_groups.class)
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Found resource: db/migration/R__Log_something.sql
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Found resource: db/migration/V1.11__Add_geofence_group_tables.sql
DEBUG [org.flywaydb.core.internal.command.DbValidate] Validating migrations ...
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning for classpath resources at 'classpath:db/migration' (Prefix: 'V', Suffix: '.sql')
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning URL: file:/Users/juanliska/Documents/startrack/test/tmp/db/migration
DEBUG [org.flywaydb.core.internal.util.FeatureDetector] JBoss VFS v2 available: false
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Filtering out resource: db/migration/R__Log_something.sql (filename: R__Log_something.sql)
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Filtering out resource: db/migration/R__New_perm_refpoint_groups.class (filename: R__New_perm_refpoint_groups.class)
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Found resource: db/migration/V1.11__Add_geofence_group_tables.sql
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning for classpath resources at 'classpath:db/migration' (Prefix: 'R', Suffix: '.sql')
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning URL: file:/Users/juanliska/Documents/startrack/test/tmp/db/migration
DEBUG [org.flywaydb.core.internal.util.FeatureDetector] JBoss VFS v2 available: false
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Filtering out resource: db/migration/R__New_perm_refpoint_groups.class (filename: R__New_perm_refpoint_groups.class)
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Filtering out resource: db/migration/V1.11__Add_geofence_group_tables.sql (filename: V1.11__Add_geofence_group_tables.sql)
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Found resource: db/migration/R__Log_something.sql
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning for classes at 'classpath:db/migration' (Implementing: 'org.flywaydb.core.api.migration.jdbc.JdbcMigration')
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Scanning URL: file:/Users/juanliska/Documents/startrack/test/tmp/db/migration
DEBUG [org.flywaydb.core.internal.util.FeatureDetector] JBoss VFS v2 available: false
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Filtering out resource: db/migration/R__Log_something.sql (filename: R__Log_something.sql)
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Filtering out resource: db/migration/V1.11__Add_geofence_group_tables.sql (filename: V1.11__Add_geofence_group_tables.sql)
DEBUG [org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner] Found class: db.migration.R__New_perm_refpoint_groups
INFO [org.flywaydb.core.internal.command.DbValidate] Successfully validated 4 migrations (execution time 00:00.064s)
DEBUG [org.flywaydb.core.internal.command.DbSchemas] Schema "public" already exists. Skipping schema creation.
DEBUG [org.flywaydb.core.internal.dbsupport.Table] Locking table "public"."schema_version"...
DEBUG [org.flywaydb.core.internal.dbsupport.Table] Lock acquired for table "public"."schema_version"
INFO [org.flywaydb.core.internal.command.DbMigrate] Current version of schema "public": 1.9.30
INFO [org.flywaydb.core.internal.command.DbMigrate] Migrating schema "public" with repeatable migration Log something
DEBUG [org.flywaydb.core.internal.dbsupport.SqlScript] Found statement at line 1: INSERT into al_log (message) values ('Hello v1.9.30')
DEBUG [org.flywaydb.core.internal.dbsupport.SqlScript] Executing SQL: INSERT into al_log (message) values ('Hello v1.9.30')
DEBUG [org.flywaydb.core.internal.dbsupport.JdbcTemplate] Update Count: 1
DEBUG [org.flywaydb.core.internal.command.DbMigrate] Successfully completed migration of schema "public" with repeatable migration Log something
DEBUG [org.flywaydb.core.internal.metadatatable.MetaDataTableImpl] MetaData table "public"."schema_version" successfully updated to reflect changes
DEBUG [org.flywaydb.core.internal.dbsupport.Table] Locking table "public"."schema_version"...
DEBUG [org.flywaydb.core.internal.dbsupport.Table] Lock acquired for table "public"."schema_version"
INFO [org.flywaydb.core.internal.command.DbMigrate] Migrating schema "public" with repeatable migration New perm refpoint groups
...
DEBUG [org.flywaydb.core.internal.command.DbMigrate] Successfully completed migration of schema "public" with repeatable migration New perm refpoint groups
DEBUG [org.flywaydb.core.internal.metadatatable.MetaDataTableImpl] MetaData table "public"."schema_version" successfully updated to reflect changes
DEBUG [org.flywaydb.core.internal.dbsupport.Table] Locking table "public"."schema_version"...
DEBUG [org.flywaydb.core.internal.dbsupport.Table] Lock acquired for table "public"."schema_version"
INFO [org.flywaydb.core.internal.command.DbMigrate] Successfully applied 2 migrations to schema "public" (execution time 00:00.176s).
@axelfontaine
Copy link
Contributor

@axelfontaine axelfontaine commented Mar 29, 2016

Thanks for reporting. Please post a GitHub repo to reproduce this.

@pabl0rg
Copy link
Author

@pabl0rg pabl0rg commented Apr 9, 2016

Hi. You can find an example here https://github.com/pabl0rg/kobalt-mixed-example/

Steps:

  1. produce jar
    ./kobaltw assemble
  2. run from jar
    java -cp :./kobaltBuild/libs/mixed-example-0.1.jar com.guatec.KExample
    message will say validated 0 migrations.
  3. unzip the jar into tmp folder, then run
    java -cp :tmp com.guatec.KExample
    message will say validated 2 migrations and will have run them

@pabl0rg
Copy link
Author

@pabl0rg pabl0rg commented Apr 27, 2016

Hello, has anyone been able to run the example project to reproduce the error?

In another project, i've used this library which does a good job of scanning inside jars.
https://github.com/lukehutch/fast-classpath-scanner

@pabl0rg
Copy link
Author

@pabl0rg pabl0rg commented Jun 30, 2016

Hello this is still marked as waiting for feedback but i provided the feedback and an example almost three months ago. Has anyone been able to check out the example?

thanks!

@wagnerma
Copy link

@wagnerma wagnerma commented Aug 25, 2016

I observe a similar problem. I work on a java application with an in memory database. Flyway finds only the migration sql scripts in the filesystem or in the classpath of the application startet from eclipse. If the application is packaged into jars with the sql files included, flyway don't find them. As a workaround I copy the sql scripts from the jar to a temporary folder in the filesystem and all it fine. There seems a bug in the loading from the classpath from jar files. (I use version 4.0.3)

@pabl0rg
Copy link
Author

@pabl0rg pabl0rg commented Aug 25, 2016

Would you be interested in sharing the cost of sponsoring this fix?

@sandokandias
Copy link

@sandokandias sandokandias commented Oct 14, 2016

Same problem here, only works by filesystem.
SO: OS X El Capitan and archlinux.
Versions: 4.0, 4.0.1, 4.0.2, 4.0.3
Database: Postgresql 9.5
Application: Spring Boot

@pabl0rg
Copy link
Author

@pabl0rg pabl0rg commented Oct 14, 2016

My company has sponsored a fix for this and the last snapshot we got is working. So, hold tight for the next release which will finally have this fixed.

@EmilDafinov
Copy link

@EmilDafinov EmilDafinov commented Dec 12, 2016

@pabl0rg Just out of curiosity, what is the structure of your jar like? I just had a similar problem with a Spring Boot app, turned out to be because Spring Boot structures its fat jars in a non-standard way.

@pabl0rg
Copy link
Author

@pabl0rg pabl0rg commented Dec 12, 2016

The jar was a standard maven jar. The jar could have been modified to work with flyway but the flyway guys actually found there was a slight issue in how they were searching the classpath. They haven't released a new flyway due to lack of time, but they have this fixed in git.

@axelfontaine
Copy link
Contributor

@axelfontaine axelfontaine commented Jan 31, 2017

@pabl0rg Thanks again for sponsoring. Fixed.

@tasosz
Copy link

@tasosz tasosz commented May 25, 2017

Hi,
Has this fix been applied to any releases?

Thanks

@derabbink
Copy link

@derabbink derabbink commented May 25, 2017

I'm intermittently seeing - what looks to be - this issue with org.flywaydb:flyway-gradle-plugin:4.2.0.

@teixeluis
Copy link

@teixeluis teixeluis commented Jun 22, 2020

Hi, have this issue been actually fixed? In what release in that case?

@urbaez22
Copy link

@urbaez22 urbaez22 commented Sep 22, 2020

It seems like they haven't fixed the issue. I'm recently using Flyway v6.5.7 and is still happening, Flyway can't find the migrations inside the Spring Boot JAR.. Maybe it doesn't recognize the way Maven packs Spring Boot binary, mine are located in the route path BOOT-INF/classes/db/migration inside the JAR..
The key question is, why they closed the issue if it's not resolved..?

@teixeluis
Copy link

@teixeluis teixeluis commented Sep 22, 2020

Cheers @pabl0rg what is your take on this?

Best regards

@urbaez22
Copy link

@urbaez22 urbaez22 commented Sep 22, 2020

Well, finally I found a workaround..

I simply put:
flyway.locations=classpath:BOOT-INF/classes/db/migration

And Flyway was able to find and execute the migrations.

Hope this can help somebody else..

Cheers..

@neelam-007
Copy link

@neelam-007 neelam-007 commented Jan 4, 2021

In my case, the migration scripts are within another jar file within the main jar file. How can I set the location in such a case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants