liquibase-core.jar contains ServiceLoader files

[madorb]

Description
Moving this over from Jira (https://liquibase.jira.com/browse/CORE-3522) as it seems like it should be fixed soon.

Starting with version 3.8.1 (observed with 3.8.2 as well), the liquibase-core.jar contains files in META-INF/services that belong to Jackson. This leads to runtime errors when Liquibase and Jackson are used in the same project. I've built a minimal example project at https://github.com/MenschNestor/liquibase-bug that exposes the issue, which revolves around the ServiceLoader mechanism in Java.

Acceptance Criteria

1. Please make sure that the JAR file does not contain foreign service provider configuration files.
2. Users who work with both Liquibase and Jackson in the same project can do so without errors

┆Issue is synchronized with this Jira Bug by Unito
┆fixVersions: Liquibase 3.10.1

[SteveDonie]

Thanks for moving this over! We will be moving more of the existing Jira issues here into github, we are still just working on how best to do that. I'll try to ensure this get prioritized.

[wutzebaer]

as a temporary fix you can add the dependencies to your project

	implementation 'com.fasterxml.jackson.core:jackson-databind'
	implementation 'com.fasterxml.jackson.module:jackson-module-jaxb-annotations'

[robinvanpraet]

This issue also results in our system not being able to startup again.
We are working in the java module system. And due to the fact that these ServiceLoader files (for another module) are here, the java module system fails to extract the module name.

Following warning is logged in maven:
[WARNING] Can't extract module name from liquibase-core-3.8.9.jar: Provider class com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule not in module

We are forced to remain at version 3.8.0 until this is fixed.

[jakubgondar]

@robinvanpraet Exactly the same with me. When working with Java 11 modules, application is not able to start. We also have to stick with version 3.8.0 until this is fixed.

[ro-rah]

Hello Team!

Thanks @madorb for bringing in this issue, and @SteveDonie , @robinvanpraet , @wutzebaer and @Gondy for responding. This issue has been added to an upcoming sprint and look to have this fixed in Q2.

-Ronak

[liquibot]

➤ Mario Champion commented:

This ticket depends on a spike ticket to determine and document a proposal from @nvoxland on dealing with the licensing which underpins the use of Jackson. That spike has been set for 7.4, so hopefully this gets resolved in the sprint after that.

[GreyTeardrop]

We have a similar impact: one of our apps uses both Liquibase and logstash-logback-encoder to format application logs as JSON. Logstash encoder uses ObjectMapper.findAndRegisterModules() under the hood that finds com.fasterxml.jackson.databind.Module file and chokes on it as there's no com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule referenced in that file on application classpath. As this happens very early on application startup in the logger code that turns into a very hard to debug "silent shutdown" kind of issue.

[benmadore-glassdoor]

just got bit by this again (and, of course completely forgot that this was the root cause, so had to re-investigate). Really really would love to see this fixed.

[dpratt]

I've had to fix this several times across a few projects internally, and I've had to explain it to a bunch of people.

This is really a fit and finish problem with the project - I love liquibase, I've used it heavily for nearly a decade now, but the fact that something that has been around this long and has such a dramatic effect hasn't been addressed is not a good sign. I really don't want to have to fork and internally publish a copy of the latest version with these files removed, and I suspect that I'm not the only one.

[madorb]

err, guess i was signed into work when i made that last comment, but yeah, 100% agree with @dpratt this is a really frustrating issue to have kicking around for this long. incredibly painful and difficult to diagnose. I wouldn't want to hazard a guess to how many person-hours have been lost sussing this out, and how many folks just gave up on liquibase when adding it to their project caused everything to fail in weird silent ways.

[ro-rah]

@dpratt and @madorb totally get the frustration being communicated, and I agree you are probably not alone in that. I too share that frustration, but we in the community team are working to:

• work through the backlog of existing PRs and issues (some very old, sorry for the neglect)
• streamline the process so that moving forward folks don't have to wait eons to realize a change

We are working through it right now with @nvoxland, thanks for sticking with this issue and walking with us through this journey of enabling our community.

[filiphr]

What I would suggest is to align the OSS release of Liquibase with what is in this repository. To me it looks like you have a really complex release pipeline that merges your Pro features and the Community version into one single jar.

If you want to keep using the same artifact for the pro and community maybe release a new one that would only contain the Community code and would be entirely build from this repo. Please let us know if you need some help with that.

[Treehopper]

I think this could be fixed with a one-line addition to the MANIFEST.MF: Automatic-Module-Name: liquibase.core Reference.
I just hacked the jar file in my local .m2, and it did work (using JDK14).
This is of course only for those who are affected by this issue (there may be other reasons to get rid of those files - haven't read all comments):

[WARNING] Can't extract module name from liquibase-core-3.8.9.jar: Provider class com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule not in module

However, since having this attribute is good practice anyway to avoid future breakage during modularization, and it would probably fix this issue for most people in a downward compatible manner, I wanted to mention it here in case there isn't already a better solution in progress.

[sync-by-unito]

➤ Erzsebet Carmean commented:

Liquibase 3.10.1-LB-162-SNAPSHOT, # 2Verify build excludes service loader files. PASS
Verify test project at https://github.com/MenschNestor/liquibase-bug no longer throws error with fix build. PASS

Output from Successful Maven Build{CODE}
C:\dev\liquibase-bug>mvn clean package exec:exec
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------< org.rewedigital:liquibase-bug >--------------------
[INFO] Building liquibase-bug 0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from maven.oracle.com: https://maven.oracle.com/org/liquibase/liquibase-core/3.10.1-LB-162-SNAPSHOT/maven-metadata.xml
Downloading from libs-snapshot-local: https://artifactory.datical.net/artifactory/libs-snapshot-local/org/liquibase/liquibase-core/3.10.1-LB-162-SNAPSHOT/maven-metadata.xml
Downloaded from libs-snapshot-local: https://artifactory.datical.net/artifactory/libs-snapshot-local/org/liquibase/liquibase-core/3.10.1-LB-162-SNAPSHOT/maven-metadata.xml (340 B at 195 B/s)
Downloading from libs-snapshot-local: https://artifactory.datical.net/artifactory/libs-snapshot-local/org/liquibase/liquibase-core/3.10.1-LB-162-SNAPSHOT/liquibase-core-3.10.1-LB-162-SNAPSHOT.pom
Downloaded from libs-snapshot-local: https://artifactory.datical.net/artifactory/libs-snapshot-local/org/liquibase/liquibase-core/3.10.1-LB-162-SNAPSHOT/liquibase-core-3.10.1-LB-162-SNAPSHOT.pom (2.6 kB at 10 kB/s)
Downloading from libs-snapshot-local: https://artifactory.datical.net/artifactory/libs-snapshot-local/org/liquibase/liquibase-core/3.10.1-LB-162-SNAPSHOT/liquibase-core-3.10.1-LB-162-SNAPSHOT.jar
Downloaded from libs-snapshot-local: https://artifactory.datical.net/artifactory/libs-snapshot-local/org/liquibase/liquibase-core/3.10.1-LB-162-SNAPSHOT/liquibase-core-3.10.1-LB-162-SNAPSHOT.jar (3.2 MB at 1.7 MB/s)
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ liquibase-bug ---
[INFO] Deleting C:\dev\liquibase-bug\target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ liquibase-bug ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\dev\liquibase-bug\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ liquibase-bug ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent!
[INFO] Compiling 1 source file to C:\dev\liquibase-bug\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ liquibase-bug ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\dev\liquibase-bug\src\test\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ liquibase-bug ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ liquibase-bug ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ liquibase-bug ---
[INFO] Building jar: C:\dev\liquibase-bug\target\liquibase-bug-0-SNAPSHOT.jar
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:exec (default-cli) @ liquibase-bug ---
[]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.959 s
[INFO] Finished at: 2020-06-30T11:02:34-05:00
[INFO] ------------------------------------------------------------------------
{CODE}

Thank you, @ madorb for the test project. It was very helpful. --erz (@ XDelphiGrl)

[sync-by-unito]

➤ Erzsebet Carmean commented:

No UAT; moving to ready to merge.

[madorb]

Awesome, glad to see this is getting resolved!!! I take no credit for the test project, i just moved the ticket over from Jira :-)