[JENKINS-51965] Add support of JAXB unpackaging for Java 9+ in WAR Exploder

[jenkins-infra-bot]

As discussed in JENKINS-51821, Jenkins won't work on Java 11 due to removed JAXB modules. I propose the following approach:

• Package 3 JAXB libs (jaxb-api-2.3.0.jar, jaxb-core-2.3.0.1.jar, jaxb-impl-2.3.0.1.jar) to a WAR resource which does not get classloaded by default. E.g. "WEB-INF/platform-compat/java11" (or 9 ?)
• When Jenkins starts up on Java 9+, Jenkins WAR exploder copies the libraries to "war/WEB-INF/lib" so that they get picked by classloader automatically

It will allow to have a transparent support of Java 9..11 within the WAR file (without adding modules, etc.)

A better way forward would be Multi-Version JAR, but we need a massive tooling update to support it

---

Originally reported by oleg_nenashev, imported from: Add support of JAXB unpackaging for Java 9+ in WAR Exploder

• assignee: kohsuke
• status: Closed
• priority: Minor
• component(s): core
• label(s): fosdem2019, java10_hackathon
• resolution: Won't Fix
• resolved: 2019-02-08T14:01:41+00:00
• votes: 0
• watchers: 7
• imported: 2025-11-24

Raw content of original issue

As discussed in JENKINS-51821, Jenkins won't work on Java 11 due to removed JAXB modules. I propose the following approach:

	
Package 3 JAXB libs (jaxb-api-2.3.0.jar, jaxb-core-2.3.0.1.jar, jaxb-impl-2.3.0.1.jar) to a WAR resource which does not get classloaded by default. E.g. "WEB-INF/platform-compat/java11" (or 9 ?)
	
When Jenkins starts up on Java 9+, Jenkins WAR exploder copies the libraries to "war/WEB-INF/lib" so that they get picked by classloader automatically



It will allow to have a transparent support of Java 9..11 within the WAR file (without adding modules, etc.)

A better way forward would be Multi-Version JAR, but we need a massive tooling update to support it

[jenkins-infra-bot]

• Remote link associated with this issue:

  • [JENKINS-51965] - Added JAXB plugin as a split dependency #3510

[jenkins-infra-bot]

kohsuke:

• Original comment link
• Raw content of original comment:

  "war exploder" is a Winstone local concept, so it isn't sufficient to make Jenkins work in other servlet containers, no?
  
   
  
  Why don't we just bundle JAXB in WEB-INF/lib instead?
  

"war exploder" is a Winstone local concept, so it isn't sufficient to make Jenkins work in other servlet containers, no?

 

Why don't we just bundle JAXB in WEB-INF/lib instead?

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  The recent JAXB versions require Java 9 or above.
  We we just bundle them, we won't be able to retain Java 8 compatibility in the same WARs.
  Shipping separate releases for Java 8 and Java 10+ is possible, but it is a serious overhead in terms of infrastructure changes needed.
  
  > "war exploder" is a Winstone local concept, so it isn't sufficient to make Jenkins work in other servlet containers, no?
  
  Yes, it will be limited to Jetty which is our "OOTB experience"
  For other Web containers keeping the module requirement is fine IMHO.
  

The recent JAXB versions require Java 9 or above.
We we just bundle them, we won't be able to retain Java 8 compatibility in the same WARs.
Shipping separate releases for Java 8 and Java 10+ is possible, but it is a serious overhead in terms of infrastructure changes needed.

> "war exploder" is a Winstone local concept, so it isn't sufficient to make Jenkins work in other servlet containers, no?

Yes, it will be limited to Jetty which is our "OOTB experience"
For other Web containers keeping the module requirement is fine IMHO.

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  kohsuke We also have other options which may be preferable && compatible
  
  
  	
  Cleanup usages of JAXB from Jenkins Core and Modules (like you proposed)
  	
  Create a new JAXB API Plugin
  	
  
  		
  The plugin is compatible with Java 8
  		
  The plugin implements multi-release JAR (will require tooling updates IIUC)
  		
  For Java 8 the plugin does nothing
  		
  For Java 9+ we bundle the libraries into the plugin
  	
  	
  	
  Release the plugin and mark it as detached from the core
  
  
  
  In such case we will have a clean WAR startup without JAXB modules + the plugin classloaders will be used to add the dependencies from JAXB API Plugin. It will allow running the approach in all Web containers.
  
  The main problem is that JAXB API Plugin will be present on many instances until we update all main plugins beyond the version with detached version. But it applies to all detached plugins so it should be fine
  
  WDYT?
  

kohsuke We also have other options which may be preferable && compatible

• Cleanup usages of JAXB from Jenkins Core and Modules (like you proposed)
• Create a new JAXB API Plugin
  • The plugin is compatible with Java 8
  • The plugin implements multi-release JAR (will require tooling updates IIUC)
  • For Java 8 the plugin does nothing
  • For Java 9+ we bundle the libraries into the plugin
• Release the plugin and mark it as detached from the core

In such case we will have a clean WAR startup without JAXB modules + the plugin classloaders will be used to add the dependencies from JAXB API Plugin. It will allow running the approach in all Web containers.

The main problem is that JAXB API Plugin will be present on many instances until we update all main plugins beyond the version with detached version. But it applies to all detached plugins so it should be fine

WDYT?

[jenkins-infra-bot]

kohsuke:

• Original comment link
• Raw content of original comment:

  Today we made a good progress on the proposed approach of detached JAXB plugin, but in the very end I discovered that this didn't work.
  
  The way plugins use JAXB makes JAXB to look up the implementation through thread context classloader, which is servlet classloader and not uber classloader. So it fails to find it. Plugins can invoke JAXB API a little differently, but it'd be nice if plugins need not make any changes.
  
  So after all that, I come full circle back to my original suggestion of just bundling JAXB in WEB-INF/lib. I discussed this with Paul, and this is a very common practice for JAXB, because JavaSE carries one version of it, EE container carries another version of it, and because those two platforms move very slowly, webapp often wants to use its own version of it. The servlet container is used to handle this kind of situation, and I'm saying this as a former JAXB RI & spec lead, so I know a thing or two about JAXB!
  
  We just need to bundle the version of JAXB that works with Java 8 (which is 2.2.x) in order to work in any Java 8+ runtime.
  
  I'll check in with Oleg tomorrow.
  
   
  
   
  

Today we made a good progress on the proposed approach of detached JAXB plugin, but in the very end I discovered that this didn't work.

The way plugins use JAXB makes JAXB to look up the implementation through thread context classloader, which is servlet classloader and not uber classloader. So it fails to find it. Plugins can invoke JAXB API a little differently, but it'd be nice if plugins need not make any changes.

So after all that, I come full circle back to my original suggestion of just bundling JAXB in WEB-INF/lib. I discussed this with Paul, and this is a very common practice for JAXB, because JavaSE carries one version of it, EE container carries another version of it, and because those two platforms move very slowly, webapp often wants to use its own version of it. The servlet container is used to handle this kind of situation, and I'm saying this as a former JAXB RI & spec lead, so I know a thing or two about JAXB!

We just need to bundle the version of JAXB that works with Java 8 (which is 2.2.x) in order to work in any Java 8+ runtime.

I'll check in with Oleg tomorrow.

 

 

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  kohsuke works for me if it works without breaking on Java 8, 10 and 11
  I doubt we want to tweak plugin classloading even more to have this plugin working somehow.
  

kohsuke works for me if it works without breaking on Java 8, 10 and 11
I doubt we want to tweak plugin classloading even more to have this plugin working somehow.

[jenkins-infra-bot]

jglick:

• Original comment link
• Raw content of original comment:

  thread context classloader, which is servlet classloader and not uber classloader
  
  I wonder if we should revisit that decision.
  
  Plugins can invoke JAXB API a little differently, but it'd be nice if plugins need not make any changes.
  
  Which plugins would actually be affected? It is maybe a five-line change to switch the thread context loader¹, and it would be nice to get JAXB out of core if we could.
  
  ¹And could be made even simpler with a utility in core:
  
  
  public final class WithJenkinsContextClassLoader implements AutoCloseable {
      private final Thread t;
      private final ClassLoader orig;
      public WithJenkinsContextClassLoader() {
          this(Jenkins.get().getPluginManager().uberClassLoader);
      }
      public WithJenkinsContextClassLoader(Class<?> pluginClass) {
          this(pluginClass.getClassLoader());
      }
      private WithJenkinsContextClassLoader(ClassLoader loader) {
          t = Thread.currentThread();
          orig = t.getContextClassLoader();
          t.setContextClassLoader(loader);
      }
      @Override public void close() {
          t.setContextClassLoader(orig);
      }
  }
  
  
  Thus:
  
  try (WithJenkinsContextClassLoader context = new WithJenkinsContextClassLoader(ThisClass.class)) {
      myUsageOfJAXB();
  }
  
  
  

thread context classloader, which is servlet classloader and not uber classloader

I wonder if we should revisit that decision.

Plugins can invoke JAXB API a little differently, but it'd be nice if plugins need not make any changes.

Which plugins would actually be affected? It is maybe a five-line change to switch the thread context loader¹, and it would be nice to get JAXB out of core if we could.

¹And could be made even simpler with a utility in core:

public final class WithJenkinsContextClassLoader implements AutoCloseable {
    private final Thread t;
    private final ClassLoader orig;
    public WithJenkinsContextClassLoader() {
this(Jenkins.get().getPluginManager().uberClassLoader);
    }
    public WithJenkinsContextClassLoader(Class pluginClass) {
this(pluginClass.getClassLoader());
    }
    private WithJenkinsContextClassLoader(ClassLoader loader) {
t = Thread.currentThread();
orig = t.getContextClassLoader();
t.setContextClassLoader(loader);
    }
    @​Override public void close() {
t.setContextClassLoader(orig);
    }
}

Thus:

try (WithJenkinsContextClassLoader context = new WithJenkinsContextClassLoader(ThisClass.class)) {
    myUsageOfJAXB();
}

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  I am still a fan of the "Multi-Release WAR" in this case (or of embedding bootstrap loading into the core). https://gist.github.com/rkx-bloodshed/badc6d0798b145123897a68ae4e3d6bd shows example of how modules could be loaded dynamically. Our main problem is that we may end up bundling all modules being detached from Java 11+ if we want to retain full compatibility going forward
  
  I suspect that we will need to tweak the Servlet Context Classloader anyway in this case (similar to the bootstrap code in https://github.com/jenkinsci/jenkins/pull/2754/files ?)
  
  > Which plugins would actually be affected?
  
  Many: https://github.com/search?l=Java&q=org%3Ajenkinsci+java.xml.bind&type=Code . There is a lot of test code and runtime annotations referenced there, so creating wrappers is unlikely a good option. 
  
   
  

I am still a fan of the "Multi-Release WAR" in this case (or of embedding bootstrap loading into the core). https://gist.github.com/rkx-bloodshed/badc6d0798b145123897a68ae4e3d6bd shows example of how modules could be loaded dynamically. Our main problem is that we may end up bundling all modules being detached from Java 11+ if we want to retain full compatibility going forward

I suspect that we will need to tweak the Servlet Context Classloader anyway in this case (similar to the bootstrap code in https://github.com/jenkinsci/jenkins/pull/2754/files ?)

> Which plugins would actually be affected?

Many: https://github.com/search?l=Java&q=org%3Ajenkinsci+java.xml.bind&type=Code . There is a lot of test code and runtime annotations referenced there, so creating wrappers is unlikely a good option. 

 

[jenkins-infra-bot]

batmat:

• Original comment link
• Raw content of original comment:

  I've worked a bit in the last few days on checking the current state of Jenkins without the --add-modules switches: I've tried looking for an easy case to reproduce this issue, but it's actually not anymore easy to make Jenkins fail for a JAXB error, which is good. That is mostly because since when the issues were filed, many were fixed in plac to not use JAXB anymore where this was possible.
  
  On the opensource side, it's close to impossible to check through the UI because of a few plugins have the majority javax.xml.bind imports
  https://github.com/search?q=org%3Ajenkinsci+%22javax.xml.bind%22&unscoped_q=%22javax.xml.bind%22, so I had to go through the GitHub API using curl and some shell scripting to get the full list.
  
  On the OSS side, the full list of repositories using "javax.xml.bind":
  
      286 "coverity-plugin"
      194 "inflectra-spira-integration-plugin"
      105 "zephyr-enterprise-test-management-plugin"
       24 "codebeamer-coverage-publisher-plugin"
       21 "codesonar-plugin"
       14 "java-client-api"
       13 "packageversion-plugin"
        9 "performance-signature-dynatrace-plugin"
        9 "hp-operations-orchestration-plugin"
        8 "meliora-testlab-plugin"
        8 "mabl-integration-plugin"
        8 "jelly"
        8 "app.io-plugin"
        7 "sloccount-plugin"
        7 "jira-issues-versioning-plugin"
        7 "dom4j"
        6 "extras-ec2-launcher"
        6 "dynatrace-plugin"
        5 "xlrelease-plugin"
        5 "xcode-plugin"
        5 "libdtkit"
        5 "github-coverage-reporter-plugin"
        4 "zanata-plugin"
        3 "cppcheck-plugin"
        2 "ucm4svn-plugin"
        2 "tfs-plugin"
        2 "stride-notification-plugin"
        2 "refit-plugin"
        2 "notification-plugin"
        2 "maven-metadata-plugin"
        2 "hp-quality-center-plugin"
        2 "ec2-deployment-dashboard"
        2 "docker-workflow-plugin"
        2 "docker"
        1 "windows-azure-storage-plugin"
        1 "timestamper-plugin"
        1 "testinium-plugin"
        1 "testcomplete-plugin"
        1 "testcomplete11-xunit-plugin"
        1 "qualityclouds-plugin"
        1 "policycenter-gate-validator-plugin"
        1 "pipeline-aws-plugin"
        1 "performance-plugin"
        1 "maven-hudson-dev-plugin"
        1 "maven-artifact-choicelistprovider-plugin"
        1 "localization-plugin"
        1 "klocwork-plugin"
        1 "jwsdp-sqe-plugin"
        1 "job-import-plugin"
        1 "jenkow-plugin"
        1 "jclouds-plugin"
        1 "jaxb-plugin"
        1 "ibm-cloud-devops-plugin"
        1 "fortify360-plugin"
        1 "dtkit-plugin"
        1 "agile-cockpit-notification-plugin"
        1 "acceptance-test-harness"
  
  
  Some notes
  
  	
  the biggest user above, coverity-plugin, is actually already deprecated: https://github.com/jenkinsci/coverity-plugin and replaced by https://github.com/jenkinsci/synopsys-coverity-plugin
  	
  the second one, inflectra-spira-integration-plugin, is installed 7 times worldwide https://plugins.jenkins.io/inflectra-spira-integration
  	
  https://plugins.jenkins.io/zephyr-enterprise-test-management is installed 163 times
  	
  codebeamer-coverage-publisher-plugin installed 62 times
  	
  codesonar has a decent install number (2200+). This plugin is maintained by Praqma, generally a very active contributor in the Jenkins community, so I filed https://issues.jenkins-ci.org/browse/JENKINS-55572 to see if this could be fixed there too
  	
  the rest look generally like plugins that are either not used a lot, or quite easily fixable given they use JAXB in very few places, ala [JENKINS-51965] Removing JAXB dependency instance-identity-plugin#11 for instance that was fixed by Kohsuke.
  
  Conclusion: 
  Given:
  
  	
  the long term right fix is likely to actually not need --add-modules anymore and allow plugins to do like nothing changed
  	
  we already somehow communicated the fact we were shifting gears, and interpreting to my advantage "we are willing to break some stuff to gain what we really need" . In other words: yes we could obviously fix this, but do we really need/want to spend more time on making JAXB usable like before when there seems to be no real need .
  
  I'm inclined to say we should not bother adapting JAXB in Jenkins so plugins can work like before forever. At least I hope my summary above shows there's no real traction to do this.
  I would rather encourage plugins to not depend on JAXB anymore.
  What do you think?
  

I've worked a bit in the last few days on checking the current state of Jenkins without the --add-modules switches: I've tried looking for an easy case to reproduce this issue, but it's actually not anymore easy to make Jenkins fail for a JAXB error, which is good. That is mostly because since when the issues were filed, many were fixed in plac to not use JAXB anymore where this was possible.

On the opensource side, it's close to impossible to check through the UI because of a few plugins have the majority javax.xml.bind imports
https://github.com/search?q=org%3Ajenkinsci+%22javax.xml.bind%22&unscoped_q=%22javax.xml.bind%22, so I had to go through the GitHub API using curl and some shell scripting to get the full list.

On the OSS side, the full list of repositories using "javax.xml.bind":

    286 "coverity-plugin"
    194 "inflectra-spira-integration-plugin"
    105 "zephyr-enterprise-test-management-plugin"
     24 "codebeamer-coverage-publisher-plugin"
     21 "codesonar-plugin"
     14 "java-client-api"
     13 "packageversion-plugin"
      9 "performance-signature-dynatrace-plugin"
      9 "hp-operations-orchestration-plugin"
      8 "meliora-testlab-plugin"
      8 "mabl-integration-plugin"
      8 "jelly"
      8 "app.io-plugin"
      7 "sloccount-plugin"
      7 "jira-issues-versioning-plugin"
      7 "dom4j"
      6 "extras-ec2-launcher"
      6 "dynatrace-plugin"
      5 "xlrelease-plugin"
      5 "xcode-plugin"
      5 "libdtkit"
      5 "github-coverage-reporter-plugin"
      4 "zanata-plugin"
      3 "cppcheck-plugin"
      2 "ucm4svn-plugin"
      2 "tfs-plugin"
      2 "stride-notification-plugin"
      2 "refit-plugin"
      2 "notification-plugin"
      2 "maven-metadata-plugin"
      2 "hp-quality-center-plugin"
      2 "ec2-deployment-dashboard"
      2 "docker-workflow-plugin"
      2 "docker"
      1 "windows-azure-storage-plugin"
      1 "timestamper-plugin"
      1 "testinium-plugin"
      1 "testcomplete-plugin"
      1 "testcomplete11-xunit-plugin"
      1 "qualityclouds-plugin"
      1 "policycenter-gate-validator-plugin"
      1 "pipeline-aws-plugin"
      1 "performance-plugin"
      1 "maven-hudson-dev-plugin"
      1 "maven-artifact-choicelistprovider-plugin"
      1 "localization-plugin"
      1 "klocwork-plugin"
      1 "jwsdp-sqe-plugin"
      1 "job-import-plugin"
      1 "jenkow-plugin"
      1 "jclouds-plugin"
      1 "jaxb-plugin"
      1 "ibm-cloud-devops-plugin"
      1 "fortify360-plugin"
      1 "dtkit-plugin"
      1 "agile-cockpit-notification-plugin"
      1 "acceptance-test-harness"

Some notes

• the biggest user above, coverity-plugin, is actually already deprecated: https://github.com/jenkinsci/coverity-plugin and replaced by https://github.com/jenkinsci/synopsys-coverity-plugin
• the second one, inflectra-spira-integration-plugin, is installed 7 times worldwide https://plugins.jenkins.io/inflectra-spira-integration
• https://plugins.jenkins.io/zephyr-enterprise-test-management is installed 163 times
• codebeamer-coverage-publisher-plugin installed 62 times
• codesonar has a decent install number (2200+). This plugin is maintained by Praqma, generally a very active contributor in the Jenkins community, so I filed https://issues.jenkins-ci.org/browse/JENKINS-55572 to see if this could be fixed there too
• the rest look generally like plugins that are either not used a lot, or quite easily fixable given they use JAXB in very few places, ala [JENKINS-51965] Removing JAXB dependency instance-identity-plugin#11 for instance that was fixed by Kohsuke.

Conclusion:

Given:

• the long term right fix is likely to actually not need --add-modules anymore and allow plugins to do like nothing changed
• we already somehow communicated the fact we were shifting gears, and interpreting to my advantage "we are willing to break some stuff to gain what we really need" . In other words: yes we could obviously fix this, but do we really need/want to spend more time on making JAXB usable like before when there seems to be no real need .

I'm inclined to say we should not bother adapting JAXB in Jenkins so plugins can work like before forever. At least I hope my summary above shows there's no real traction to do this.

I would rather encourage plugins to not depend on JAXB anymore.

What do you think?

[jenkins-infra-bot]

batmat:

• Original comment link
• Raw content of original comment:

  (BTW, I'm now thinking fixing the list above might be a good candidate for FOSDEM's hackfest. It's easy enough from a code perspective and then would allow people to go through the PR cycle, and a bit more code-oriented than the usual typo-fixes we do in such events)
  

(BTW, I'm now thinking fixing the list above might be a good candidate for FOSDEM's hackfest. It's easy enough from a code perspective and then would allow people to go through the PR cycle, and a bit more code-oriented than the usual typo-fixes we do in such events)

[jenkins-infra-bot]

batmat:

• Original comment link
• Raw content of original comment:

  A new update: after Oleg recommended I try the SlocCount plugin, I did. And it does fail as expected on Java 11 by default. I filed JENKINS-55620 to track this.
  
  So then I tried dropping directly jaxb-api-2.3.0.jar and jaxb-core-2.3.0.1.jar into various locations to see how it behaves. Everything worked fine.
  
  
  
   
  Java 8
  Java 11
  
  
  Plugin's WEB-INF/lib 
  OK
  OK
  
  
  Jenkins' war/WEB-INF/lib
  OK
  OK
  
  
  
  
  
  To summarize, it seems like hardcoding the dependency against JAXB directly in the plugin would be simpler. I'm not saying it will work in every single case, where/if a given plugin does something more convoluted, trying for instance to use JAXB instances coming from another plugin (which I think, but untested yet, would result in having the classes seen as different given classloaders would be different).
  
  Circling back to my previous point, and adding up to Jesse's comment above:
  
  which plugins would actually be affected? It is maybe a five-line change to switch the thread context loader¹, and it would be nice to get JAXB out of core if we could.
  
  
  	
  I think I provided some data about "which plugins would actually be affected?": the answer is likely "very few" or something along this.
  	
  "get JAXB out of core": given my testing on sloccount-plugin, I think this is possible.
  
  
  
  Now, I'm aware I've not tested the whole Jenkins ecosystem, but so I'm still wondering if this subject is as prevalent as we were imagining. I guess, again, the fact many fixes were pushed already makes this less and less of an issue.
  
  I'm eager to hear what people think.
  

A new update: after Oleg recommended I try the SlocCount plugin, I did. And it does fail as expected on Java 11 by default. I filed JENKINS-55620 to track this.

So then I tried dropping directly jaxb-api-2.3.0.jar and jaxb-core-2.3.0.1.jar into various locations to see how it behaves. Everything worked fine.

	Java 8	Java 11
Plugin's WEB-INF/lib	OK	OK
Jenkins' war/WEB-INF/lib	OK	OK

To summarize, it seems like hardcoding the dependency against JAXB directly in the plugin would be simpler. I'm not saying it will work in every single case, where/if a given plugin does something more convoluted, trying for instance to use JAXB instances coming from another plugin (which I think, but untested yet, would result in having the classes seen as different given classloaders would be different).

Circling back to my previous point, and adding up to Jesse's comment above:

which plugins would actually be affected? It is maybe a five-line change to switch the thread context loader¹, and it would be nice to get JAXB out of core if we could.

• I think I provided some data about "which plugins would actually be affected?": the answer is likely "very few" or something along this.
• "get JAXB out of core": given my testing on sloccount-plugin, I think this is possible.

Now, I'm aware I've not tested the whole Jenkins ecosystem, but so I'm still wondering if this subject is as prevalent as we were imagining. I guess, again, the fact many fixes were pushed already makes this less and less of an issue.

I'm eager to hear what people think.

[jenkins-infra-bot]

batmat:

• Original comment link
• Raw content of original comment:

  Filed JENKINS-55944 to track removing JAXB usages where possible.
  

Filed JENKINS-55944 to track removing JAXB usages where possible.

[jenkins-infra-bot]

batmat:

• Original comment link
• Raw content of original comment:

  AFAICT, this is superseded by JENKINS-55681, which landed in 2.163 as per [JENKINS-55681] - Make JAXB a detached plugin only on Java 11 #3865
  

AFAICT, this is superseded by JENKINS-55681, which landed in 2.163 as per #3865