setting 'executable' flag to true in results in WAR file that jar -xvf silently fails to unpack

[timkettering]

Apologies if this has been raised and answered, I tried searching for past issues and found some slightly tangential issues but hoping to get an firm answer on this unusual behavior.

I have a spring-boot project which uses gradle and the spring-boot plugin: which generates two WAR files to build/lib. If I have the springBoot executable property set to true in the gradle task configuration, then the fat WAR file (the one that is not marked with .original suffix), cannot be expanded using jar -xvf. It just returns immediately with no obvious error and nothing is expanded.

Setting the executable flag to false results in a WAR file that can be expanded with jar -xvf.

I have tested and found that using the unzip command will expand the WAR successfully, regardless of the executable flag.

Also have observed that the .original WAR file expands successfully regardless of executable flag.

This unusual behavior seems to be the cause other issues we have been encountering, such as unable to deploy the fat WAR to jetty (jetty fails to expand the WAR), while Tomcat expands the WAR without any trouble.

This also causes issues in a part of our build script where we try to unpack the WAR to insert additional assets. The unpack part fails if exectuable = true. Setting executable to false enables the build script to work fine.

I am not sure if this is a known issue but the extremely silent way it fails is causing a lot of issues in our build and deployment process that all immediately goes away if we set exectuable=false.

We are on spring-boot-gradle plugin 1.5.1 currently, we have observed this issue on older 1.4.x plugins as well.

If any additional questions or further info desired, be happy to answer.

[philwebb]

The executable jar trick works by appending a small script to the front of the archive. Well behaved zip tools should work by reading the zip file from the end, then working backwards. We've seen issues with some libraries that don't do this. For example, the zip library in Go doesn't deal well with these files (we needed to make changes to the Cloud Foundry CLI to add support)

I thought that jar used to work, but I've tested things locally and it indeed just fails silently. It's not related to your build, or to the fact you're generating a .war file, it just seems to be a limitation of the tool.

We can raise an issue with Open JDK to see if they'll consider extending the tool to support the unusual format. Other than that, the best we can do is to add some more documentation warning of the limitations. If you aren't actively using the executable support, I'd suggest living without it.

[timkettering]

@philwebb - thanks for the clarification! my working assumption was that modifying the JAR to make it executable somehow made it unpackable by the jar tool.

Although if that is the case, then I think its quite dangerous to advertise it as a WAR/JAR which is executable because if Java's own tool cannot unpack it, then it really isn't a valid JAR in my definition.

Yes I would agree that at the very minimum this needs to be clearly documented. I spent countless hours last week and this week trying to boil down our build issue to this one little flag, and along the way I have observed that this behavior breaks assumptions in the following:

1. placing the WAR in jetty causes it to try to deploy but fails to deploy.
2. ant.unzip task fails. (failure message says the .war file does not exist when it actually does which is even more misleading)
3. ant.zip (with update=true) still wipes out original WAR and writes new WAR.
4. using zip task in gradle fails because of ^^
5. jar tool fails with no indication why.

Those are very commonly used tools in the developer community, and it is of my personal opinion that the executable webapp should not be advertised as an .war/ .jar if it causes those tools to behave in completely unexpected ways. They should be labelled differently or the incompatibility is broadcast to the end user clearly.

[philwebb]

Those are very commonly used tools in the developer community, and it is of my personal opinion that the executable webapp should not be advertised as an .war/ .jar if it causes those tools to behave in completely unexpected ways.

Are you suggesting a different file extension? The problem with that is that tools that do work with executable jars won't recognize our new extension.

[philwebb]

BTW, there is already a warning in the docs. Perhaps we just need to make it clearer.

[timkettering]

@philwebb - i recognize that changing the extension would raise a whole other set of problems, I don't have an easy answer for you to be honest! (and yes: that warning should be in bold, red and specifically say that tools relying on jar to read or unpack will not work.)

But I wanted to stress that this kind of problem can be potentially huge time-sinks to developer teams, if my example is anything to go by... where a project has been using the executable flag to generate and deploy artifacts with success for quite a while, then later on has a need to build an slightly modified WAR and then without any clear reason, zip tasks aren't working on WAR files that supposedly don't exist, and nobody on the team has any reason to suspect an issue with spring-boot yet.

[timkettering]

Maybe a warning message in the gradle build output indicating that an WAR that may be incompatible with tools is being built, and a link to relevant documentation?

[philwebb]

That's an option, but then again people that do know the risks and are not post-processing the war will probably get annoyed.

Out of interest why did you switch on executable in the first place? Where you experimenting with running via ./my.war rather than java -jar my.war, or was it that you thought it was needed to set that flag to have executable wars in any form?

[timkettering]

Out of interest why did you switch on executable in the first place? Where you experimenting with running via ./my.war rather than java -jar my.war, or was it that you thought it was needed to set that flag to have executable wars in any form?

When the project was first formed, it was set up to generate the executable war and then placed in a docker container and the application was launched by invoking the executable. That worked well for over six months so of course, any reasons or warnings regarding the use of the executable flag (if they were even noted to start with) was forgotten by now.