Implemented support for file permissions in webjars #3
Conversation
|
I feel like I've spent half my life fighting with executable permissions on files in zips. So I'm glad that you were able to taste that slice of life. |
This also adds file permissions as per: webjars/webjars-locator-core#3 In order to fix: sbt/sbt-js-engine#20
|
I could never figure out how to set the exec bit on files in zips using the standard Java zip libs. In the end I've resorted to using the system zip (which is what sbt-native-packager does). One challenge with your approach is that somehow the Bower and NPM WebJars are going to need to auto-generate the One thing I didn't see in this PR was a test covering cases where the Before we merge this I'd like to verify that it is in fact hard to get Maven to use the native zip / preserve exec bits. If it turns out that is easy, then I think I'd prefer to go that route. |
|
Ok, I just checked, it looks like maven does, by default, preserve permissions in its jar files when it packages. I've just updated webjars/npm#2, and verified that the files are executable. I'll update this PR to ensure that permissions are maintained when extracted. |
|
I've updated the PR to simply preserve permissions coming from the webjar. Note, this required adding a dependency on commons-compress, to get the support for reading the permissions, and upgrading to Java 7, since the APIs for setting file permissions were added in Java 7. |
This ensures that permissions are maintained when webjars are extracted. It uses commons-compress, which supports ZipInfo extensions, to read permissions. It requires Java 7, since Java 7 is required to work with POSIX file permissions. Implementing this was not quite straight forward, there were several bugs in the webjars extractor, including: * Extracting a single webjar from a filesystem classpath didn't work * When extracting webjars, the first file encountered, if it wasn't a node module and non node module was requested, was ignored. This usually wouldn't cause a problem because the first file encountered usually was not part of the webjar. The jar file web jar extraction didn't really have any knowledge of what webjars it extracted, it assumed each jar file only contained one web jar (this doesn't seem like a good assumption to make, especially when fat jars are being used), and it meant hacks had to be done to determine the moduleId, and would have required more hacks to do the permissions. So I rewrote the jar file webjar extraction to first discover and enumerate one or more webjars from each jar file, then moduleId detection and permissions could be easily applied per webjar.
|
I'm not seeing the exec bit on the bins in the the latest Are you seeing something different? |
|
What version of maven are you using? I'm using 3.3.1. |
|
And... did you do a clean? |
|
Ah... I extracted with |
Implemented support for file permissions in webjars
|
Version |
|
Very cool! Thanks guys! @jamesward @jroper I thought jars normally showed up in Maven Central within an hour? I'm still not seeing |
|
@benmccann Looks like it is there now: https://repo1.maven.org/maven2/org/webjars/webjars-locator-core/0.24/ But the http://search.maven.org index lags quite a bit. |
|
Oh, I see. It's because I was checking for webjars-locator instead of webjars-locator-core. Looks like we'll have to use webjars-locator to use the new webjars-locator-core. I'll send you a PR for that |
|
We split out the RequireJS stuff and for backwards compat we had to move the core stuff to |
|
Sent the PR here for upgrading Play depends on I kind of wanted to get this fix into the 2.4 branch and it might be too big a change to switch it from |
|
Since Play doesn't use the RequireJS parts it would actually be nice to switch to |
|
For Play 2.4, it would be better to have a new webjars-locator. The problem is, if some dependency depends on an old, pre split webjars-locator, and we depend on webjars-locator-core, then both will end up on the classpath, which won't be good. |
|
That is true. I'll work with @benmccann to get a new version of |
Ideally, we'd use file permissions from the jar files. The problem is, there is no standard for storing file permissions in zip files - Linux and other unices use ZipInfo which has its own proprietary extensions, but these are not implemented by, for example, the JDK, so that would mean any webjar that wanted to carry permissions would need to use a custom zip implementation that did support it both to extract and then package its jar (meaning a very custom maven setup that overrides mavens default package mechanism), and then in webjars itelf we'd need to have a custom zip implementation.
Rather than fighting this uphill battle, and out of band mechanism, in the form of a .webjars.permissions file, is defined. This is a properties file mapping webjars paths to permissions.
Implementing this was not quite straight forward, there were several bugs in the webjars extractor, including:
The jar file web jar extraction didn't really have any knowledge of what webjars it extracted, it assumed each jar file only contained one web jar (this doesn't seem like a good assumption to make, especially when fat jars are being used), and it meant hacks had to be done to determine the moduleId, and would have required more hacks to do the permissions.
So I rewrote the jar file webjar extraction to first discover and enumerate one or more webjars from each jar file, then moduleId detection and permissions could be easily applied per webjar.