Zip resources are exploded into output Jar #196

Closed
RishikeshDarandale opened this Issue Mar 22, 2016 · 15 comments

Comments

Projects
None yet
3 participants
@RishikeshDarandale

If src.main.resources.mydir folder has sample.zip, then shadowJar skips these file while creating the FatJar.

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman Mar 22, 2016

Owner

Correct. This would be the expected behavior. Shadow is only for merging applications dependencies. a Jar file in src/main/resources is not an application dependency...it would be treated the same as a Javascript or CSS file.

If you want a local dependency, then you should be adding the jar as a File Dependency per https://docs.gradle.org/current/userguide/dependency_management.html#sub:file_dependencies

Owner

johnrengelman commented Mar 22, 2016

Correct. This would be the expected behavior. Shadow is only for merging applications dependencies. a Jar file in src/main/resources is not an application dependency...it would be treated the same as a Javascript or CSS file.

If you want a local dependency, then you should be adding the jar as a File Dependency per https://docs.gradle.org/current/userguide/dependency_management.html#sub:file_dependencies

@RishikeshDarandale

This comment has been minimized.

Show comment
Hide comment
@RishikeshDarandale

RishikeshDarandale Mar 22, 2016

Zip file added in resources is not dependency, its resource file which gets used in the application.

I do not have any local dependencies.

Zip file added in resources is not dependency, its resource file which gets used in the application.

I do not have any local dependencies.

@RishikeshDarandale

This comment has been minimized.

Show comment
Hide comment
@RishikeshDarandale

RishikeshDarandale Mar 22, 2016

Jar task does not skip these and includes on the jar file.

Jar task does not skip these and includes on the jar file.

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman Mar 22, 2016

Owner

Sorry, I originally misread your question.
So you are saying that a Jar file in src/main/resources is not included in the shadow jar as a single file, correct?

Is it getting exploded into its contents?

Owner

johnrengelman commented Mar 22, 2016

Sorry, I originally misread your question.
So you are saying that a Jar file in src/main/resources is not included in the shadow jar as a single file, correct?

Is it getting exploded into its contents?

@johnrengelman johnrengelman reopened this Mar 22, 2016

@RishikeshDarandale

This comment has been minimized.

Show comment
Hide comment
@RishikeshDarandale

RishikeshDarandale Mar 23, 2016

Not really. I have a resource file in format of zip, take an example of postal code. This file present under src/main/resources/postalCodes/US.zip; if you execute the jar task, then this resource file gets included in final jar file.

The same doesn't happen with shadowJar. Please refer shadow-example for further details.

$gradle clean build run

Here is the details:

$ jar -tvf ./shadow-example/build/libs/shadow-example.jar 
     0 Wed Mar 23 14:05:46 IST 2016 META-INF/
    25 Wed Mar 23 14:05:46 IST 2016 META-INF/MANIFEST.MF
     0 Wed Mar 23 14:05:46 IST 2016 sample/
   536 Wed Mar 23 14:05:46 IST 2016 sample/Sample.class
     0 Wed Mar 23 14:05:46 IST 2016 io/
     0 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/
     0 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/gradle/
  1944 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/gradle/Example.class
     0 Wed Mar 23 14:05:46 IST 2016 postalCodes/
650958 Wed Mar 23 14:05:46 IST 2016 postalCodes/US.zip

$ jar -tvf ./shadow-example/build/libs/shadow-example-all.jar
     0 Wed Mar 23 14:05:50 IST 2016 META-INF/
    75 Wed Mar 23 14:05:52 IST 2016 META-INF/MANIFEST.MF
     0 Wed Mar 23 14:05:46 IST 2016 sample/
   536 Wed Mar 23 14:05:46 IST 2016 sample/Sample.class
     0 Wed Mar 23 14:05:46 IST 2016 io/
     0 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/
     0 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/gradle/
  1944 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/gradle/Example.class
     0 Wed Mar 23 14:05:46 IST 2016 postalCodes/
  2372 Wed Mar 23 14:05:52 IST 2016 readme.txt
2779415 Wed Mar 23 14:05:52 IST 2016 US.txt
     0 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/
     0 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/com.google.guava/
     0 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/com.google.guava/guava/
   129 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/com.google.guava/guava/pom.properties
  6792 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/com.google.guava/guava/pom.xml
...

Not really. I have a resource file in format of zip, take an example of postal code. This file present under src/main/resources/postalCodes/US.zip; if you execute the jar task, then this resource file gets included in final jar file.

The same doesn't happen with shadowJar. Please refer shadow-example for further details.

$gradle clean build run

Here is the details:

$ jar -tvf ./shadow-example/build/libs/shadow-example.jar 
     0 Wed Mar 23 14:05:46 IST 2016 META-INF/
    25 Wed Mar 23 14:05:46 IST 2016 META-INF/MANIFEST.MF
     0 Wed Mar 23 14:05:46 IST 2016 sample/
   536 Wed Mar 23 14:05:46 IST 2016 sample/Sample.class
     0 Wed Mar 23 14:05:46 IST 2016 io/
     0 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/
     0 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/gradle/
  1944 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/gradle/Example.class
     0 Wed Mar 23 14:05:46 IST 2016 postalCodes/
650958 Wed Mar 23 14:05:46 IST 2016 postalCodes/US.zip

$ jar -tvf ./shadow-example/build/libs/shadow-example-all.jar
     0 Wed Mar 23 14:05:50 IST 2016 META-INF/
    75 Wed Mar 23 14:05:52 IST 2016 META-INF/MANIFEST.MF
     0 Wed Mar 23 14:05:46 IST 2016 sample/
   536 Wed Mar 23 14:05:46 IST 2016 sample/Sample.class
     0 Wed Mar 23 14:05:46 IST 2016 io/
     0 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/
     0 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/gradle/
  1944 Wed Mar 23 14:05:46 IST 2016 io/rishikeshdarandale/gradle/Example.class
     0 Wed Mar 23 14:05:46 IST 2016 postalCodes/
  2372 Wed Mar 23 14:05:52 IST 2016 readme.txt
2779415 Wed Mar 23 14:05:52 IST 2016 US.txt
     0 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/
     0 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/com.google.guava/
     0 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/com.google.guava/guava/
   129 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/com.google.guava/guava/pom.properties
  6792 Wed Mar 23 14:05:52 IST 2016 META-INF/maven/com.google.guava/guava/pom.xml
...
@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman Mar 23, 2016

Owner

Yeah, it's treating. The JAR as a dependency and exploding its contents. Your shadow jar has a US.txt file in the root of the jar. That's the contents from the US.zip file.

Owner

johnrengelman commented Mar 23, 2016

Yeah, it's treating. The JAR as a dependency and exploding its contents. Your shadow jar has a US.txt file in the root of the jar. That's the contents from the US.zip file.

@RishikeshDarandale

This comment has been minimized.

Show comment
Hide comment
@RishikeshDarandale

RishikeshDarandale Mar 23, 2016

Okay. what would be the workaround?

Okay. what would be the workaround?

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman Mar 23, 2016

Owner

The only workaround at the moment would be to include the zip file contents in your project instead of the zip file and referencing them that way.

Owner

johnrengelman commented Mar 23, 2016

The only workaround at the moment would be to include the zip file contents in your project instead of the zip file and referencing them that way.

@johnrengelman johnrengelman added the bug label Mar 23, 2016

@johnrengelman johnrengelman changed the title from shadowJar skips the zip files present under main.resources folder to Zip resources are exploded into output Jar Mar 23, 2016

@RishikeshDarandale

This comment has been minimized.

Show comment
Hide comment
@RishikeshDarandale

RishikeshDarandale Mar 25, 2016

For workaround, just change the extension from .zip to .zipx and it works!

Though, the code needs to modified to refer the file with extension .zipx and looks to be a simpler change.

$ jar -tvf ./build/libs/shadow-example.jar 
     0 Fri Mar 25 14:27:52 IST 2016 META-INF/
    25 Fri Mar 25 14:27:52 IST 2016 META-INF/MANIFEST.MF
     0 Fri Mar 25 14:27:52 IST 2016 sample/
   536 Fri Mar 25 14:27:52 IST 2016 sample/Sample.class
     0 Fri Mar 25 14:27:52 IST 2016 io/
     0 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/
     0 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/gradle/
  1945 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/gradle/Example.class
     0 Fri Mar 25 14:27:52 IST 2016 postalCodes/
650958 Fri Mar 25 14:27:52 IST 2016 postalCodes/US.zipx

$ jar -tvf ./build/libs/shadow-example-all.jar
     0 Fri Mar 25 14:27:54 IST 2016 META-INF/
    75 Fri Mar 25 14:27:54 IST 2016 META-INF/MANIFEST.MF
     0 Fri Mar 25 14:27:52 IST 2016 sample/
   536 Fri Mar 25 14:27:52 IST 2016 sample/Sample.class
     0 Fri Mar 25 14:27:52 IST 2016 io/
     0 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/
     0 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/gradle/
  1945 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/gradle/Example.class
     0 Fri Mar 25 14:27:52 IST 2016 postalCodes/
650958 Fri Mar 25 14:27:52 IST 2016 postalCodes/US.zipx
     0 Fri Mar 25 14:27:54 IST 2016 META-INF/maven/
     0 Fri Mar 25 14:27:54 IST 2016 META-INF/maven/com.google.guava/
     0 Fri Mar 25 14:27:54 IST 2016 META-INF/maven/com.google.guava/guava/
   129 Fri Mar 25 14:27:54 IST 2016 META-INF/maven/com.google.guava/guava/pom.properties
  6792 Fri Mar 25 14:27:56 IST 2016 META-INF/maven/com.google.guava/guava/pom.xml
...

For workaround, just change the extension from .zip to .zipx and it works!

Though, the code needs to modified to refer the file with extension .zipx and looks to be a simpler change.

$ jar -tvf ./build/libs/shadow-example.jar 
     0 Fri Mar 25 14:27:52 IST 2016 META-INF/
    25 Fri Mar 25 14:27:52 IST 2016 META-INF/MANIFEST.MF
     0 Fri Mar 25 14:27:52 IST 2016 sample/
   536 Fri Mar 25 14:27:52 IST 2016 sample/Sample.class
     0 Fri Mar 25 14:27:52 IST 2016 io/
     0 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/
     0 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/gradle/
  1945 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/gradle/Example.class
     0 Fri Mar 25 14:27:52 IST 2016 postalCodes/
650958 Fri Mar 25 14:27:52 IST 2016 postalCodes/US.zipx

$ jar -tvf ./build/libs/shadow-example-all.jar
     0 Fri Mar 25 14:27:54 IST 2016 META-INF/
    75 Fri Mar 25 14:27:54 IST 2016 META-INF/MANIFEST.MF
     0 Fri Mar 25 14:27:52 IST 2016 sample/
   536 Fri Mar 25 14:27:52 IST 2016 sample/Sample.class
     0 Fri Mar 25 14:27:52 IST 2016 io/
     0 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/
     0 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/gradle/
  1945 Fri Mar 25 14:27:52 IST 2016 io/rishikeshdarandale/gradle/Example.class
     0 Fri Mar 25 14:27:52 IST 2016 postalCodes/
650958 Fri Mar 25 14:27:52 IST 2016 postalCodes/US.zipx
     0 Fri Mar 25 14:27:54 IST 2016 META-INF/maven/
     0 Fri Mar 25 14:27:54 IST 2016 META-INF/maven/com.google.guava/
     0 Fri Mar 25 14:27:54 IST 2016 META-INF/maven/com.google.guava/guava/
   129 Fri Mar 25 14:27:54 IST 2016 META-INF/maven/com.google.guava/guava/pom.properties
  6792 Fri Mar 25 14:27:56 IST 2016 META-INF/maven/com.google.guava/guava/pom.xml
...

@johnrengelman johnrengelman added this to the 1.4.0 milestone Jun 23, 2016

johnrengelman added a commit that referenced this issue Jul 11, 2016

@johnrengelman johnrengelman modified the milestones: 1.2.4, 1.3.0 Nov 3, 2016

@Lanchon

This comment has been minimized.

Show comment
Hide comment
@Lanchon

Lanchon Jul 22, 2017

hi,

FYI, for the longest time i've been double-zipping my resources to stop shadow 1.3 1.2.3 from exploding my jars:
https://github.com/DexPatcher/dexpatcher-tool/blob/v1.2.0/tool/build.gradle#L67-L86

i'll try 1.4 at some point and report whether this has been fixed.

thanks!

Lanchon commented Jul 22, 2017

hi,

FYI, for the longest time i've been double-zipping my resources to stop shadow 1.3 1.2.3 from exploding my jars:
https://github.com/DexPatcher/dexpatcher-tool/blob/v1.2.0/tool/build.gradle#L67-L86

i'll try 1.4 at some point and report whether this has been fixed.

thanks!

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman Jul 22, 2017

Owner

@Lanchon this is fixed in shadow 1.2.4

Owner

johnrengelman commented Jul 22, 2017

@Lanchon this is fixed in shadow 1.2.4

@Lanchon

This comment has been minimized.

Show comment
Hide comment
@Lanchon

Lanchon Jul 22, 2017

oh, awful mind trickery converted my used version from 1.2.3 to 1.3, sorry!
ok i'll test it when i can, thank you!

Lanchon commented Jul 22, 2017

oh, awful mind trickery converted my used version from 1.2.3 to 1.3, sorry!
ok i'll test it when i can, thank you!

@Lanchon

This comment has been minimized.

Show comment
Hide comment
@Lanchon

Lanchon Jul 22, 2017

it doesn't seem to be fixed.

when replacing 1.2.3 for 1.2.4 i expected a single "dexpatcher-annotation-nested" jar to be included in the shadow jar, but no change: the contents of that jar are included in the shadow instead.

Lanchon commented Jul 22, 2017

it doesn't seem to be fixed.

when replacing 1.2.3 for 1.2.4 i expected a single "dexpatcher-annotation-nested" jar to be included in the shadow jar, but no change: the contents of that jar are included in the shadow instead.

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman Jul 22, 2017

Owner

Correct because this issue is for zip files not jars. Embedding a jar file into the shadow jar is not supported. There isn't a way for shadow at the moment to determine your intent here because the dependencies are jars too.

Owner

johnrengelman commented Jul 22, 2017

Correct because this issue is for zip files not jars. Embedding a jar file into the shadow jar is not supported. There isn't a way for shadow at the moment to determine your intent here because the dependencies are jars too.

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman Jul 22, 2017

Owner

See the referenced #111 for what you are tying to do.

Owner

johnrengelman commented Jul 22, 2017

See the referenced #111 for what you are tying to do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment