Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PathMatchingResourcePatternResolver provides duplicate resources when using classpath* prefix combined with ant-style [SPR-15989] #20539

Closed
spring-issuemaster opened this issue Sep 21, 2017 · 12 comments

Comments

Projects
None yet
2 participants
@spring-issuemaster
Copy link
Collaborator

commented Sep 21, 2017

Maël OUAHI opened SPR-15989 and commented

PathMatchingResourcePatternResolver provides duplicate resources when using "classpath*" prefix combined with Ant-style patterns.

I'll attach an application that reproduces the problem.The application uses the location pattern "classpath*:/*-perm.json" to resolve Resources.
If you package and run it

mvn clean package && java -jar target/resources-0.0.1-SNAPSHOT.jar

You should see the following output:

file:/D:/resources/target/resources-0.0.1-SNAPSHOT.jar!/
file:D:\resources\target\resources-0.0.1-SNAPSHOT.jar!/
file:/C:/Program%20Files/Java/jdk1.8.0_40/jre/lib/ext/dnsns.jar!/
...
file:/D:/resources/target/resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json
file:D:\resources\target\resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json

Affects: 4.3.8, 4.3.9, 4.3.10, 4.3.11

Attachments:

Issue Links:

  • #18260 PathMatchingResourcePatternResolver does not consider manifest based classpaths
  • #20665 PathMatchingResourcePatternResolver returns duplicate resources when using classpath* prefix

Referenced from: commits 9d8e3d4, a839477

Backported to: 4.3.12

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 22, 2017

Juergen Hoeller commented

Running the attached app, all I can see are regular entries along the lines of D:\resources\target in your output - no duplicates here, I'm afraid. There have to be some specific conditions involved in your setup, so please try to debug PathMatchingResourcePatternResolver accordingly: Somewhere in doFindAllClassPathResources, possibly in addAllClassLoaderJarRoots / addClassPathManifestEntries, the same jar might be seen twice but with different URL format.

I'm pretty sure this only happens when searching for such patterns in the root of your jar files... since the root of such files is unfortunately not easily discoverable with a standard ClassLoader (since ClassLoader.getResources("") is not guaranteed to deliver a complete result). As a consequence, it is usually recommendable to put such files into a specific subdirectory (even META-INF) in each jar file which can be more reliably searched through a standard call (e.g. ClassLoader.getResources("META-INF")).

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 25, 2017

Christophe Blanchard commented

I do reproduce the problem as well :

"C:\Program Files\Java\jdk1.8\bin\java.exe" -jar target/resources-0.0.1-SNAPSHOT.jar

file:/D:/tmp/resources/resources/target/resources-0.0.1-SNAPSHOT.jar!/
file:D:\tmp\resources\resources\target\resources-0.0.1-SNAPSHOT.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/access-bridge-64.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/cldrdata.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/dnsns.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/jaccess.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/jfxrt.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/localedata.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/nashorn.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/sunec.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/sunjce_provider.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/sunmscapi.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/sunpkcs11.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/zipfs.jar!/
file:/D:/tmp/resources/resources/target/resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json
file:D:\tmp\resources\resources\target\resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json

The problem does not appear when running with spring-boot-maven-plugin instead of maven-shade-plugin. weird...

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 25, 2017

Maël OUAHI commented

thanks for your reply, i confirm that the problem does not appear when i put the file into subdirectory, but i reproduce the problem when i run the app on windows 7 OS and centOs 7.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 25, 2017

Juergen Hoeller commented

I'm afraid I still can't reproduce the problem on Windows 10 here... but it indeed seems to be related to the maven-shade-plugin.

Any chance you could attach a debugger to this? Which Resource type are those duplicated entries? Which step in PathMatchingResourcePatternResolver creates them (I suppose addAllClassLoaderJarRoots / addClassPathManifestEntries)?

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2017

Maël OUAHI commented

I reproduce the problem when using maven-assembly-plugin to create the fat jar instead of maven-shade-plugin.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2017

Juergen Hoeller commented

I can reproduce the problem as well now: on JDK 8 but not on JDK 9 (which I was accidentally always running on before - sorry). On JDK 9, the system class loader never extends URLClassLoader but on JDK 8, it apparently may do so... even in combination with the "java.class.path" system property (which we're not taking into account in our code).

I'll try to sanitize the paths in our code accordingly before adding them to the result set, properly identifying the same jar there and only including it once.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2017

Juergen Hoeller commented

We're explicitly checking for duplicates in addClassPathManifestEntries now, with or without extra leading slash in the existing result (typically from URLClassLoader.getURLs()).

This is in master now for the 5.0 GA release. I'll also backport it to 4.3.12 later today. It'd be great if you could give 4.3.12.BUILD-SNAPSHOT a try once it's in; I'll let you know.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2017

Maël OUAHI commented

thanks for you responsiveness, let me know when backporting was done and i will try it.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 27, 2017

Maël OUAHI commented

It's OK. I tried it and this is the output

file:/D:/dev/workspaces/oxygen/hyperia/resources/target/resources-0.0.1-SNAPSHOT.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/access-bridge-64.jar!/
...
file:/D:/dev/workspaces/oxygen/hyperia/resources/target/resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json
@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 25, 2017

Udayakumar commented

I am facing exactly the same issue even with 5.0.1.RELEASE.
The pattern i am using is - classpath*:/**/*.rdrl to fetch all the .rdrl files inside jars in the class path. Basically we have multiple jars in the class path containing .rdrl file. Each file is returned two times one with / and another with \ in the path.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 25, 2017

Juergen Hoeller commented

Udayakumar, could you please create a new JIRA issue for your scenario? Please dump the full URLs that you're seeing, like above... Plain slash differences shouldn't matter since UrlResource.equals considers them equal, so there's probably something else involved. In any case, let's try to fix this for 5.0.2 / 4.3.13.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 25, 2017

Udayakumar commented

Thanks. opened a new issue.
#20665

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.