[native-image] URLClassLoader getResourceAsStream returns null #1956
Comments
|
Did you include the resource while generating the native-image? See: https://github.com/oracle/graal/blob/master/substratevm/RESOURCES.md As a note: dynamic classloading / resource-loading from jars is not supported, because everything reachable must be available during generation-time. If the resource is unavailable/ unknown during generation-time, it will not be found. See: https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md |
|
@SergejIsbrecht No, the point of this code is to search through a collection of directories and .jar files for a file at runtime. The string consisting of directories and jar files is not known at compile time. |
|
Well, as I see it you are out of luck on this one. I also updated my previous answer. Please have a look at the limitation of native-image for further information: |
|
Note that the below program does exactly what I expected with import java.net.*;
import sun.misc.*;
import java.io.File;
import java.io.InputStream;
import java.io.IOException;
public class ClassLoader {
public static void main(String [] args) throws IOException {
String classpath = args[0];
URL[] urls = URLClassPath.pathToURLs(classpath);
URLClassPath cp = new URLClassPath(urls);
Resource r = cp.getResource("foo/bar.clj");
if (r != null) {
System.out.println(r.getInputStream());
}
}
}EDIT: this code as is doesn't work with native image JDK 11 since the package A workaround for me would be to re-implement |
|
IIUC, the official position has to do with dynamic "class" loading.
|
|
I had a similar need, but I never considered using a URLClassLoader to load resources out of JARs on disk. Instead I just used the JarFile API directly to read the JARs. That seems to work just fine in a native-image. |
|
@tjwatson That is also how I solved that problem. I basically replicated classpath functionality myself. |
|
I've tested the given examples. This issue does not longer exist. |
|
@jovanstevanovic I'm still seeing this problem with native image 22.3.1 and 23.0.0-dev: I think this may be due to the substitution: |
|
I tried to work around this by undo-ing the substitution, but I'm getting: Instead I wrote my own extension of // This file is mostly a workaround for https://github.com/oracle/graal/issues/1956
package babashka.impl;
import java.util.WeakHashMap;
import java.io.*;
import java.util.Objects;
import java.net.*;
import java.util.jar.*;
public class URLClassLoader extends java.net.URLClassLoader implements Closeable {
private WeakHashMap<Closeable,Void>
closeables = new WeakHashMap<>();
public URLClassLoader(java.net.URL[] urls) {
super(urls);
}
public URLClassLoader(java.net.URL[] urls, java.net.URLClassLoader parent) {
super(urls, parent);
}
public void _addURL(java.net.URL url) {
super.addURL(url);
}
// calling super.getResource() returned nil in native-image
public java.net.URL getResource(String name) {
return findResource(name);
}
// calling super.getResourceAsStream() returned nil in native-image
public InputStream getResourceAsStream(String name) {
Objects.requireNonNull(name);
URL url = getResource(name);
try {
if (url == null) {
return null;
}
URLConnection urlc = url.openConnection();
InputStream is = urlc.getInputStream();
if (urlc instanceof JarURLConnection) {
JarFile jar = ((JarURLConnection)urlc).getJarFile();
synchronized (closeables) {
if (!closeables.containsKey(jar)) {
closeables.put(jar, null);
}
}
} else {
synchronized (closeables) {
closeables.put(is, null);
}
}
return is;
} catch (IOException e) {
return null;
}
}
public java.util.Enumeration<java.net.URL> getResources(String name) throws java.io.IOException {
return findResources(name);
}
public void close() throws IOException {
super.close();
java.util.List<IOException> errors = new java.util.ArrayList<IOException>();
synchronized (closeables) {
java.util.Set<Closeable> keys = closeables.keySet();
for (Closeable c : keys) {
try {
c.close();
} catch (IOException ex) {
errors.add(ex);
}
}
closeables.clear();
}
if (errors.isEmpty()) {
return;
}
IOException firstEx = errors.remove(0);
for (IOException error: errors) {
firstEx.addSuppressed(error);
}
throw firstEx;
}
} |
|
Hey @borkdude, maybe I made a mistake. Could you please then create a GitHub repo with a minimalistic reproducer and a set of commands that I can use locally to reproduce? (native image agent, build and run) |
|
@jovanstevanovic Will do! |
|
@jovanstevanovic The repro is here: https://github.com/borkdude/native-image-1956-repro You can run the repro with https://github.com/borkdude/native-image-1956-repro/actions/runs/4084260102/jobs/7040753788 Note that the output is: and with native-image: Note that |
|
@jovanstevanovic Btw, the workaround I have above works perfectly for now and this issue may be an edge case that is pretty niche so if the answer is that I should be using the workaround that I currently have, I would be fine with that (as long as it keeps working in future releases!) |
|
I was on vacation for some time, so I'm back now. You're right, it is a problem from our side. Here is an internal ticket GR-44424 so that you can track it on master once it is done. |
|
Thank you! I'm subscribed to this Github issue but not sure how I can track master without reading the all commit messages on master 🤣 |
|
No worries, I'll send you the exact commit. 😄 |
|
The fix is on the master. |
|
Thanks, I’ll try to use it with the next dev build
On Wed, 24 May 2023 at 08:45, Jovan Stevanovic ***@***.***> wrote:
Fix is on the master.
—
Reply to this email directly, view it on GitHub
<#1956 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACFSBTYE55WS3JLBHBKU53XHWVBRANCNFSM4JYA2Z2Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@jovanstevanovic I finally got around testing this but it still doesn't work completely. An example: import java.net.URL;
import java.io.File;
import java.io.InputStream;
import java.io.IOException;
public class ClassLoader {
public static void main(String [] args) throws IOException {
String classPath = args[0];
String toFind = args[1];
String[] parts = classPath.split(java.io.File.pathSeparator);
var urls = new java.util.ArrayList<URL>();
for (String u : parts) {
urls.add(new File(u).toURL());
}
var cp = new java.net.URLClassLoader(urls.toArray(URL[]::new));
URL r = cp.getResource(toFind);
System.out.println(r);
}
// call with java ClassLoader "$(clojure -Spath)" "clojure/core.clj"
}Note that when called through |
|
@borkdudeI've focused on a reproducer that you sent me. It's possible that finding it in a JAR is not exercising the same part of the code. Two things:
|
I didn't quite understand this sentence. Do you want me to create a new Github issue, if so, why did you re-open this one? Or did you want me to create a new repro? Didn't I post this just above? |
I've reopened this issue already.
|
|
Here is the example that I've tried: new_reproducer.txt |
|
@jovanstevanovic Gotcha! |
|
@borkdude I've reproduced it locally. For some reason, NI can handle folders in URLClassLoader but not jars. I'll come back to you once I solve it. |
|
Yep, that's what I found too. |
|
@borkdude can you rerun the app with |
|
Confirmed, that was it! |
|
Thanks a lot |
Please use the API option variant: |
|
Thanks @olpaw |
Problem statement:
I want to find and retrieve a file in a given collection of .jar files or directories.
URLClassLoaderseems like the perfect fit and works on the JVM. But not on GraalVM.Repro:
Given the following Java code in
ClassLoader.java:and a file
/tmp/foo/bar.clj, when I compile and run this with the JVM:I see:
but when compiled and run with native image:
I get
null.Expected result:
I expected the same result as on the JVM.
The text was updated successfully, but these errors were encountered: