Call to classgraph inside JAX-RS endpoint does not find any results

[Restage]

Hi,

I’ve started a small project in which I use the neo4j-ogm library, which itself uses classgraph to find relevant entity classes.
Since this application was never able to find any of my entities, no matter in which package they reside, I did some research to find the root cause of the problem.

In the end I discovered that using classgraph inside a servlet works perfect.

@WebServlet(name = "TestServlet", urlPatterns = "/servlet")
public class TestServlet extends HttpServlet {

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        try (ScanResult scanResult =
                     new ClassGraph()
                             .verbose()               // Log to stderr
                             .enableAllInfo()         // Scan classes, methods, fields, annotations
                             .acceptPackages("de.test.classgraph")
                             .scan()) {               // Start the scan
            for(String name : scanResult.getAllClasses().getNames()) {
                resp.getOutputStream().println(name);
            }
        }

        resp.setStatus(HttpServletResponse.SC_OK);
    }
}

I got de.test.classgraph.ClassToFind as the expected result.

But as soon as I put the same code inside a JAX-RS endpoint classgraph fails to find my class.

@Path("/rest")
public class TestRest {
    @GET
    public Response ping() {
        String allClasses = "";

        try (ScanResult scanResult =
                     new ClassGraph()
                             .verbose()               // Log to stderr
                             .enableAllInfo()         // Scan classes, methods, fields, annotations
                             .acceptPackages("de.bitandgo.classgraph")
                             .scan()) {               // Start the scan
            for(String name : scanResult.getAllClasses().getNames()) {
                allClasses += name + "\n";
            }
        }

        return Response.ok().entity(allClasses).build();
    }
}

The application with both endpoints has been deployed on apache-tomee-webprofile-8.0.6.

I then took a look at the classgraph log output and discovered that different classloaders where used.
Calling the servlet brought

ClassGraph	---- Found ClassLoaders:
ClassGraph	------ org.apache.tomee.catalina.TomEEWebappClassLoader
ClassGraph	------ sun.misc.Launcher$AppClassLoader
ClassGraph	------ java.net.URLClassLoader

while accessing the JAX-RS endpoint gave me

ClassGraph	---- Found ClassLoaders:
ClassGraph	------ org.apache.openejb.server.cxf.transport.util.CxfContainerClassLoader
ClassGraph	------ sun.misc.Launcher$AppClassLoader
ClassGraph	------ java.net.URLClassLoader

So I was hoping that overriding the classloader might give me the expected result. But enhancing the configuration to

new ClassGraph()
    .verbose()               // Log to stderr
    .enableAllInfo()         // Scan classes, methods, fields, annotations
    .acceptPackages("de.test.classgraph")
    .overrideClassLoaders(new TomEEWebappClassLoader())
    .scan())

didn't find the class either.

This repository contains the full example.

So is this a bug or am I doing something wrong?

Best regards

[lukehutch]

Your first guess is probably correct -- this classloader is almost certainly not supported. You can see the supported classloaders here, as well as many examples of how to support a classloader:

https://github.com/classgraph/classgraph/tree/latest/src/main/java/nonapi/io/github/classgraph/classloaderhandler

I even wrote a fallback ClassLoaderHandler that tries to guess at the names of fields and methods that might hold the classpath!

https://github.com/classgraph/classgraph/blob/latest/src/main/java/nonapi/io/github/classgraph/classloaderhandler/FallbackClassLoaderHandler.java

Please try to introspect/debug your instance of CxfContainerClassLoader, or look at the source if available, to determine where the classpath entries are stored in this classloader. If you're feeling ambitious, please try writing a ClassLoaderHandler for this classloader, using the patterns shown in the classloaderhandler package. For most classloaders it's only a few lines of code. Thanks!

[lukehutch]

@Restage Please let me know when you get a chance to look at the internal fields and methods of the classloader. I cloned the git repo you provided, but I have no idea how to launch it the two different ways you mentioned (let alone how to launch it in the debugger)...

[lukehutch]

Hello @Restage! I try to fix and close bug reports quickly -- typical turnaround time is around 24 hours once I have all the information I need. I would appreciate your response so I can get this fixed! Thanks.

[Restage]

Hi @lukehutch,

depending on you IDE you need to build the sample application and deploy it to TomEE. In my case I used the webprofile 8.0.6 version. You then need to to call either the servlet or rest endpoint to see the difference.

I also tried to write a new ClassLoaderHandler. But I'm stuck right now because I'm not sure how to implement findClassLoaderOrder and findClasspathOrder methods. canHandle was very easy.

For findClassLoaderOrder I tried the "standard"

    classLoaderOrder.delegateTo(classLoader.getParent(), /* isParent = */ true, log);
    classLoaderOrder.add(classLoader, log);

For findClasspathOrder I started with a very simple approach which does not lead to any success

 final ClassLoader cl = (ClassLoader) ReflectionUtils.invokeMethod(classLoader, "tccl", false);

My plan was to analyze how other ClassLoaderHandler do their work but the time I can put into this topic is very limited at the moment. Maybe in the course of the weekend.

The target CxfContainerClassLoader class is part of this Maven dependency:

<dependency>
        <groupId>org.apache.tomee</groupId>
        <artifactId>openejb-cxf-transport</artifactId>
        <version>8.0.6</version>
</dependency>

[lukehutch]

You're on the right track. The order you call delegateTo and add inside findClassLoaderOrder should represent the order in which classes are loaded using another classloader, or this classloader respectively.

If this is the right definition of CxfContainerClassLoader, then you can see the delegation order in findClass, and the class actually doesn't do any classloading of its own, it only delegates to two other classloaders, in this order:

1. The classloader returned by CxfUtil.class.getClassLoader(), if the class name starts with "org.apache.cxf."
2. Otherwise, the classloader returned by tccl().

The findClassLoaderOrder mechanism isn't set up to only conditionally delegate to another classloader, so we'll have to just ignore the "if the class name starts with "org.apache.cxf."" part, and assume that CxfUtil.class.getClassLoader() will only load classes it's supposed to (i.e. classes with that package prefix).

There's nothing to do in findClasspathOrder since this classloader doesn't itself load any classes.

Sorry that that's not properly documented, I see how it would be confusing for this situation.

I checked a quick prototype of the ClassLoaderHandler into git, but I really don't have a clue how to run TomEE, and I don't have a lot of time to learn how to do that in order to test this. But it would be great if you can please test and/or tweak this ClassLoaderHandler until it works! (Or you could send me complete minimal instructions for how to set up TomEE on Linux and launch your example.)

[lukehutch]

Incidentally tccl() may return another unsupported classloader. If so, you would need another ClassLoaderHandler for that one. Hopefully it's already supported though, just as TomEEWebappClassLoader is.

[lukehutch]

Actually I couldn't understand why TomEEWebappClassLoader was working... I checked the ClassLoaderHandler instances in ClassGraph, and none of them support this classloader. Also TomEEWebappClassLoader doesn't have any fields or methods that the FallbackClassLoaderHandler can guess. So the classes you needed to scan or load must have been handled by a different classloader (probably the parent of the TomEEWebappClassLoader).

I took a shot at writing a ClassLoaderHandler for TomEEWebappClassLoader though too, based on this code. Please test this too to make sure the run that was formerly succeeding with TomEEWebappClassLoader is still succeeding. Plus if you scan with this new TomEEWebappClassLoaderHandler in place, ClassGraph may be able to find some more jars that it couldn't find on the classpath before.

Thanks!

[lukehutch]

@Restage do you want me to push out a release with these two (untested) ClassLoaderHandler instances? They should work, fingers crossed, but I don't know how to test them.

[lukehutch]

@Restage I released the new ClassLoaderHandler implementations in 4.8.106. They should work in theory -- please comment here and let me know. If they don't work, I'll reopen this and push out another release once they're fixed. Thanks for your feature request!

[Restage]

Hi @lukehutch,
so first of all thank you for your effort. I have tested against version .106 and here are my findings:

• As implemented this version is still able to successfully run the Servlet part. The expected class is found and the console output confirms that TomEEWebappClassLoader is used.

• But I wonder why both new loaders have not been registered in the ClassLoaderHandlerRegistry? So I did this and the TomEEWebappClassLoader stopped working. Although the code of the new TomEEWebappClassLoaderHandler is executed, it does not return a result anymore.

• The same applies to the CxfContainerClassLoaderHandler. It does not find the class with or without registration in the registry. But the new version at least shows it as

  ClassGraph	---- Found ClassLoaders:
  ClassGraph	------ org.apache.openejb.server.cxf.transport.util.CxfContainerClassLoader
  ClassGraph	------ sun.misc.Launcher$AppClassLoader
  ClassGraph	------ java.net.URLClassLoader
  

You might try these steps to run the example:

• Download Apache TomEE Webprofile version 8.0.6. from https://tomee.apache.org/download.html
• Without an IDE you need to build the example via Maven: mvn package. This creates a ClassGraph_Test-1.0-SNAPSHOT.war artifact.
• You need to put this artifact into the webapps folder of TomEE.
• To start the server bin/startup.sh needs to be executed. It might be necessary to set the execute flag for this script. Personally I do this for all files in the bin/ folder.
• You can than check the catalina.2021-xx-xx.log to get information about the server start. This output also shows the context path of the application. It should be http://localhost:8080/ClassGraph_Test-1.0-SNAPSHOT/api/rest
• Call this endpoint via commandline or your browser (it needs as simple GET request).
• No matter if the ClassToFind example class is found, the response should be http code 200. When the class was found the full package name is written to the body. Otherwise the body is empty.
• You can than check the log again, where the classgraph debug output gets shown.
• To start the application server call bin/shutdown.sh.

Depending on your IDE you might be able to integrate TomEE, start the example and debug the application.

Servlet Example (TomEEWebappClassLoader)

JAX-RS Example (CxfContainerClassLoaderHandler)

[lukehutch]

Thank you very much for the detailed instructions for running the code. ClassGraph supports dozens of runtime environments, and I can't possibly learn them all, so this was very helpful.

You are right, I forgot to add the two new ClassLoaderHandler instances to the registry! Good catch.

The reason that adding TomEEWebappClassLoaderHandler to the registry is that TomcatWebappClassLoaderBaseHandler was already handling TomEEWebappClassLoader better than my attempt at TomEE-specific support was handling it. I didn't see the logs before:

Classloader TomEEWebappClassLoader ... is handled by nonapi.io.github.classgraph.classloaderhandler.TomcatWebappClassLoaderBaseHandler

It turns out that TomEEWebappClassLoader extends TomcatWebappClassLoaderBaseHandler, which I didn't notice before.

The issue with CxfContainerClassLoaderHandler comes down to a very strange implementation detail of TomEE: the CxfContainerClassLoader instance and the TomEEWebappClassLoader instance in the current JAX-RS instance pass the .equals() test, even though they are separate classloaders! CxfContainerClassLoader delegates to TomEEWebappClassLoader, but not the other way around. But CxfContainerClassLoaderInstance.equals(TomEEWebappClassLoaderInstance) and also TomEEWebappClassLoaderInstance.equals(CxfContainerClassLoaderInstance). I have no idea why they did this...

The result of this is that a Set<ClassLoader> that is used to ensure that classloader instances are not added twice to the classloader order fails to add the TomEEWebappClassLoader instance to the order, because the CxfContainerClassLoader instance that delegates to it has already been added to the order. Therefore the TomEEWebappClassLoader instance that is needed to actually find the classpath entry that contains de.test.classgraph.ClassToFind never gets processed.

Fixing this will require modifying ClassGraph a bit to handle this weird case, so that I don't use equals() equality, but rather pointer equality for set keys. I'll get to work on it...

[lukehutch]

@Restage : OK, with this change to use IdentityHashMap rather than HashMap and HashSet, so that TomEEWebappClassLoader and CxfContainerClassLoader are disambiguated by reference, rather than by .equals() result, I can get both your servlet example and your JAX-RS example to work. I released this fixed version as classgraph-4.8.107.

Thank you for all your detailed assistance to get this fixed!