Exception with multi-maven builds (exclude current folder from scanned classpath)

[ptahchiev]

Hello,

I have the following project setup:

master
    |----> module1
        |----> class ClassA (has annotation @Annotation)
    |----> module2
        |----> class ClassB (also has annotation @Annotation)

and I'm building it with maven, and I use an AbstractProcessor during compilation to find all types annotated with @Annotation. When I build it from the module1 level it all works fine. The problem appears when I build it from master level. Then, what happens is I get an exception that class B cannot be found.

Caused by: org.reflections.ReflectionsException: could not get type for name ClassB
    at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:389)
    at org.reflections.ReflectionUtils.forNames(ReflectionUtils.java:398)
    at org.reflections.Reflections.getTypesAnnotatedWith(Reflections.java:385)
    at org.reflections.Reflections.getTypesAnnotatedWith(Reflections.java:370)

Here's how I find the types annotated with @Annotation:

        final Reflections reflections =
                        new Reflections(new ConfigurationBuilder().setUrls(ClasspathHelper.forClassLoader(ClasspathHelper.staticClassLoader())).setScanners(
                                        new TypeAnnotationsScanner(), new SubTypesScanner()));
        return reflections.getTypesAnnotatedWith(Annotation.class);

What happens is that the classloader will find also the following url:

/home/petar/workspace/master/./

and then it will be given to Reflections (method scan on line 235 in Reflections class ) where it will be treated as a SystemDir so reflections will recursively go thorough all the subfolders and add all the classes it finds (which also includes module2/target/classes/ClassB.class). Then it will try to load it and it will produce the error I pasted above. Now i'm not sure who's passing the master folder to the classpath, but I checked the maven classpath and I'm 100% sure it is not coming from maven. I also tried to add a filterInputsBy with a FilterBuilder.Exclude but that only allows me to filter on incoming files, and not the master folder (plus I have a project with more than 50 modules so I can't really exclude all the classes).

Also in my case I have a project with more than 50 modules, so reflections will be really slow if it scans the master folder. I think the proper solution is not the scan the master folder at all.

[martin-g]

I think you should use the context class loader instead, i.e. org.reflections.util.ClasspathHelper#contextClassLoader().

Reflections uses the urls that you provide with the configuration builder. It doesn't care whether you use Maven/Gradle/SBT/..., with 1 or 100 modules. It is your job to provide the most appropriate ClassLoader(s) that sees only the classes you want to be seen.

[ptahchiev]

Hi Martin,

thanks for your reply. I tried using the context class loader, but unfortunately then it returns only 13 urls:

file:/home/petar/.m2/repository/org/apache/maven/plugins/maven-compiler-plugin/3.1/maven-compiler-plugin-3.1.jar
file:/home/petar/.m2/repository/org/codehaus/plexus/plexus-utils/1.5.1/plexus-utils-1.5.1.jar
file:/home/petar/.m2/repository/org/apache/maven/shared/maven-shared-utils/0.1/maven-shared-utils-0.1.jar
file:/home/petar/.m2/repository/com/google/code/findbugs/jsr305/2.0.1/jsr305-2.0.1.jar
file:/home/petar/.m2/repository/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar
file:/home/petar/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar
file:/home/petar/.m2/repository/org/codehaus/plexus/plexus-compiler-api/2.2/plexus-compiler-api-2.2.jar
file:/home/petar/.m2/repository/org/codehaus/plexus/plexus-compiler-manager/2.2/plexus-compiler-manager-2.2.jar
file:/home/petar/.m2/repository/org/codehaus/plexus/plexus-compiler-javac/2.2/plexus-compiler-javac-2.2.jar
file:/home/petar/.m2/repository/org/apache/xbean/xbean-reflect/3.4/xbean-reflect-3.4.jar
file:/home/petar/.m2/repository/log4j/log4j/1.2.12/log4j-1.2.12.jar
file:/home/petar/.m2/repository/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar
file:/home/petar/.m2/repository/com/google/collections/google-collections/1.0/google-collections-1.0.jar
file:/home/petar/.m2/repository/junit/junit/3.8.2/junit-3.8.2.jar

I believe those are the maven core libraries. Meanwhile If I use the static classloader it returns 167 jars (all my dependencies in the pom.xml + the folder that I mentioned /home/petar/workspace/master/./)

[martin-g]

Then try with ClassA.class.getClassLoader().
I'd bet that Maven appends the current dir in the classpath.
Yet another way is to build your own ClassLoader. Just filter out all jars from the static class loader that are needed and put them in UrlClassLoader. This way you will make the scanning faster too.

[ptahchiev]

Hi Martin,

thanks for the suggestion. I thought of doing so but unfortunately I can't :( .. I'm making a platform which I will be giving to customer, and I don't know what will be classA's name in my customer's project. I want to be able to find class A, classB and also if there's a classC defined by the customer. I really need to scan ALL the classpath to find ALL the classes that have the given annotation.

Plus it definitely looks like a bug in reflections to recursively scan the subdirectories. As I said my project has more than 50 modules, each one having at least 5 subdirectories. This is super inefficient to scan all of them, and later not being able to use what you scanned (classB is not in the classpath).

[martin-g]

Let's see what response you'll get at Maven mailing lists (I've noticed your question there).

Which version of Reflections do you use ?

[ptahchiev]

0.9.9 but I don't think it's a problem of maven... From what I can see maven will invoke javac with correct -cp argument (lists all the jars in the pom.xml) and the directory is not there :( I have no clue why java's Classloader is including this folder.

[ptahchiev]

This thread:

https://community.oracle.com/thread/2456122

explains that it's always been like that (the current working directory is getting added to the classpath). So i guess now it's more an issue of reflections to exclude it and not scan it recursively.

[ronmamo]

You should avoid scanning all urls, but only those that contains classes whose fqn starts with your clients package prefix(es), and these must be known a priori.
Check here for constructor variants.
Another option, if that is your case, is to extract the urls from the MavenProject object - see here
Second, use filterInputsBy to filter classes from those urls, if needed.
Does it help?