NPE thrown during scanning

[ronshemerws]

Scanning artifact org.plumelib:plume-util:1.6.5 with latest classgraph release (4.8.156) throws an NPE. This is happening with both java 11 & 17.
It looks like it is happening when processing the method

public @org.checkerframework.checker.formatter.qual.UnknownFormat boolean removeAll(java.util.Collection<?>)

which is defined in org.plumelib.util.IdentityArraySet
Unfortunately, I could not come up with a solution yet.

This is a short program that reproduces the issue, assuming the plume-util jar file is placed in the resources folder:

package org.example;

import io.github.classgraph.ClassGraph;
import io.github.classgraph.ScanResult;

import java.util.HashSet;
import java.util.Set;

public class Main {
    public static void main(String[] args) {
        Set<String> classpath = new HashSet<>();
        classpath.add(Main.class.getClassLoader().getResource("plume-util-1.6.5.jar").getPath());
        try (ScanResult scanResult =
                     new ClassGraph()
                             .verbose()
                             .enableInterClassDependencies()
                             .overrideClasspath(classpath)
                             .scan()) {
            System.out.println("success");
        }
    }
}

This is the output of this program:

Exception in thread "main" io.github.classgraph.ClassGraphException: Uncaught exception during scan
	at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1610)
	at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1627)
	at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1640)
	at org.example.Main.main(Main.java:18)
Caused by: java.lang.NullPointerException: Cannot invoke "io.github.classgraph.ReferenceTypeSignature.addTypeAnnotation(java.util.List, io.github.classgraph.AnnotationInfo)" because "this.typeSignature" is null
	at io.github.classgraph.TypeArgument.addTypeAnnotation(TypeArgument.java:107)
	at io.github.classgraph.ClassRefTypeSignature.addTypeAnnotation(ClassRefTypeSignature.java:234)
	at io.github.classgraph.Classfile$2.decorate(Classfile.java:1649)
	at io.github.classgraph.MethodInfo.getTypeSignature(MethodInfo.java:259)
	at io.github.classgraph.MethodInfo.findReferencedClassInfo(MethodInfo.java:799)
	at io.github.classgraph.MethodInfoList.findReferencedClassInfo(MethodInfoList.java:99)
	at io.github.classgraph.ClassInfo.findReferencedClassInfo(ClassInfo.java:3224)
	at io.github.classgraph.ScanResultObject.findReferencedClassInfo(ScanResultObject.java:72)
	at io.github.classgraph.ScanResult.indexResourcesAndClassInfo(ScanResult.java:327)
	at io.github.classgraph.ScanResult.<init>(ScanResult.java:269)
	at io.github.classgraph.Scanner.performScan(Scanner.java:1012)
	at io.github.classgraph.Scanner.openClasspathElementsThenScan(Scanner.java:1114)
	at io.github.classgraph.Scanner.call(Scanner.java:1148)
	at io.github.classgraph.Scanner.call(Scanner.java:82)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

Adding also the full project:
plume-util-scanner.zip
Once extracted, running the following should reproduce the issue:
mvn package
java -jar target/bug_report-1.0-SNAPSHOT-shaded.jar

And here is the full verbose log:
verbose_log.txt

[lukehutch]

@ronshemerws Thanks for the bug report and the complete testcase. I can reproduce the issue.

This seems to be a compiler bug in whatever compiled plume-util-1.6.5.jar.

The error occurs while scanning class org.plumelib.util.IdentityArraySet, for the method: boolean removeAll(java.util.Collection<?>).

The classfile specifies there's an annotation on the generic paramater ? of Collection, in the form ? extends @org.checkerframework.checker.formatter.qual.UnknownFormat X. However ? does not extend anything in this case, so X does not exist.

Was this compiled from Kotlin? The Kotlin compiler has many classfile generation bugs that in the past I have had to work around.

I'll keep looking into this in case I'm missing something.

[lukehutch]

I don't even know where this annotation comes from, because the source code doesn't have it:

https://github.com/plume-lib/plume-util/blob/master/src/main/java/org/plumelib/util/IdentityArraySet.java#L239

  public boolean removeAll(Collection<?> c) {

However, definitely the type bound ? does not use extends.

[lukehutch]

So this was compiled from Java, and the annotations were injected after the fact, using bytecode manipulation? If so, the bug may be in the bytecode manipulation stage.

[ronshemerws]

@lukehutch thank you for the quick response!

Actually I also failed to find a method with that signature in the sources so I also believe that this is some bytecode manipulation as you suggest.
plume-util is some 3rd party that got into my project so I am not familiar with it's compilation, maybe I can get some insights on this if I build it myself.

[lukehutch]

The fully annotated method (without the parameter problem) is

@org.checkerframework.checker.formatter.qual.UnknownFormat boolean (java.util.Collection<?>)

but with the type parameter annotation, the classfile says the type is supposed to be (for some non-existent/non-present X):

@org.checkerframework.checker.formatter.qual.UnknownFormat boolean (java.util.Collection<? extends @org.checkerframework.checker.formatter.qual.UnknownFormat X>)

[lukehutch]

I double-checked everything, and as far as I can see, the classfile is definitely wrong.

[lukehutch]

OK, so since this isn't even code you maintain, and it's probably created by a buggy tool (so there are probably other libraries out there with the same problem), I chose to just ignore this case where an NPE is obviously going to be triggered. The type annotation will simply be dropped in this case, because its target doesn't even exist.

Fixed and released as 4.8.157.

Thanks again for the bug report!