Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unexpected exception occurred initializing CPE Analyzer (NPE) #2909

Closed
danielbraeutigam opened this issue Oct 20, 2020 · 8 comments
Closed

Unexpected exception occurred initializing CPE Analyzer (NPE) #2909

danielbraeutigam opened this issue Oct 20, 2020 · 8 comments
Labels
bug dependencies

Comments

@danielbraeutigam
Copy link

Describe the bug
Analyze fails with the error "Unexpected exception occurred initializing CPE Analyzer". The reason for this is the following NPE:

java.lang.NullPointerException
	at org.owasp.dependencycheck.data.cpe.AbstractMemoryIndex.parseQuery(AbstractMemoryIndex.java:277)
	at org.owasp.dependencycheck.analyzer.CPEAnalyzer.searchCPE(CPEAnalyzer.java:388)
	at org.owasp.dependencycheck.analyzer.CPEAnalyzer.determineCPE(CPEAnalyzer.java:263)
	at org.owasp.dependencycheck.analyzer.CPEAnalyzer.analyzeDependency(CPEAnalyzer.java:689)
	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)

This happens multiple times for several of different libraries, e.g.:
An unexpected error occurred during analysis of '/home/myUser/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm-commons/7.2/ca2954e8d92a05bacc28ff465b25c70e0f512497/asm-commons-7.2.jar' (CPE Analyzer): null

Version
The problem occurs using version 6.0.2 of the gradle plugin.

To Reproduce
Steps to reproduce the behavior:

  1. Execute Gradle Task dependencyCheckAnalyze

Expected behavior
No error is thrown and CPE's can be analyzed.

Additional context
Gradle cache has been deleted, but it doesn't solve the problem. Purge and update by the plugin was performed as well without success.
@jeremylong
Copy link
Owner

I can't reproduce this with the information given.

build.gradle

plugins {
  id 'java'
  id 'org.owasp.dependencycheck' version '6.0.2'
}

dependencies {
  compile group: 'org.ow2.asm', name: 'asm-commons', version: '7.2'
}
$ gradle dependencyCheckAnalyze

> Task :dependencyCheckAnalyze
Verifying dependencies for project gradle-test
Checking for updates and analyzing dependencies for vulnerabilities
Generating report for project gradle-test
Found 0 vulnerabilities in project gradle-test

BUILD SUCCESSFUL in 3m 24s
1 actionable task: 1 executed

What OS are you using, can you provide a sample project that exhibits the behavior, can you provide a full log file, etc.?

@danielbraeutigam
Copy link
Author

danielbraeutigam commented Oct 20, 2020 •
edited

It's a manjaro linux system.
Okay, after a long search I maybe found the reason what causes the problem.
Can you please add the nebula library in your test?

plugins {
    id 'java'
    id 'nebula.project' version '7.0.9'
    id 'org.owasp.dependencycheck' version '6.0.2'
}

I get the error when this library is added. If I remove it, everything works as expected like in your example.

@jeremylong
Copy link
Owner

Ah the fun of dependency conflicts... When I add nebula.project I got:

08:25:51.619 [DEBUG] [org.owasp.dependencycheck.Engine] 
java.lang.NoClassDefFoundError: org/apache/lucene/codecs/lucene41/Lucene41PostingsFormat
        at java.lang.Class.getDeclaredConstructors0(Native Method)
        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
        at java.lang.Class.getConstructor0(Class.java:3075)
        at java.lang.Class.newInstance(Class.java:412)
        at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:72)
        at org.apache.lucene.util.NamedSPILoader.<init>(NamedSPILoader.java:51)
        at org.apache.lucene.util.NamedSPILoader.<init>(NamedSPILoader.java:38)
        at org.apache.lucene.codecs.PostingsFormat$Holder.<clinit>(PostingsFormat.java:49)
        at org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
        at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:312)
        at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:395)
        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:114)
        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:83)
        at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:66)
        at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:720)
        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
        at org.owasp.dependencycheck.data.cpe.AbstractMemoryIndex.open(AbstractMemoryIndex.java:130)

So I took a look at the build environment (gradle buildEnvironment):

          |    |    +--- org.elasticsearch:elasticsearch:1.7.2
          |    |    |    +--- org.apache.lucene:lucene-core:4.10.4 -> 8.6.2
          |    |    |    +--- org.apache.lucene:lucene-analyzers-common:4.10.4 -> 8.6.2 (*)
          |    |    |    +--- org.apache.lucene:lucene-queries:4.10.4 -> 8.6.2 (*)
          |    |    |    +--- org.apache.lucene:lucene-memory:4.10.4
          |    |    |    |    \--- org.apache.lucene:lucene-core:4.10.4 -> 8.6.2

Unfortunately - until elastic search and nebula upgrade to using a modern version of Lucene (4.10.4 is ~5 years old) I don't think these plugins are compatible.

@danielbraeutigam danielbraeutigam mentioned this issue Oct 21, 2020
Closed
@aikebah aikebah mentioned this issue Oct 31, 2020
Closed
@hpoettker
Copy link
Contributor

Their has been a new release of the nebula project plugin with updated dependencies. I've tested version 8.0.0 of the nebula plugin with version 6.1.0 of dependency check, and they work fine together.

I think the issue can be closed.

@cforce
Copy link

cforce commented Mar 23, 2021

I have same/similar issue with using maven when i upgrade from 6.0.3 to 6.1.3
mvn org.owasp:dependency-check-maven:6.0.3:aggregate
-f ${BUILD_SOURCESDIRECTORY}/pom.xml
-s ${BUILD_SOURCESDIRECTORY}/mvn/settings.xml
-B
-Dlicense.skipAddThirdParty
-Dformat=ALL
-DskipProvidedScope=true
-DskipSystemScope=true
-DoutputDirectory=${BUILD_SOURCESDIRECTORY}/${APP_NAME}-app/target/release-server-upload/${APP_VERSION}/owasp/dependency-check-report.html"

[INFO] Created CPE Index (1 seconds)
[WARNING] An unexpected error occurred during analysis of '/home/vsts/.m2/repository/org/khronos/opengl-api/gl1.1-android-2.1_r1/opengl-api-gl1.1-android-2.1_r1.jar' (CPE Analyzer): null
[ERROR] 
java.lang.NullPointerException
    at java.lang.String.endsWith (String.java:1449)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.lambda$null$3 (CPEAnalyzer.java:380)
    at java.util.stream.ReferencePipeline$2$1.accept (ReferencePipeline.java:174)
    at java.util.HashMap$KeySpliterator.forEachRemaining (HashMap.java:1556)
    at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:482)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto (AbstractPipeline.java:472)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential (ForEachOps.java:150)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential (ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate (AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach (ReferencePipeline.java:485)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.lambda$addMajorVersionToTerms$5 (CPEAnalyzer.java:383)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept (ForEachOps.java:183)
    at java.util.stream.ReferencePipeline$2$1.accept (ReferencePipeline.java:175)
    at java.util.HashMap$EntrySpliterator.forEachRemaining (HashMap.java:1699)
    at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:482)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto (AbstractPipeline.java:472)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential (ForEachOps.java:150)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential (ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate (AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach (ReferencePipeline.java:485)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.addMajorVersionToTerms (CPEAnalyzer.java:378)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.determineCPE (CPEAnalyzer.java:279)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.analyzeDependency (CPEAnalyzer.java:764)
    at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze (AbstractAnalyzer.java:131)
    at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:88)
    at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:37)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)
[INFO] Finished CPE Analyzer (6 seconds)

@rgrosjean
Copy link

rgrosjean commented Mar 25, 2021 •
edited

the issue happened for me with the version 6.1.3 but not with the 6.1.2

[ERROR] 
java.lang.NullPointerException: null
	at java.base/java.lang.String.endsWith(String.java:1510)
	at org.owasp.dependencycheck.analyzer.CPEAnalyzer.lambda$null$3(CPEAnalyzer.java:380)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:176)
	at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1694)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at org.owasp.dependencycheck.analyzer.CPEAnalyzer.lambda$addMajorVersionToTerms$5(CPEAnalyzer.java:383)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
	at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1837)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at org.owasp.dependencycheck.analyzer.CPEAnalyzer.addMajorVersionToTerms(CPEAnalyzer.java:378)
	at org.owasp.dependencycheck.analyzer.CPEAnalyzer.determineCPE(CPEAnalyzer.java:279)
	at org.owasp.dependencycheck.analyzer.CPEAnalyzer.analyzeDependency(CPEAnalyzer.java:764)
	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)

@benjamin-sailer
Copy link

benjamin-sailer commented Mar 26, 2021 •
edited

Me too with a similar stack; I guess this is a different issue (no old nebula dependency in the classpath, the elasticsearch dependency is 7.1.1 with lucene 8).
In case that could make any difference: The issue arises when an internal snapshot dependency is traversed:

[WARNING] An unexpected error occurred during analysis of '/var/lib/jenkins/.m2/repository/com/axonactive/insight/ai-spring-boot/6.12.0-ES7-SNAPSHOT/ai-spring-boot-6.12.0-ES7-SNAPSHOT.jar' (CPE Analyzer): null
[ERROR] 
java.lang.NullPointerException
    at java.lang.String.endsWith (String.java:1485)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.lambda$null$3 (CPEAnalyzer.java:380)
    at java.util.stream.ReferencePipeline$2$1.accept (ReferencePipeline.java:176)
    at java.util.HashMap$KeySpliterator.forEachRemaining (HashMap.java:1603)
    at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:484)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto (AbstractPipeline.java:474)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential (ForEachOps.java:150)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential (ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate (AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach (ReferencePipeline.java:497)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.lambda$addMajorVersionToTerms$5 (CPEAnalyzer.java:383)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept (ForEachOps.java:183)
    at java.util.stream.ReferencePipeline$2$1.accept (ReferencePipeline.java:177)
    at java.util.HashMap$EntrySpliterator.forEachRemaining (HashMap.java:1746)
    at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:484)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto (AbstractPipeline.java:474)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential (ForEachOps.java:150)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential (ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate (AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach (ReferencePipeline.java:497)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.addMajorVersionToTerms (CPEAnalyzer.java:378)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.determineCPE (CPEAnalyzer.java:279)
    at org.owasp.dependencycheck.analyzer.CPEAnalyzer.analyzeDependency (CPEAnalyzer.java:764)
    at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze (AbstractAnalyzer.java:131)
    at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:88)
    at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:37)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:834)

Working like charm with the maven plugin version 6.1.2 but fails like above with 6.1.3.

Any more Information I could support you with?

@jeremylong
Copy link
Owner

The NPE already has been reported and fixed in code. It will be included in the 6.1.4 release.

Repository owner locked and limited conversation to collaborators Mar 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug dependencies
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@cforce @jeremylong @hpoettker @rgrosjean @benjamin-sailer @danielbraeutigam