Dependency-check failing for aggregate project due to in-reactor war-dependency with a vulnerable library in a previous snapshot hosted in the repository

[aikebah]

Describe the bug
Looks like #1600 has resurfaced. Will look into it further myself and propose a patch if needed

Version of dependency-check used
The problem occurs using version 5.3.2 of the maven plugin

Log file
Properietary project, so cannot share the log. Will look into getting a minimal project to sample it and then attach logs.

To Reproduce
Steps to reproduce the behavior:

1. Create a SNAPSHOT multi-module project with a war-file containing a vulnerable library
2. Build and deploy that SNAPSHOT version
3. Update the war submodule to depend on the non-vulnerable version of the library
4. Perform an aggregate depdendency-check on the project and observe Dependency-Check complain about the vulnerable library in the PREVIOUS version of the war as retrieved from the artifact repository

Expected behavior
Dependency-check only depending on the reactor-internal virtual dependencies (as this is an aggregate scan for a SNAPSHOT project) and ignoring the (still vulnerable) war-file currently stored in the artifact repository for the same snapshot-version.

Additional context
Add any other context about the problem here.

[aikebah]

The way I managed to reproduce the issue we encountered indicates that it may be hard to create an IT to prevent it in the future. The scenario that I run to reproduce the issue that we encountered in our project:

• For ease of immediately spotting the issue by vulnerabilities found make IT org.owasp.test.aggregate.issue-1527:lib (old) depend on jackson-databind 2.9.10.3 to get a vulnerability that is within the current timespan of IT vulnerabilities (2018 and beyond)

• mvn clean install -f old/pom.xml -Dcve.startyear=2020 -Dodc.version=5.3.2 -Danalyzer.ossindex.enabled=false to get the artifacts in the repository and observe the output signalling vulnerabilites found for jackson-databind 2.9.10.3

• mvn clean verify -f new/pom.xml -Dcve.startyear=2020 -Dodc.version=5.3.2 -Danalyzer.ossindex.enabled=false and observe the 'using virtual dependencies' of aggregate goal ignoring reactor modules blocks a re-scan of an updated aggregate project when vulnerable dependencies have been upgraded in case the offending module is a composite archive such as war-modules #1600 kicking in to resolve the dependencies from the local structure no longer finding a vulnerability (as the dependency got removed in the 'new' snapshot version of lib)

• mvn clean deploy -f old/pom.xml -Dcve.startyear=2020 -Dodc.version=5.3.2 -Danalyzer.ossindex.enabled=false to get the artifacts in a repository

• mvn clean verify -f new/pom.xml -Dcve.startyear=2020 -Dodc.version=5.3.2 -Danalyzer.ossindex.enabled=false and observe the 'using virtual dependencies' of aggregate goal ignoring reactor modules blocks a re-scan of an updated aggregate project when vulnerable dependencies have been upgraded in case the offending module is a composite archive such as war-modules #1600 still kicking in to resolve the dependencies from the local structure no longer finding a vulnerability

• rm -rf ~/.m2/repository/org/owasp/test/aggregate/issue-1527 to clean out maven localRepo so that the old -SNAPSHOTs are only available in the repository manager

• mvn clean verify -f new/pom.xml -Dcve.startyear=2020 -Dodc.version=5.3.2 -Danalyzer.ossindex.enabled=false observe that the vulnerabilities of IT 1527/old are reported now that the old snapshots are resolved from the repository

• do another mvn clean verify -f new/pom.xml -Dcve.startyear=2020 -Dodc.version=5.3.2 -Danalyzer.ossindex.enabled=false now that the old snapshot version is back in the local repository and observe that the vulnerabilities of IT 1527/old are still reported

• mvn clean install -f old/pom.xml -Dcve.startyear=2020 -Dodc.version=5.3.2 -Danalyzer.ossindex.enabled=false to get a new build into localrepo

• mvn clean verify -f new/pom.xml -Dcve.startyear=2020 -Dodc.version=5.3.2 -Danalyzer.ossindex.enabled=false and observe the 'using virtual dependencies' of aggregate goal ignoring reactor modules blocks a re-scan of an updated aggregate project when vulnerable dependencies have been upgraded in case the offending module is a composite archive such as war-modules #1600 kicking in again to resolve the dependencies from the local structure no longer finding the issue

So only in the case 'SNAPSHOT in localrepo that originates from a maven repository' it appears that the SNAPSHOTS for reactor use virtual dependency are no longer kicking in.

@jeremylong is there some way to use an isolated maven snapshot repository and introduce the relevant parts of the reproduction scenario into an integration test:

• mvn deploy project IT issue 1527-old
• remove project IT issue1527-old artifacts from the localrepo so that they are forced to be 'resolved from snapshots repository'
• mvn verify project IT issue 1527-new
  (and then preferably even clean-up the repository (remove the IT testcase artifacts), so that no repo-storage is wasted with the IT test snapshot artifact after the testing)

[aikebah]

Note: a quick verification shows that behavior above on 5.3.2 (the version for which we observed the issue) is still present in 6.0.2. I'll investigate for it's root cause and work on fixing it while awaiting a response regarding the regression-testability of its root-cause to prevent re-introduction of the issue in the future.

[aikebah]

Root cause appears to be the second PR (#1773) for #1751 which doesn't take into account the possibility that dependencies that are reactor projects have already been resolved and should (as a snapshot-reactor-dependency be transformed into a virtual reactor dependency for aggregate builds.

[DEBUG] org.owasp.test.aggregate.issue-1527:ear:ear:1.0.0-SNAPSHOT
[DEBUG] org.owasp.test.aggregate.issue-1527:war:war:1.0.0-SNAPSHOT:compile
[DEBUG] Could not find metadata org.owasp.test.aggregate.issue-1527:war:1.0.0-SNAPSHOT/maven-metadata.xml in local (/Users/aikebah/.m2/repository)
[DEBUG] Skipped remote request for org.owasp.test.aggregate.issue-1527:war:1.0.0-SNAPSHOT/maven-metadata.xml, locally cached metadata up-to-date.
[DEBUG] Skipping artifact war, already resolved
....
[DEBUG] org.owasp.test.aggregate.issue-1527:war:war:1.0.0-SNAPSHOT
[DEBUG] org.apache.james:apache-mime4j-dom:jar:0.7.2:compile
[DEBUG] org.apache.james:apache-mime4j-core:jar:0.7.2:compile
[DEBUG] org.owasp.test.aggregate.issue-1527:lib:jar:1.0.0-SNAPSHOT:compile
[DEBUG] Could not find metadata org.owasp.test.aggregate.issue-1527:lib:1.0.0-SNAPSHOT/maven-metadata.xml in local (/Users/aikebah/.m2/repository)
[DEBUG] Skipped remote request for org.owasp.test.aggregate.issue-1527:lib:1.0.0-SNAPSHOT/maven-metadata.xml, locally cached metadata up-to-date.
[DEBUG] Skipping artifact apache-mime4j-dom, already resolved
[DEBUG] Adding new maven identifier org.apache.james:apache-mime4j-dom:0.7.2
[DEBUG] Adding project reference war on dependency apache-mime4j-dom-0.7.2.jar
[DEBUG] Skipping artifact apache-mime4j-core, already resolved
[DEBUG] Adding new maven identifier org.apache.james:apache-mime4j-core:0.7.2
[DEBUG] Adding project reference war on dependency apache-mime4j-core-0.7.2.jar
[DEBUG] Skipping artifact lib, already resolved
[DEBUG] Checking the reactor projects (4) for org.owasp.test.aggregate.issue-1527:lib:1.0.0-20201001.125638-2
[DEBUG] Comparing org.owasp.test.aggregate.issue-1527:lib:1.0.0-20201001.125638-2 to org.owasp.test.aggregate.issue-1527:parent:1.0.0-SNAPSHOT
[DEBUG] Comparing org.owasp.test.aggregate.issue-1527:lib:1.0.0-20201001.125638-2 to org.owasp.test.aggregate.issue-1527:lib:1.0.0-SNAPSHOT
[DEBUG] Comparing org.owasp.test.aggregate.issue-1527:lib:1.0.0-20201001.125638-2 to org.owasp.test.aggregate.issue-1527:war:1.0.0-SNAPSHOT
[DEBUG] Comparing org.owasp.test.aggregate.issue-1527:lib:1.0.0-20201001.125638-2 to org.owasp.test.aggregate.issue-1527:ear:1.0.0-SNAPSHOT
[DEBUG] Adding new maven identifier org.owasp.test.aggregate.issue-1527:lib:1.0.0-20201001.125638-2
[DEBUG] Adding project reference war on dependency lib-1.0.0-SNAPSHOT.jar

[aikebah]

Turns out to be not that change, as both a 'working' (fresh build) and a 'not working' (localrepo contains repository-resolved build) scenario contain the Skipping artifact war, already resolved in debug-log

[aikebah]

The issue appears to be caused by the checking for reactor projects as seen from the lines following the 'already resolved':
Checking the reactor projects (4) for org.owasp.test.aggregate.issue-1527:lib:1.0.0-20201001.125638-2
a Timestamped snapshot dependency versus
Checking the reactor projects (4) for org.owasp.test.aggregate.issue-1527:lib:1.0.0-SNAPSHOT
for a fresh build

[aikebah]

Most likely this scenario never worked and just took a while to surface.

As it took our developers a while to get around to maintenance on the project after the build failed the local snapshot got evicted from the local repository of our build-server.

Dependency-check is configured to be build-breaking, so new periodically scheduled builds did not (re)create the snapshot locally (nor in the repository manager) and to avoid depending on stale snapshots the local repository get periodically purged from snapshots by file-date so that builds fail when a project still depends on an already released snapshot)

[jeremylong]

@aikebah thanks for the PR!