sarif contains duplicate artifacts

[mprins]

Describe the bug
The sarif file produced on an aggregate maven project holds duplicate entries in the artifacts which - at least github says so - is invalid (the projects in the multimodule have shared dependencies).
This prevents uploading into the github "security" tab.

I'm not sure what is used for identifying artifacts in the sarif file, I would gues either uri or id1 - so not sure what qualifies a "duplicate".
I'm still looking at the log for the actual duplicate...

Version of dependency-check used
The problem occurs using version 6.1.4 of the maven plugin

Log file
I have a Github Action workflow that shows this at: https://github.com/B3Partners/brmo/pull/1039/checks?check_run_id=2229073601#step:6:14 relevant part is shown below, full log at: https://gist.github.com/mprins/b9d39bbd9156d9da3954da9de557c213

...

2021-03-30T15:29:20.6011828Z [WARNING] Cannot include project artifact: nl.b3p:brmo-dist:pom:2.0.4-SNAPSHOT; it doesn't have an associated file or directory.
2021-03-30T15:29:20.6023453Z [WARNING] The following patterns were never triggered in this artifact inclusion filter:
2021-03-30T15:29:20.6025445Z o  'jakarta.mail:jakarta.mail-api'
2021-03-30T15:29:20.6025917Z 
2021-03-30T15:31:36.9077132Z [WARNING] Exception extracting archive 'iso19139-20060504.zip'.
2021-03-30T15:31:36.9185943Z [WARNING] Exception extracting archive 'iso19139-20070417.zip'.
2021-03-30T15:31:36.9708708Z [WARNING] Exception extracting archive 'xlink-1_0_0.zip'.
2021-03-30T15:31:45.5791421Z 00:00  INFO: Vulnerability found: jquery below 1.9.0b1
2021-03-30T15:31:45.5799322Z 00:00  INFO: Vulnerability found: jquery below 1.12.0
2021-03-30T15:31:45.5823948Z 00:00  INFO: Vulnerability found: jquery below 1.12.0
2021-03-30T15:31:45.5910538Z 00:00  INFO: Vulnerability found: jquery below 3.4.0
2021-03-30T15:31:45.5911741Z 00:00  INFO: Vulnerability found: jquery below 3.5.0
2021-03-30T15:31:45.5912637Z 00:00  INFO: Vulnerability found: jquery below 3.5.0
2021-03-30T15:31:50.5526896Z ##[group]Run github/codeql-action/upload-sarif@v1
2021-03-30T15:31:50.5527466Z with:
2021-03-30T15:31:50.5528156Z   sarif_file: target/dependency-check-report.sarif
2021-03-30T15:31:50.5528962Z   checkout_path: /home/runner/work/brmo/brmo
2021-03-30T15:31:50.5529871Z   token: ***
2021-03-30T15:31:50.5530271Z   matrix: {
  "java": 8
}
2021-03-30T15:31:50.5530643Z env:
2021-03-30T15:31:50.5531169Z   JAVA_HOME_8.0.282_x64: /opt/hostedtoolcache/jdk/8.0.282/x64
2021-03-30T15:31:50.5531838Z   JAVA_HOME: /opt/hostedtoolcache/jdk/8.0.282/x64
2021-03-30T15:31:50.5533213Z   JAVA_HOME_8_0_282_X64: /opt/hostedtoolcache/jdk/8.0.282/x64
2021-03-30T15:31:50.5533763Z ##[endgroup]
2021-03-30T15:31:51.3992561Z Uploading sarif files: ["target/dependency-check-report.sarif"]
2021-03-30T15:31:51.5262171Z ##[group]Error details: instance.runs[0].artifacts contains duplicate item
2021-03-30T15:31:51.5271875Z {
2021-03-30T15:31:51.5272531Z   "property": "instance.runs[0].artifacts",
2021-03-30T15:31:51.5273281Z   "message": "contains duplicate item",
2021-03-30T15:31:51.5273787Z   "schema": {
2021-03-30T15:31:51.5274386Z     "description": "An array of artifact objects relevant to the run.",
2021-03-30T15:31:51.5275016Z     "type": "array",
2021-03-30T15:31:51.5275421Z     "minItems": 0,
2021-03-30T15:31:51.5275885Z     "uniqueItems": true,
2021-03-30T15:31:51.5276309Z     "items": {
2021-03-30T15:31:51.5276793Z       "$ref": "#/definitions/artifact"
2021-03-30T15:31:51.5277232Z     }
2021-03-30T15:31:51.5277564Z   },
2021-03-30T15:31:51.5277925Z   "instance": [
2021-03-30T15:31:51.5278297Z     {
2021-03-30T15:31:51.5278679Z       "description": {
2021-03-30T15:31:51.5280259Z         "text": "Open Source implementation of the Fast Infoset Standard for Binary XML (http://www.itu.int/ITU-T/asn1/)."
2021-03-30T15:31:51.5281079Z       },
2021-03-30T15:31:51.5281435Z       "location": {
2021-03-30T15:31:51.5282545Z         "uri": "file:////home/runner/.m2/repository/com/sun/xml/fastinfoset/FastInfoset/1.2.15/FastInfoset-1.2.15.jar"
2021-03-30T15:31:51.5283332Z       },
2021-03-30T15:31:51.5283696Z       "hashes": {
2021-03-30T15:31:51.5284218Z         "md5": "57f3894ad7e069ae740b277d92d10fa0",
2021-03-30T15:31:51.5285006Z         "sha1": "bb7b7ec0379982b97c62cd17465cb6d9155f68e8",
2021-03-30T15:31:51.5286217Z         "sha256": "785861db11ca1bd0d1956682b974ad73eb19cd3e01a4b3fa82d62eca97210aec"
2021-03-30T15:31:51.5287171Z       },
2021-03-30T15:31:51.5287556Z       "properties": {
2021-03-30T15:31:51.5288356Z         "license": "http://www.opensource.org/licenses/apache2.0.php",
2021-03-30T15:31:51.5289698Z         "id1": "pkg:maven/com.sun.xml.fastinfoset/FastInfoset@1.2.15"
2021-03-30T15:31:51.5290315Z       }
2021-03-30T15:31:51.5290644Z     },

...

2021-03-30T15:31:51.7793636Z ##[endgroup]
2021-03-30T15:31:51.7799322Z ##[error]Unable to upload "target/dependency-check-report.sarif" as it is not valid SARIF:
- instance.runs[0].artifacts contains duplicate item
2021-03-30T15:31:51.7813312Z Error: Unable to upload "target/dependency-check-report.sarif" as it is not valid SARIF:
2021-03-30T15:31:51.7814556Z - instance.runs[0].artifacts contains duplicate item
2021-03-30T15:31:51.7815871Z     at validateSarifFileSchema (/home/runner/work/_actions/github/codeql-action/v1/lib/upload-lib.js:155:15)
2021-03-30T15:31:51.7817308Z     at uploadFiles (/home/runner/work/_actions/github/codeql-action/v1/lib/upload-lib.js:214:9)
2021-03-30T15:31:51.7818763Z     at Object.uploadFromActions (/home/runner/work/_actions/github/codeql-action/v1/lib/upload-lib.js:91:18)
2021-03-30T15:31:51.7820262Z     at async run (/home/runner/work/_actions/github/codeql-action/v1/lib/upload-sarif-action.js:34:29)
2021-03-30T15:31:51.7821724Z     at async runWrapper (/home/runner/work/_actions/github/codeql-action/v1/lib/upload-sarif-action.js:46:9)

To Reproduce
Steps to reproduce the behavior: run the workflow in https://github.com/B3Partners/brmo/blob/2198870b00ea3a88b5a2997ee1376bcd4eb1e243/.github/workflows/owasp-dependency-check.yml

Expected behavior
Duplicate entries should be filtered out so upload into github "security" tab succeeds

[mprins]

I can see various item of

        {
          "description": {
            "text": "BGT GML light module voor de BRMO loader"
          },
          "location": {
            "uri": "file:////home/mark/dev/projects/brmo/bgt-gml-loader/pom.xml"
          },
          "properties": {
            "license": "GNU GENERAL PUBLIC LICENSE 3.0 https://raw.githubusercontent.com/B3Partners/brmo/master/LICENSE",
            "id1": "pkg:maven/nl.b3p/bgt-gml-loader@2.0.4-SNAPSHOT"
          }
        },

as well as

        {
          "description": {
            "text": "Bevat functies voor het opzoeken van databronnen en parameters voor de BRMO."
          },
          "location": {
            "uri": "file:////home/mark/dev/projects/brmo/brmo-init-util/pom.xml"
          },
          "properties": {
            "license": "GNU GENERAL PUBLIC LICENSE 3.0 https://raw.githubusercontent.com/B3Partners/brmo/master/LICENSE",
            "id1": "pkg:maven/nl.b3p/brmo-init-util@2.0.4-SNAPSHOT"
          }
        },

and some of the other projects modules are also repeated in the sarif file

[jeremylong]

Hopefully d2b5cf9 resolves the issue.

[mprins]

Thanks!

I can see succesful upload when using 6.1.6-SNAPSHOT in https://github.com/B3Partners/brmo/actions/runs/708607190

Run github/codeql-action/upload-sarif@v1
  with:
    sarif_file: target/dependency-check-report.sarif
    checkout_path: /home/runner/work/brmo/brmo
    token: ***
    matrix: {
    "java": 8
  }
  env:
    JAVA_HOME_8.0.282_x64: /opt/hostedtoolcache/jdk/8.0.282/x64
    JAVA_HOME: /opt/hostedtoolcache/jdk/8.0.282/x64
    JAVA_HOME_8_0_282_X64: /opt/hostedtoolcache/jdk/8.0.282/x64
Uploading sarif files: ["target/dependency-check-report.sarif"]
Uploading results
Successfully uploaded results

[xmlking]

@jeremylong @mprins I am still getting artifacts contains duplicate item error with my github actions.
I am using dependencycheck 6.5.3 . please advise if I am doing wrong.

My action file: https://github.com/xmlking/micro-apps/blob/develop/.github/workflows/owasp-dep-check.yml

dependency-check-report.sarif.txt

[jeremylong]

see #3993 - we have a few more things to resolve before releasing 7.0.0, but it shouldn't be too much longer.