Could not perform Node Audit analysis #3717

sranney · 2021-10-08T20:03:18Z

My team has been working over the last couple of days to understand why this error is being thrown in our Jenkins build pipeline:

[2021-10-08T18:09:32.862Z] [DependencyCheck] [INFO] Checking for updates
[2021-10-08T18:09:32.869Z] [DependencyCheck] [INFO] Skipping NVD check since last check was within 4 hours.
[2021-10-08T18:09:32.958Z] [DependencyCheck] [INFO] Check for updates complete (96 ms)
[2021-10-08T18:09:33.112Z] [DependencyCheck] [INFO] 
[2021-10-08T18:09:33.112Z] [DependencyCheck] 
[2021-10-08T18:09:33.112Z] [DependencyCheck] Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user’s risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.
[2021-10-08T18:09:33.112Z] [DependencyCheck] 
[2021-10-08T18:09:33.112Z] [DependencyCheck] 
[2021-10-08T18:09:33.112Z] [DependencyCheck]    About ODC: https://jeremylong.github.io/DependencyCheck/general/internals.html
[2021-10-08T18:09:33.112Z] [DependencyCheck]    False Positives: https://jeremylong.github.io/DependencyCheck/general/suppression.html
[2021-10-08T18:09:33.112Z] [DependencyCheck] 
[2021-10-08T18:09:33.112Z] [DependencyCheck] 💖 Sponsor: https://github.com/sponsors/jeremylong
[2021-10-08T18:09:33.112Z] [DependencyCheck] 
[2021-10-08T18:09:33.112Z] [DependencyCheck] 
[2021-10-08T18:09:33.112Z] [DependencyCheck] [INFO] Analysis Started
[2021-10-08T18:09:33.117Z] [DependencyCheck] [INFO] Finished Archive Analyzer (0 seconds)
[2021-10-08T18:09:33.125Z] [DependencyCheck] [INFO] Finished File Name Analyzer (0 seconds)
[2021-10-08T18:09:33.127Z] [DependencyCheck] [INFO] Finished Dependency Merging Analyzer (0 seconds)
[2021-10-08T18:09:33.131Z] [DependencyCheck] [INFO] Finished Version Filter Analyzer (0 seconds)
[2021-10-08T18:09:33.239Z] [DependencyCheck] [INFO] Finished Hint Analyzer (0 seconds)
[2021-10-08T18:09:33.245Z] [DependencyCheck] [INFO] Finished False Positive Analyzer (0 seconds)
[2021-10-08T18:09:33.248Z] [DependencyCheck] [INFO] Finished NVD CVE Analyzer (0 seconds)
[2021-10-08T18:09:33.250Z] [DependencyCheck] [WARN] The Node Package Analyzer has been disabled; the resulting report will only contain the known vulnerable dependency - not a bill of materials for the node project.
[2021-10-08T18:09:33.687Z] [DependencyCheck] [WARN] dependency skipped: package.json contain an alias for vue-loader-v16 => vue-loader@16.8.1 npm audit doesn't support aliases
[2021-10-08T18:09:34.336Z] [DependencyCheck] [ERROR] NodeAuditAnalyzer failed on /var/lib/jenkins/workspace/NL_ui-components_PR-280/package-lock.json
[2021-10-08T18:09:34.336Z] [DependencyCheck] [WARN] An error occurred while analyzing '/var/lib/jenkins/workspace/NL_ui-components_PR-280/package-lock.json' (Node Audit Analyzer).
[2021-10-08T18:09:34.336Z] [DependencyCheck] [INFO] Finished Node Audit Analyzer (1 seconds)
[2021-10-08T18:09:34.337Z] [DependencyCheck] [INFO] Finished Sonatype OSS Index Analyzer (0 seconds)
[2021-10-08T18:09:34.465Z] [DependencyCheck] [INFO] Finished Vulnerability Suppression Analyzer (0 seconds)
[2021-10-08T18:09:34.465Z] [DependencyCheck] [INFO] Finished Dependency Bundling Analyzer (0 seconds)
[2021-10-08T18:09:34.477Z] [DependencyCheck] [INFO] Analysis Complete (1 seconds)
[2021-10-08T18:09:34.581Z] [DependencyCheck] [INFO] Writing report to: /var/lib/jenkins/workspace/NL_ui-components_PR-280/analysis/dependency-check-report.xml
[2021-10-08T18:09:34.719Z] [DependencyCheck] [INFO] Writing report to: /var/lib/jenkins/workspace/NL_ui-components_PR-280/analysis/dependency-check-report.html
[2021-10-08T18:09:35.107Z] [DependencyCheck] [ERROR] Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API.

I have attached the package.json file and the package-lock.json file for this project in the issue. Can you give me some guidance on what could be causing the above issue, and how to fix it, given this information?
dependency check attachments - package and package-lock.zip

The text was updated successfully, but these errors were encountered:

sranney · 2021-10-19T14:42:14Z

wanted to bump this, in hopes we can get some help. Thanks!

rlucha · 2021-11-03T18:55:37Z

We are observing the same failure, and so far it looks flaky. We cannot see anything in the affected package.json or package-lock.json that could trigger this error.

Jakeb98 · 2021-11-04T22:17:40Z

Also getting lots of these errors when trying to implement Dependency-Check into our project. We have a fairly large project with many sub-projects that each have their own package-lock.json, and it works fine for many and gives the above error for others. We haven't been able to find anything in the failing ones that differentiates them from the passing ones to cause the error.

jeremylong · 2021-11-05T12:26:45Z

Pretty sure this will be fixed in 6.5.0 which was slightly delayed but should be released this weekend.

jonnybbb · 2021-11-11T09:38:28Z

@jeremylong it doesn't seem to be fixed for us. Can you point me to the code change that supposed to fix the problem. What information do you need to get the insides you need to better understand the issue we face?

racedale · 2021-11-11T14:53:25Z

6.5.0 did not fix this for us. Seems to be due to package-locks generated with newer versions of npm (7, 8)

StudentBlake · 2021-11-17T14:10:50Z

Can confirm, it’s totally broken on package-lock.json files generated with latest stable version of npm (8). For us, this is on multiple projects, not sure if it’s every case. This has been an issue for a while, hoping it can get a bump in priority.

Samuel-Schober-USU · 2021-12-23T15:32:24Z

We have the same issue with package-lock.json files generated by npm 8, so many of our CI/CD pipelines are breaking. The current workaround to allow the dependency scan job to fail is not really a good approach, so we would also love if the priority of this issue could be bumped.

Just an idea, but maybe the change from lockfileVersion 1 to lockfileVersion 2 causes some troubles?
https://docs.npmjs.com/cli/v8/configuring-npm/package-lock-json#lockfileversion

oliverlockwood · 2022-01-19T15:44:23Z

Interestingly, I'm finding that:

if I force usage of Node 14 (and hence NPM version 6.4.15, at the time of writing) this issue does not occur.
if I force usage of Node 12 (and hence NPM version 6.4.15, at the time of writing) this issue does not occur.
if I force usage of Node 10 (and hence NPM version 6.14.12, at the time of writing) this issue does not occur.

Whereas:

if I force usage of Node 16 (and hence NPM version 8.1.2, at the time of writing) this issue occurs consistently.

So this seems to confirm @racedale, @StudentBlake and @Samuel-Schober-USU's suspicions; and makes it clear that there is a sort-of-workaround for now (i.e. force Node 14 and regenerate package-lock.json), though it is clearly a bit ugly for any projects that are using NodeJS >14.

Hopefully @jeremylong this is helpful to you in narrowing down the issue. Perhaps you can experiment locally with NodeJS 16 and thereby identify the root of the problem.

Oh, and I upgraded to the very latest Dependency-Check 6.5.3 - same behaviour.

diogofscmariano · 2022-02-17T14:37:45Z

Same thing happening here. CI/CD are failing with this error and the move was to live with the existing findings.
Project relying on Node 14, and NodeJS Audit checks are failing. Any update?

kascaks · 2022-03-11T13:00:24Z

I had the same problem.

In my case the situation was that on dev machine node14/npm6 was installed which generated package-lock.json with
lockfileVersion 1 that was commited to git (node_modules are not commited) from which it was picked up by CI with node16/npm8. DependencyCheck was running fine on dev, but failing on CI.

After some try-and-error I solved it by removing node14/npm6, deleting node_modules and package-lock.json then installing node16/npm8 and recreating node_modules and package-lock.json (by npm install) on dev and commited new package-lock.json which now had lockfileVersion 2. DependencyCheck now runs fine on both dev and CI.

It's also probably worth mentioning that I'm running DependencyCheck as maven plugin 7.0.0

vidal7 · 2022-05-09T13:59:12Z

So, I understand that we cannot use DependencyCheck anymore with Node 16+ or npm 7/8 and a package-lock.json with lockFileVersion 2 ?

ryandutton · 2022-05-19T08:20:06Z

That's correct, the payload which is sent to the NPM API is invalid resulting in an error being returned by the API. The npm audit cli command against the same project with the same package-lock.json file will succeed.

Janpopan · 2022-06-20T08:49:30Z

There is an other ticket around this checker (and the problem ? )
#4536

mum-viadee · 2022-06-21T14:34:59Z

I made an interesting observation. I took the submitted payload from the logfile and posted it using curl in a shell. Surprisingly it worked and I got an HTTP status 200 and a valid response. Then I tried it with a small Java program that uses code snippets from the owasp dependency check. That didn't work. The difference I noticed, after looking at the curl request in detail was the content-type header. Curl, per default, posts files as "application/x-www-form-urlendoded". After changing the code to set the same header as curl does, it worked. Changing the header in curl got me a 400 Bad request likewise.

It looks like a strange behaviour to me, does anyone have any documentation for die npm audit api?

ryandutton · 2022-06-21T15:16:41Z

I've been doing a bit of digging and one of the things I found was that the content-type needs to be set as application/json if running using curl, as suggested by @mum-viadee. The second thing I found was that since npm v7 the npm/cli project uses Arborist to generate the dependency tree. It looks like it then submits the tree to the /-/npm/v1/security/advisories/bulk endpoint, exposing the report inside the npm/cli codebase. As Dependency Check is Java it makes this much more complicated to fix as it will have to essentially mirror the structuring that arborist does.

itsecforu · 2022-06-28T08:42:07Z

the same isuue with:

Dependency-Check Core version 7.1.1

if i grep "error" depcheck.log:

2022-06-28 16:08:29,482 org.owasp.dependencycheck.App:211
DEBUG - unexpected [32;102;21Merror
org.owasp.dependencycheck.analyzer.exception.SearchException: Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API.
<------>at org.owasp.dependencycheck.data.nodeaudit.NodeAuditSearch.submitPackage(NodeAuditSearch.java:209)
<------>at org.owasp.dependencycheck.data.nodeaudit.NodeAuditSearch.submitPackage(NodeAuditSearch.java:133)
<------>at org.owasp.dependencycheck.analyzer.NodeAuditAnalyzer.analyzePackage(NodeAuditAnalyzer.java:189)
<------>at org.owasp.dependencycheck.analyzer.NodeAuditAnalyzer.analyzeDependency(NodeAuditAnalyzer.java:146)
<------>at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
<------>at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
<------>at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
<------>at java.util.concurrent.FutureTask.run(FutureTask.java:266)
<------>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
<------>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
<------>at java.lang.Thread.run(Thread.java:748)
2022-06-28 16:08:29,482 org.owasp.dependencycheck.utils.Settings:821
DEBUG - Deleting ALL temporary files from `/tmp/dctempf41abb9a-4037-44d1-bb85-775cf18c273c`
2022-06-28 16:08:29,520 org.owasp.dependencycheck.App:84
DEBUG - Exit code: -14

could the fact that the package-lock.json file is 2.5 megabytes be a problem?

Benoit-Vasseur · 2022-07-12T08:43:16Z

Hello,

Is there a fix planned and or identified :) ?
We've just upgraded our npm version on CI (v6 -> v8) and it now fail on some projects. (Node16 is the new LTS)

v1shva · 2022-07-19T15:13:40Z

I had the same problem.

In my case the situation was that on dev machine node14/npm6 was installed which generated package-lock.json with lockfileVersion 1 that was commited to git (node_modules are not commited) from which it was picked up by CI with node16/npm8. DependencyCheck was running fine on dev, but failing on CI.

After some try-and-error I solved it by removing node14/npm6, deleting node_modules and package-lock.json then installing node16/npm8 and recreating node_modules and package-lock.json (by npm install) on dev and commited new package-lock.json which now had lockfileVersion 2. DependencyCheck now runs fine on both dev and CI.

It's also probably worth mentioning that I'm running DependencyCheck as maven plugin 7.0.0

We already had our package-lock.json upgraded to "lockfileVersion": 2 still it was failing. I removed the package-lock.json and node_modules folder and did a npm install (in node 16.14.0) and it did fix the issue.

alex-arzaghi · 2023-01-15T14:35:48Z

Pretty sure this will be fixed in 6.5.0 which was slightly delayed but should be released this weekend.

@jeremylong Has this version/fix been releases?

oliverlockwood · 2023-02-02T11:42:24Z

@alex-arzaghi FYI in our pipelines, we're still using Node 14 specifically for Dependency-Check, due to this issue 😞

delenikov · 2023-05-17T07:05:15Z

Check this #3716 (comment)

StudentBlake · 2023-05-26T13:10:39Z

Check this #3716 (comment)

Is there a way to specify the endpoint for the cli? I don't see that param in the documentation.

itineric · 2023-06-06T15:00:52Z

The option does not exist on command line but can be provided using the properties file.

Add the following to your existing properties file
analyzer.node.audit.url=/-/npm/v1/security/advisories/bulk

And if not already using a properties file, create one and then add -P my-config.properties to the command line

paul-dyson · 2023-08-03T11:11:18Z

What seemed to work for me was to reinstate the resolved and integrity values in package-lock.json that npm v8 had lost by running

npx npm@6.14.18 install && npm install

After which the previously failing Node Audit analysis completed without error.

adcorduneanu · 2023-08-11T06:47:43Z

This still happens in 8.2.1, with node 18.x.

fascynacja · 2023-11-22T10:40:13Z

Did anyone found a workaround for this issue? I am facing same problem with command:

mvn org.owasp:dependency-check-maven:7.3.2:aggregate
and node version: 14.x.x

Samuel-Schober-USU mentioned this issue Dec 23, 2021

Invalid payload submitted to Node Audit API. Received response code: 400 Bad Request #3716

Open

ryandutton mentioned this issue Jun 23, 2022

[FEATURE](arborist) A way to use new Arborist() by providing the package.json / lock file directly npm/cli#4295

Closed

nhumblot mentioned this issue Aug 9, 2022

@ sign in npm package-names leads to an exception #4412

Open

phoenix741 mentioned this issue Nov 21, 2022

fix: add package-lock v2 and v3 #5078

Merged

alex-arzaghi mentioned this issue Jan 15, 2023

The workflow fails with locally referenced modules dependency-check/Dependency-Check_Action#18

Closed

phoenix741 mentioned this issue May 5, 2023

fix: fix npm alias present in requires of dependencies #5703

Merged

grauds added a commit to grauds/money.tracker.ui that referenced this issue May 19, 2023

[+] Fixed dep check (jeremylong/DependencyCheck#3717)

ac0798d

grauds added a commit to grauds/money.tracker.ui that referenced this issue May 19, 2023

[+] Fixed dep check (jeremylong/DependencyCheck#3717)

1705f6b

grauds added a commit to grauds/money.tracker.ui that referenced this issue May 19, 2023

[+] Fixed dep check (jeremylong/DependencyCheck#3717)

2572edf

aikebah mentioned this issue Jun 27, 2023

[ERROR] Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API. #5792

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Could not perform Node Audit analysis #3717

Could not perform Node Audit analysis #3717

sranney commented Oct 8, 2021 •

edited

sranney commented Oct 19, 2021

rlucha commented Nov 3, 2021

Jakeb98 commented Nov 4, 2021

jeremylong commented Nov 5, 2021

jonnybbb commented Nov 11, 2021

racedale commented Nov 11, 2021

StudentBlake commented Nov 17, 2021

Samuel-Schober-USU commented Dec 23, 2021 •

edited

oliverlockwood commented Jan 19, 2022 •

edited

diogofscmariano commented Feb 17, 2022

kascaks commented Mar 11, 2022

vidal7 commented May 9, 2022 •

edited

ryandutton commented May 19, 2022

Janpopan commented Jun 20, 2022

mum-viadee commented Jun 21, 2022 •

edited

ryandutton commented Jun 21, 2022

itsecforu commented Jun 28, 2022 •

edited

Benoit-Vasseur commented Jul 12, 2022 •

edited

v1shva commented Jul 19, 2022 •

edited

alex-arzaghi commented Jan 15, 2023

oliverlockwood commented Feb 2, 2023

delenikov commented May 17, 2023 •

edited

StudentBlake commented May 26, 2023

itineric commented Jun 6, 2023 •

edited

paul-dyson commented Aug 3, 2023

adcorduneanu commented Aug 11, 2023

fascynacja commented Nov 22, 2023

Could not perform Node Audit analysis #3717

Could not perform Node Audit analysis #3717

Comments

sranney commented Oct 8, 2021 • edited

sranney commented Oct 19, 2021

rlucha commented Nov 3, 2021

Jakeb98 commented Nov 4, 2021

jeremylong commented Nov 5, 2021

jonnybbb commented Nov 11, 2021

racedale commented Nov 11, 2021

StudentBlake commented Nov 17, 2021

Samuel-Schober-USU commented Dec 23, 2021 • edited

oliverlockwood commented Jan 19, 2022 • edited

diogofscmariano commented Feb 17, 2022

kascaks commented Mar 11, 2022

vidal7 commented May 9, 2022 • edited

ryandutton commented May 19, 2022

Janpopan commented Jun 20, 2022

mum-viadee commented Jun 21, 2022 • edited

ryandutton commented Jun 21, 2022

itsecforu commented Jun 28, 2022 • edited

Benoit-Vasseur commented Jul 12, 2022 • edited

v1shva commented Jul 19, 2022 • edited

alex-arzaghi commented Jan 15, 2023

oliverlockwood commented Feb 2, 2023

delenikov commented May 17, 2023 • edited

StudentBlake commented May 26, 2023

itineric commented Jun 6, 2023 • edited

paul-dyson commented Aug 3, 2023

adcorduneanu commented Aug 11, 2023

fascynacja commented Nov 22, 2023

sranney commented Oct 8, 2021 •

edited

Samuel-Schober-USU commented Dec 23, 2021 •

edited

oliverlockwood commented Jan 19, 2022 •

edited

vidal7 commented May 9, 2022 •

edited

mum-viadee commented Jun 21, 2022 •

edited

itsecforu commented Jun 28, 2022 •

edited

Benoit-Vasseur commented Jul 12, 2022 •

edited

v1shva commented Jul 19, 2022 •

edited

delenikov commented May 17, 2023 •

edited

itineric commented Jun 6, 2023 •

edited