npm audit and audit-resolve.json

[naugtur]

[edit] This RFC is very old, some progress on developing the idea led to a Collab Space in OpenJSF and this format proposal: openjs-foundation/pkg-vuln-collab-space#11

Was: npm audit resolve
Updated to only cover the support for audit-resolve.json file in npm.

Currently implemented in userland here: https://www.npmjs.com/package/npm-audit-resolver

The check-audit command from the above package is providing the exact functionality proposed here.

Other places where this was discussed:

npm/cli#10
https://npm.community/t/interactive-tool-to-manage-audit-findings-npm-audit-resolve/197

[ljharb]

The next working day is often more than 24 hours later; is this just meant as an example?

[naugtur]

Yes. I didn't intend it to count in working days.

If noone's at work and the build turns red, does it make a sound?

[ljharb]

Presumably this could also narrow the list of action items, if some of them are from transitive deps of the removed item?

[naugtur]

In my experience audit grouped these together nicely, so when I choose remove it's unlikely the dependency or any of its children would come up again.

But we could easily run audit again after the remove command and thanks to audit-resolv being written after each decision, it'd automatically continue from the same point on the list.

[CADBOT]

What's the latest on this? Would be great to have something to replace .nsprc/exceptions from nsp.

[naugtur]

I don't really know what's next other than moving my repo to npm org on GitHub.
There's no disagreement on anything here.

I have more free time than usual for the next few days so I'd be happy to move this forward too.

But you can use it now. Install npm-audit-resolver. I'm using it at work with my team for a while now

[evilpacket]

How do we handle the situation in which somebody keeps hitting snooze (postpone) on a vulnerability. Do we really have to rely on git commit history to see this? Should we track all history not just the current state / decisions? This adds complexity that we maybe don't need but also could be a useful audit trail?

[naugtur]

I thought about a few cases of audit trail with similar conclusions. IMHO the benefit of audit-resolv file's simplicity outweighs the inconvenience of using git history. Also, people are quite familiar with git so the less we do on our end, the less learning they have to do.

I'd like to avoid the file growing to overwhelming lengths, but other than that I'm fine with adding more capability to this file format if it's useful for the server side ignores etc.

[naugtur]

Leaving only the 24h option it should be annoying enough for most reasonable people to eventually fix. :)
Unless someone gets a job postponing vulnerabilities every morning before the team comes in.

With the audit-resolve format change I just pushed we can also add a count${dependency path}er. I'd consider that a nice touch, not the core functionality though.

[naugtur]

Update - instead of letting people postpone for more than 24h, due to a popular demand, I got an option in to make ignores expire. So if you decide to ignore something because there's no fix and it doesn't seem accessible from the code that depends on it, it's possible to make it expire.

Added expiresAt field to the decision object in resolve file

[evilpacket]

Can you document the format of this file? Also adding in the option to have a reason for each decision.

[evilpacket]

Another though is that the audit-resolve.json file should not be published as part of a module publication. We will want to assert exceptions outside of the package.json and update those exceptions outside of the normal publication lifecycle.

[naugtur]

Will update the RFC to contain the file format spec

[naugtur]

only the file next to top level package.json should be taken into account at all (and that's how it's implemented in npm-audit-resolver)

[naugtur]

After some tweaking I updated the RFC to include a precise format definition. Same format definition is used in a branch where I'm working on refactoring.

npm-audit-resolver package after the refactoring should be trivial to integrate into npm audit for the purpose of respecting decisions made in audit-resolve.json

[naugtur]

Schema is defined here: https://github.com/naugtur/npm-audit-resolver/blob/refactoring/src/core/auditFile/versions/v1.js as part of the app - for validation

[evilpacket]

In general I'm very excited about the potential of the npm audit resolve feature. I think some work needs to be done to really make it a good interactive experience presenting the information in the best way possible for users.

[zkat]

(Heads up, we are tracking this internally and it's something we intend to get to, but I can't give you an exact timeline right now. Thanks for your patience.)

[naugtur]

If you want to consider one bigger push to move this forward, my family is out for winter vacation 25-28Feb while I stay at work, so I'll have unusually many hours in the evenings in that period.

[ersel]

This would be an awesome addition to npm CLI. We used to run npm audit on CI with a custom script outlined here.

Recently we've switched to @naugtur 's npm-resolve-audit module by installing it as a dev dependency and adding a script to run node node_modules/npm-audit-resolver/check.js

Thanks for building this tool @naugtur !

[naugtur]

@ersel if you're running it inside an npm script, you can use it as a command check-audit and it resolves from dependencies. Otherwise I'd go with npx -p npm-audit-resolver check-audit - feels a little cleaner and you don't need to install resolver as a dependency - npx will download and cache it. (npx is shipped with npm so no need to install it)

Check out my slides on using audit in CI (and other things) https://naugtur.pl/pres3/securedev/#/18

[naugtur]

I've updated the RFC and I'm working on a refactoring to separate the core features from npm-related and interactive ones.

Could we agree on making npm audit use audit-resolve.json being the first step?
That way people could produce the file manually or with the interactive resolver, but the means of ignoring a package would be there.

[naugtur]

Hi,

I'm finishing the refactoring I mentioned before.
My suggestion to proceed now is that npm (and potentially other package managers) can use the core of npm-audit-resolver to include facts from audit-reslve.json when checking the repository with npm audit command - therefore skipping issues that have been addressed before.
The interactive part can (initially or forever) remain in the separate package with potential competitors, while the spec for audit-resolve.json becomes something we can slowly standardize.

I will open another PR to npm some time soon introducing the line to include decisions from audit-resolve.json in the npm audit logic.

Edit:
I should probably let you know @zkat @evilpacket

[naugtur]

@jfaylon There's a collaboration space being set up under the OpenJS Foundation to tackle this.

For now you can use https://www.npmjs.com/package/npm-audit-resolver for your vulnerability management. My team's been doing that for a few years now.

[bnb]

I would prefer either doing this through a more generic audit.json or through an audit property in package.json. Having a whole new file for a single purpose in the npm CLI feels… off.

There was discussion of additional config setups that I had with @darcyclarke that it could also potentially fit in, but they don’t exist yet so I can’t really recommend them.

[naugtur]

I didn't go with a package.json field because everyone kept saying we use it for too many things already and the audit file tends to be larger (from my practical experience of running the tool for 3-4 years)
Also, there's a plan to use it as vehicle for sharing recommendation. More on this here:
https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-3olf

I'm not opinionated about the name. The final decision on the name will have to happen when we standardize the format in OpenJSF

[ljharb]

Other advantages of keeping it in a separate file include more granular codeowners targeting.

[bnb]

I'll make it explicit here so it can be referred to in whatever OpenJSF work you're talking about: I'm a hard -1 on audit-resolve.json.

[naugtur]

audit.json is fine.
OpenJSF work here
https://github.com/openjs-foundation/pkg-vuln-collab-space

[karlhorky]

that's not exactly good security practice

users should actually be doing their due diligence here with their dependencies and that

I think it should be fully considered that a large number of projects will either:

• not have maintainers that have the bandwidth to do this
• not have maintainers that know how to do this
• not have maintainers that know this needs to be done at all
• not have maintainers (unmaintained / insufficiently maintained)

I work with a bunch of beginners every three months in our coding bootcamp at @upleveled and this is one example of maintainers who will not do this. And if you pick random packages on npm, more likely than not, you'll have something that is not well maintained.

So to save and check the audit decision as close as possible to the actual usage of the vulnerable code seems like actually a better security practice overall for the industry (at least as a fallback option).

Affecting sea change so that everyone in the industry spends time to check the security of their dependencies is unrealistic.

I think @naugtur's suggestion here could be one interesting way to implement this:

package maintainers could publish ignore lists for the dependencies of their package when they did the research to verify the package is not affected.

Such lists could be aggregated into recommendations maintained by security professionals.

To be clear, again, I'm not proposing this as the ONLY security measure - teams which have security processes, budget and practice in place should continue to do that and thus will be even more secure.

But for the majority of projects which will not have this level of focus on security, there should be a base level of security by default, provided by the ecosystem.

[naugtur]

The idea is to give people tools to do the best they can with the bandwidth they have. If we force them to address everything on their own, they're likely to just drop security from their pipeline

[dead-claudia]

And I'm just saying we should just not hide it by default, and instead force the user to take some explicit action, even if it's not necessarily actually looking at the code and confirming everything.

[ljharb]

We shouldn't - the user should never see something that isn't actionable for them.

[dead-claudia]

I'll partially agree to disagree and move on.

[bnb]

[edit] This RFC is very old, some progress on developing the idea led to a Collab Space in OpenJSF and this format proposal: openjs-foundation/pkg-vuln-collab-space#11

if this RFC is not reflective of a proposal, should it be closed? It's already been extremely misleading given that it's not the "real" representation of what the proposal is.

[naugtur]

if this RFC is not reflective of a proposal, should it be closed? It's already been extremely misleading given that it's not the "real" representation of what the proposal is.

After the last meeting I knew I should rewrite it now and I know what to put in it. Family matters and chores got in the way. I will rewrite it to reflect what it should.

[nukithelegend]

Are there any progress updates on this? With the new move of the advisory database of NPM now on GitHub (https://github.blog/changelog/2021-10-03-the-npm-advisory-database-is-now-part-of-the-github-advisory-database/) some of the moderate vulnerabilities that we can't resolve right now have been tagged as High - breaking our builds.

It would be great if we can accept the risks on them specifically to allow the build to pass again.

[trevyn]

Looks like it didn't actually get discussed at the meeting: https://youtu.be/nkohPHyvzw4?t=3582

@bnb Sounds like this was an action item for you; did you get a chance to look at the state of this?

[bnb]

@bnb Sounds like this was an action item for you; did you get a chance to look at the state of this?

I'm not going to try to be speculative on an RFC that the creator has very specific ideas about without it actually being in a PR. I don't think that'd be particularly fair to them or me 😅

[trevyn]

Ok, so we're waiting on @naugtur , got it! :)

[devdev-dev]

Waiting for this like many others...

[bertolo1988]

Still waiting for this.

[naugtur]

BTW. I'm very near releasing a new major version of npm-audit-resolver, if anyone wants to help out.

Happy to go back to discussing getting the policy check into npm cli. Still bullish on not doing it per package name but per dependency path.

Also, it used to be a PR before it was an RFC :D
I'm gonna need to figure out what the expectations are now.

[bertolo1988]

if anyone wants to help out.

I can help, you can reach me using the contacts in my Github profile.

[hax]

Any update? We need it to bypass some common packages in the third-party dependencies (eg. lodash/lodash#5808 ) which block our build steps. The best we can do now is just totally disable npm audit in the build step. 🫤

[naugtur]

You can use npm-audit-resolver - I've released a version with updated support last year.

This RFC is unlikely to move forward. npm is maintained by a small team and unlikely to make major new work happen.