[wip] yarn 1.16, safety, audit in CI 🧶🧷🚨

[bollwyvl]

References

• fixes Upgrade to yarn 1.16, use audit (and safety)? #6331

Code changes

• Upgrade staging and devDependency yarn to 1.16.0
  • from https://github.com/yarnpkg/yarn/releases/download/v1.16.0/yarn-legacy-1.16.0.js
  • enhances yarn audit
  • bonus: adds some better retry behavior after timeout
• Adds safety
• Adds audit CI step to invoke yarn audit and safety check
  • me: handle the seemingly common 503 error for audit (but probably other errors for pypi)
  • we: decide what to do with findings
    • publish as build artifact?
    • create issue?

User-facing changes

• None yet.

Backwards-incompatible changes

• None.

Alternatives

While upgrading yarn seems like a fine idea, and audit could allow us to help users understand potential ramifications of installing extensions, we could alternatively go with...

audit-ci

IBM/audit-ci is much more configurable than the built-in yarn audit. For example, for Reasons, let's say we can't upgrade marked right now, which generates a couple hundred moderate warnings. audit-ci would let us ignore the specific, numbered advisories related to regex explosion by adding a line to its config file. This would give us an "out" when a vulnerability shows up, as the chances of a well-sorted JSON file leading to merge conflicts from multiple PRs is far less impactful than a bunch of package.json and yarn.lock changes.

snyk.io, pyup.io

Instead of rolling our own, go with bot-based services. Both offer free accounts for open source projects. As I mentioned on #6331, yarn audit depends on the largesse of npm, but is a POST request of basically our whole yarn.lock, which seems to 503 fairly frequently (at least in the last 24 ours). By having the bots call us, we'll likely get a better workflow, and burn less CI coal.

🛎️ @jasongrout

[jupyterlab-dev-mode]

Thanks for making a pull request to JupyterLab!

To try out this branch on binder, follow this link:

[bollwyvl]

So it found one high in staging:

++ jlpm audit --level high
~/work/1/s/jupyterlab/staging ~/work/1/s
yarn audit v1.16.0
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ high          │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ handlebars                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=4.0.14 <4.1.0 || >=4.1.2                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ handlebars                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ handlebars                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/755                         │
└───────────────┴──────────────────────────────────────────────────────────────┘
231 vulnerabilities found - Packages audited: 618847
Severity: 230 Moderate | 1 High

Hooray data! However, it didn't make it to the dev yarn.lock.

Perhaps the right place to put these in CI in an xunit so it can be aggregated with other stuff. I set about doing a bit of this (lazily, templating with jinja, which you Should Never Do, Right? because stdlib xml is awful) and got to thinking: it might in fact make sense to make a jupyter lab audit right out of the gate.

In the initial case of using it for CI, it should be able to emit --xunit (or --json... heck, all of our commands should have a --json, but that's another story). It would need a --dev-mode (not strictly accurate, but...) to pick up the root yarn.lock.

(Paranoid) end users would be able to make use of it, and heck, lab build could even accept an --audit flag to reuse the machinery before even attempting the build... though who knows what damage would already be done at that point. Anyhooo...

Building it "right" would also mean it would work with custom --app-dirs. I would be pretty happy to have this information for my crazy confections.

Thoughts?

[bollwyvl]

So i have yet to get a real bead on why the root doesn't work. It might be a number of things:

• yarn audit does not work in a monorepo if the root does not contain name and version yarnpkg/yarn#6734
• Address yarn audit service reliability yarnpkg/yarn#7202 (comment)

[jasongrout]

@bollwyvl - do you want to take this up for jlab 2.0? We'd like to minimally upgrade yarn, but if some more of this work can get in, that would be great too.

[vidartf]

Some xrefs: #3759 #333

[blink1073]

Closing as stale