User experience of certifying multiple version bumps is slow

[eqrion]

This may totally be just me, or I may be missing something. I recently bumped the version of 4 crates in mozilla-central.

This lead to:

 rhunt@Ryans-MacBook-Pro-2 fx % ./mach cargo vet
Vetting Failed!

4 unvetted dependencies:
  wasm-smith:0.11.8 missing ["safe-to-run"]
  wasmparser:0.94.0 missing ["safe-to-deploy"]
  wast:49.0.0 missing ["safe-to-deploy"]
  wasm-encoder:0.19.1 likely missing ["safe-to-deploy"]

recommended audits for safe-to-deploy:
    cargo vet diff wast 48.0.0 49.0.0          (used by jsrust)              (9 files changed, 40 insertions(+), 34 deletions(-))
    cargo vet diff wasmparser 0.93.0 0.94.0    (used by jsrust, wasm-smith)  (13 files changed, 2228 insertions(+), 2265 deletions(-))
    cargo vet diff wasm-encoder 0.19.0 0.19.1  (used by wast, wasm-smith)    (4 files changed, 4 insertions(+), 4 deletions(-))

recommended audits for safe-to-run:
    cargo vet diff wasm-smith 0.11.7 0.11.8  (used by gluesmith)  (10 files changed, 2598 insertions(+), 1403 deletions(-))

estimated audit backlog: 8576 lines

Use |cargo vet certify| to record the audits.

My first reaction was to just run cargo vet certify as that's the last thing on the screen (and also because I've audited these crates before, and been involved in their development). This takes me to:

rhunt@Ryans-MacBook-Pro-2 fx % ./mach cargo vet certify

choose criteria to certify for wast:44.0.0
  0. <clear selections>
  1. safe-to-run
  2. safe-to-deploy

Note: this is for wast 44.0.0 which was from months ago when I last did an update. So running cargo vet immediately before this didn't seem to clear the state for certify, or set it up for one of diffs it prompted about.

I then decide to run each diff invocation followed by vet certify. The easiest way to do this correctly seemed to be to just copy and paste the diff invocation from the original vet command run. This did the correct thing, but was a bit tedious.

Am I missing something? If not, it seems like it would be nice if the magic of just running 'certify' and it guessing what you need could be extended to make this process a bit quicker.

[Gankra]

Yeah vet isn't really particularly optimized for "yes I wrote this, it's fine" as a workflow.

You can do cargo vet wasmparser 0.93.0 0.94.0 and it should do the same intelligent stuff that bare cargo vet certify would do for that version. Maybe we should have the date of the cached cargo vet diff command stored so we can have it expire after a ~day or something.

We have a prototype of #330 (which will eventually grow to cover m-c), and in that system you just quickly click on a bunch of tabs and click a bunch of checkboxes).

[bholley]

You can do cargo vet wasmparser 0.93.0 0.94.0

I think you mean cargo vet certify wasmparser 0.93.0 0.94.0

I think #353 should significantly improve the general case of managing stuff we already know is good. That said, I think one small thing we should do here is to expire "last inspected crate" recall after a day or so.