[RRFC] make npm update useful for modern package management

[bnb]

Motivation ("The Why")

Today, I wouldn't be surprised if updating packages is more common than installing packages. The npm install DX is pretty good, but npm update is sorely lacking to the point that we've collectively built a lot of tools to try to help.

IMO the growth of tooling in this space is a result of the lack of strong update management being a norm in the package management tools we rely on at the root. I'd really like to see npm be able to support the average (modern) user's dependency update requirements, rather than those users reach to third-party tooling to achieve good dependency updates.

Example

• open up any project older than 1 month and try to update dependencies. it's going to be rough. it should be better.

How

Current Behaviour

• npm update
  • does something but good luck figuring out if your dependencies are actually as up to date as is possible. they're probably not.

Desired Behaviour

• user runs npm update
  • npm does a fresh fetch of dependencies outside of what's already resolved in node_modules and pacakge-lock.json
    • tell user what modules are different
    • allow -y flag for auto-apply
    • let users interactively update dependencies if the -y flag is missing

a different way to look at this:

• GitHub Actions workflow runs npm update -y
  • set up to automatically PR changes applied to package.json if anything changes
  • I should never have to see a Dependabot PR if this workflow were to run and be merged prior to Dependabot being triggered, since npm update would have already updated everything.

yet another way to look at this:

• Dependabot should be able to adopt npm update with limited additional configuration to power their JavaScript update tooling.

References

• Feature Request: npm update : output summary of packages updated and (if available) changelog url #555 is similar but not fundamentally the same as this

[bnb]

from today's meeting, I suggest we:

• add a --range flag
  • this flag's values are semver verbs (i.e. *, ~, ^, etc.)
  • the verbs inform npm update on how to look for new versions that can exceed the versions within the ranges defined in the package.json
  • this flag should have interplay with other features:
    • --save should save the version output by npm update to the package.json
    • --save-prefix should match the existing --save-prefix DX when saving updated packages to package.json

[feelepxyz]

I'm excited about the potential here!

@jurre @mctofu 👋 This npm discussion rfc was created to improve the dx for developers wanting to update their deps but seems like it could be very relevant for Dependabot, ideally a drop-in-replacement for a bunch of logic in core. Do you have any thoughts on how this should work so that Dependabot to make use of it?

If anything, there's quite a few years of customers feedback around npm updates enshrined in dependabot's codebase.

A few things that come to mind:

• Apps and published packages often want to update the requirement in package.json differently, e.g. packages often want to "widen" or relax the range to allow the current and latest version, whereas an app with a lockfile usually cares less about the exact requirement (if updates are easy), and you can update the requirement to the latest version
  • Example for app with committed lockfile: current version is 1.1.1 in lockfile, package.json asks for ^1.0.0, update installs 1.5.0 in lockfile and updates package.json to ^1.5.0
  • Example for a published package: package.json asks for ~1.0.0, the lockfile might not be checked into git so matters less, update the requirement in package.json to ~1.5.0 if we're asked to increase the requirement to the latest version, or to >=1.0.0 <1.6.0 if we're asked to include the current version (widen)
  • This could be set with a separate arg, something like --versioning-strategy=increase (cribbed from dependabot's config)
• Separately to how the package.json requirement is updated, you might only want to allow certain update ranges
  • e.g. something like --allow-updates=patch,minor

[mctofu]

This sounds great!

Like @feelepxyz mentioned, Dependabot does tend to get into some more complicated allowed/ignored version configurations. We can have a mix of user supplied ranges, user preference to ignore patch, minor or major updates as well as specific versions being ignored (so we won't keep opening the same PR if closed by the user).

It would also be helpful to have structured output to describe what changed or what went wrong. Knowing what changed lets us create description PR messages (bumping these deps from version x to version y). Knowing what's wrong without needing to parse it out from text makes our error detection more reliable.

We do also have quite a bit of complexity in Dependabot as a result of needing to patch up the manifest files to work in Dependabot's environment (rewriting ssh deps to https for example). It would help if there were additional options we could use that would bypass that need but that may be out of scope for this issue.

[bnb]

Also additional context on what Cargo does:

https://doc.rust-lang.org/cargo/commands/cargo-update.html

[bnb]

@darcyclarke curious if there's been progress on thinking about this/discussing this?

[bnb]

@darcyclarke should this be closed? Not sure how to parse the lack of reply 😅