npm shrinkwrap: from

[charlierudolph]

When you first create a shrinkwrap the from field is something like

"JSONStream@>=0.8.3 <0.9.0",

If you then update one module and run shrinkwrap again, the from fields are updated to be equal to the resolved field.

Personally I don't think the from field is very useful and would prefer it to be removed if this type of thing is going to happen.

On that note I don't see why shrinkwrap needs to hold the version either. I would think the resolved path is enough.

[smikes]

The from and version fields in the shrinkwrap file are used as backups in the case that the resolved is missing. See the code here: https://github.com/npm/npm/blob/master/lib/install.js#L357

Since this is an existing feature of npm and npm ls (the base command that generates the shrinkwrap data), it could not be removed without breaking backwards compatibility. Is there some scenario where this is causing you trouble, or is your concern primarily aesthetic?

[charlierudolph]

Okay cool. Thanks for the explanation. I'll give a more specific example.

Lets say I have a simple project with the following in my package.json

"dependencies": {
  "lodash": "^3.0.0"
}

And I npm install and then npm shrinkwrap which gives me the following npm-shrinkwrap.json:

{
  "name": "test",
  "version": "1.0.0",
  "dependencies": {
    "lodash": {
      "version": "3.1.0",
      "from": "lodash@>=3.0.0 <4.0.0",
      "resolved": "https://registry.npmjs.org/lodash/-/lodash-3.1.0.tgz"
    }
  }
}

Lets say I commit that to my repo. Then someone else clones the repo and runs npm install.
and then npm shrinkwrap which gives them the following npm-shrinkwrap.json:

{
  "name": "test",
  "version": "1.0.0",
  "dependencies": {
    "lodash": {
      "version": "3.1.0",
      "from": "https://registry.npmjs.org/lodash/-/lodash-3.1.0.tgz",
      "resolved": "https://registry.npmjs.org/lodash/-/lodash-3.1.0.tgz"
    }
  }
}

Now that original from data has been lost. Thus installing from a shrinkwrap and then running npm shrinkwrap causes the shrinkwrap to change. That is the problem to me.

A more realistic example would be I install via shrinkwrap, update one module, and then shrinkwrap. When I create pull request with the updated shrinkwrap file, it will not only have updated the module I wanted to, but have also changed all the from fields.

[smikes]

It sounds like you want the differences between successive shrinkwrap files to reflect only meaningful changes. I don't know if that is possible with the way npm handles installation now (in the 2.x line), but things are slated to change soon, in npm@3, when the multi-stage-install feature lands.

For now, though, it might help if you think of the shrinkwrap file as an opaque, even a binary file, and use a different communications channel such as commit messages or a ChangeLog to describe the difference between successive versions.

/cc @othiym23 as a feature-request and /cc @iarna for your comments, if any, on changes to volatility of npm-shrinkwrap.json fields as a consequence of multi-stage-install

[othiym23]

As @smikes has described, from and resolved capture metadata for use by the shrinkwrap implementation, both for fallback purposes and to provide context to the installer. from tells the installer how the version of the dependency that was in node_modules at the time npm shrinkwrap was run got there, and resolved is the explicit tarball URL that from got resolved from. from is taken from the package.json files in node_modules, which is why it changes from a semver range to the tarball URL when you rerun shrinkwrap.

We should probably document how all that is meant to be interpreted at some point, and maybe provide a means to turn that information back into something that conveys that context back to users, but @smikes is right that the correct way to deal with npm-shrinkwrap.json is to think of it as being a binary blob that's just going to change from run to run. Its volatility is normal and expected behavior, and isn't going to change.

[wagenet]

@othiym23 This makes it extremely difficult to tell if unintentional changes snuck into your npm-shrinkwrap.json. I strongly encourage you to reconsider this.

[mixonic]

I'm looking into the possibility of removing the from field from the npm-shrinkwrap.json file. Ideologically having an entry in shrinkwrap change for an unchanged dependency seems wrong, but practically the noise in commit diffs and logs makes using shrinkwrap almost impractical.