[FEATURE] Say which package failed during npm install

[HopefulLlama]

What / Why

During an npm install, getting an error (specifically, in this case a code EINTEGRITY error) does not state which package is throwing the error. This complicates debugging a lot, which the installer should clearly have the information about which package failed.

When

• You run npm install but one or more packages fail to install

Where

• Everywhere

How

Current Behavior

• When a package fails to install, you get a message such as:

npm ERR! code EINTEGRITY
npm ERR! sha512-QucnT0lfX7vxvYctJHARfDm6eaVnwQk37GDMw+NJheWvWYtFCYhdr4ABGnyL7MsNWlBB3LQta7EO9OwXucUhXQ== integrity checksum failed when using sha512: wanted sha512-QucnT0lfX7vxvYctJHARfDm6eaVnwQk37GDMw+NJheWvWYtFCYhdr4ABGnyL7MsNWlBB3LQta7EO9OwXucUhXQ== but got sha512-wZdSQMfJh+ZlD4BLXkC+1Y2VHyN2HDGUJQOh/QBlKPXkGQlwDQEk7/ufGrPhE2O2LkShMK19nNjNCyRP8zimFg==. (4976 bytes)

Expected Behavior

• It should state the package name in addition to the error:

npm ERR! example module failed to install with the following message:

npm ERR! code EINTEGRITY
npm ERR! sha512-QucnT0lfX7vxvYctJHARfDm6eaVnwQk37GDMw+NJheWvWYtFCYhdr4ABGnyL7MsNWlBB3LQta7EO9OwXucUhXQ== integrity checksum failed when using sha512: wanted sha512-QucnT0lfX7vxvYctJHARfDm6eaVnwQk37GDMw+NJheWvWYtFCYhdr4ABGnyL7MsNWlBB3LQta7EO9OwXucUhXQ== but got sha512-wZdSQMfJh+ZlD4BLXkC+1Y2VHyN2HDGUJQOh/QBlKPXkGQlwDQEk7/ufGrPhE2O2LkShMK19nNjNCyRP8zimFg==. (4976 bytes)

Who

• @npm/cli

References

• ?

[vince1995]

refers to #298 maybe

[x-yuri]

#298 is too vague, and no easy way to reproduce. In any case, I came here not with a private package, npm cache verify has nothing to do with my case, and it's not intermittent.

Today, I tried in install packages (npm i), but it refused owing the "integrity checksum failed" error. Following a number of convoluted steps I found the culprit: shipit-shared@4.4.2. I have no better guess than it was unpublished, and then published again at some point. Which led to it changing its integrity checksum. It may have something to do with incorporating shipit packages into one repository, or not. I'm not sure. Or maybe some checksums on https://registry.npmjs.org were updated.

The steps to reproduce Anyways, the steps to reproduce are as follows. Sometimes it's more elaborate (wordy), sometimes not. And the reason why it is so is a race condition.

The output is supposed to be as follows:

npm WARN tarball tarball data for shipit-shared@4.4.2 (sha1-MRp5kqeH932y5IwJEZvc4EOsjfI=) seems to be corrupted. Trying one more time.
npm WARN tarball tarball data for shipit-shared@4.4.2 (sha1-MRp5kqeH932y5IwJEZvc4EOsjfI=) seems to be corrupted. Trying one more time.
npm WARN app No description
npm WARN app No repository field.
npm WARN app No license field.

npm ERR! code EINTEGRITY
npm ERR! Verification failed while extracting shipit-shared@4.4.2:
npm ERR! Verification failed while extracting shipit-shared@4.4.2:
npm ERR! sha1-MRp5kqeH932y5IwJEZvc4EOsjfI= integrity checksum failed when using sha1: wanted sha1-MRp5kqeH932y5IwJEZvc4EOsjfI= but got sha512-ze/W5oDJlq/tvycego5u1utRhZPOASkcsNr/y2vdus6RbDa5CimMXT13wU8GDLKYy4wejPbwnnDoKnFMN4LVnA== sha1-ogtcyYWb1myofyutiDlrx5j4KpI=. (5988 bytes)

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-05-28T15_36_54_324Z-debug.log

But under some conditions an exception is thrown and never caught, and the global event handler takes control. It doesn't know much, and such is the result:

npm ERR! code EINTEGRITY
npm ERR! sha1-MRp5kqeH932y5IwJEZvc4EOsjfI= integrity checksum failed when using sha1: wanted sha1-MRp5kqeH932y5IwJEZvc4EOsjfI= but got sha512-ze/W5oDJlq/tvycego5u1utRhZPOASkcsNr/y2vdus6RbDa5CimMXT13wU8GDLKYy4wejPbwnnDoKnFMN4LVnA== sha1-ogtcyYWb1myofyutiDlrx5j4KpI=. (5988 bytes)

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-05-28T15_37_07_496Z-debug.log

The reasons It all starts here. ssri package realizes that integrity checksum stored in package-lock.json and the one coming from https://registry.npmjs.org don't match. That leads to make-fetch-happen package. The error event is handled in turn by oldBody's event handler, by newBody's one, and by pipe's one.

Now, newBody is a stream, and as such an event emitter. And the thing about event emitters, is that emit() throws an exception if there are no listeners. And from my experience odds are in favour of the fact that when pipe's event handler gets to be executed, newBody has no event listeners. Although sometimes it has one. It's just that sometimes the event handler gets assigned after pipe emits the error event to newBody.

Here are both cases interspersed with debugging output. Fisrt the uncaught exception case:

$ rm -rf node_modules ~/.npm && npm i
-- remoteFetch: https://registry.npmjs.org/shipit-shared/-/shipit-shared-4.4.2.tgzaging-bd3355550c5a
-- make-fetch-happen: newBody: new event listener:
[Arguments] {
  '0': 'error',
  '1': [Function: bound onceWrapper] { listener: [Function (anonymous)] }
}
-- integrityStream: https://registry.npmjs.org/shipit-shared/-/shipit-shared-4.4.2.tgz
   sri: sha1-MRp5kqeH932y5IwJEZvc4EOsjfI=
   newSri: sha512-ze/W5oDJlq/tvycego5u1utRhZPOASkcsNr/y2vdus6RbDa5CimMXT13wU8GDLKYy4wejPbwnnDoKnFMN4LVnA== sha1-ogtcyYWb1myofyutiDlrx5j4KpI=
   match: false
-- make-fetch-happen: oldBody.once('error', ...): before newBody.emit(err): newBody's event listeners:
[Function: bound onceWrapper] { listener: [Function (anonymous)] }
-- make-fetch-happen: newBody.once('error', ...): before oldBody.emit(err): newBody's event listeners:
undefined
-- make-fetch-happen: pipe: before newBody.emit('error', ...): newBody's event listeners:
undefined
npm ERR! code EINTEGRITY
npm ERR! sha1-MRp5kqeH932y5IwJEZvc4EOsjfI= integrity checksum failed when using sha1: wanted sha1-MRp
5kqeH932y5IwJEZvc4EOsjfI= but got sha512-ze/W5oDJlq/tvycego5u1utRhZPOASkcsNr/y2vdus6RbDa5CimMXT13wU8G
DLKYy4wejPbwnnDoKnFMN4LVnA== sha1-ogtcyYWb1myofyutiDlrx5j4KpI=. (5988 bytes)
-- tarball: res.body.on(...)
-- make-fetch-happen: newBody: new event listener:
[Arguments] { '0': 'error', '1': [Function (anonymous)] }

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-05-28T22_55_26_667Z-debug.log

Then, the expected flow:

$ rm -rf node_modules ~/.npm && npm i
-- remoteFetch: https://registry.npmjs.org/shipit-shared/-/shipit-shared-4.4.2.tgzaging-bd3355550c5a
-- make-fetch-happen: newBody: new event listener:
[Arguments] {
  '0': 'error',
  '1': [Function: bound onceWrapper] { listener: [Function (anonymous)] }
}
-- tarball: res.body.on(...)
-- make-fetch-happen: newBody: new event listener:
[Arguments] { '0': 'error', '1': [Function (anonymous)] }
-- integrityStream: https://registry.npmjs.org/shipit-shared/-/shipit-shared-4.4.2.tgzg-bd3355550c5a
   match: false
   sri: sha1-MRp5kqeH932y5IwJEZvc4EOsjfI=
   newSri: sha512-ze/W5oDJlq/tvycego5u1utRhZPOASkcsNr/y2vdus6RbDa5CimMXT13wU8GDLKYy4wejPbwnnDoKnFMN4LVnA== sha1-ogtcyYWb1myofyutiDlrx5j4KpI=
-- make-fetch-happen: oldBody.once('error', ...): before newBody.emit(err): newBody's event listeners:
[
  [Function: bound onceWrapper] { listener: [Function (anonymous)] },
  [Function (anonymous)]
]
-- make-fetch-happen: newBody.once('error', ...): before oldBody.emit(err): newBody's event listeners:
[Function (anonymous)]
-- make-fetch-happen: pipe: before newBody.emit('error', ...): newBody's event listeners:
err => stream.emit('error', err)
npm WARN tarball tarball data for shipit-shared@4.4.2 (sha1-MRp5kqeH932y5IwJEZvc4EOsjfI=) seems to be
 corrupted. Trying one more time.
-- remoteFetch: https://registry.npmjs.org/shipit-shared/-/shipit-shared-4.4.2.tgzpp/node_modules/.s
-- make-fetch-happen: newBody: new event listener:
[Arguments] {
  '0': 'error',
  '1': [Function: bound onceWrapper] { listener: [Function (anonymous)] }
}
-- tarball: res.body.on(...)
-- make-fetch-happen: newBody: new event listener:
[Arguments] { '0': 'error', '1': [Function (anonymous)] }
-- integrityStream: https://registry.npmjs.org/shipit-shared/-/shipit-shared-4.4.2.tgz
   match: false
   sri: sha1-MRp5kqeH932y5IwJEZvc4EOsjfI=
   newSri: sha512-ze/W5oDJlq/tvycego5u1utRhZPOASkcsNr/y2vdus6RbDa5CimMXT13wU8GDLKYy4wejPbwnnDoKnFMN4LVnA== sha1-ogtcyYWb1myofyutiDlrx5j4KpI=
-- make-fetch-happen: oldBody.once('error', ...): before newBody.emit(err): newBody's event listeners:
[
  [Function: bound onceWrapper] { listener: [Function (anonymous)] },
  [Function (anonymous)]
]
-- make-fetch-happen: newBody.once('error', ...): before oldBody.emit(err): newBody's event listeners:
[Function (anonymous)]
-- make-fetch-happen: pipe: before newBody.emit('error', ...): newBody's event listeners:
err => stream.emit('error', err)
npm WARN tarball tarball data for shipit-shared@4.4.2 (sha1-MRp5kqeH932y5IwJEZvc4EOsjfI=) seems to be
 corrupted. Trying one more time.
npm WARN app No description
npm WARN app No repository field.
npm WARN app No license field.

npm ERR! code EINTEGRITY
npm ERR! Verification failed while extracting shipit-shared@4.4.2:
npm ERR! Verification failed while extracting shipit-shared@4.4.2:
npm ERR! sha1-MRp5kqeH932y5IwJEZvc4EOsjfI= integrity checksum failed when using sha1: wanted sha1-MRp
5kqeH932y5IwJEZvc4EOsjfI= but got sha512-ze/W5oDJlq/tvycego5u1utRhZPOASkcsNr/y2vdus6RbDa5CimMXT13wU8G
DLKYy4wejPbwnnDoKnFMN4LVnA== sha1-ogtcyYWb1myofyutiDlrx5j4KpI=. (5988 bytes)

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-05-28T22_59_52_998Z-debug.log

Conclusion So, not letting exceptions being uncaught is pretty much desirable. Although, knowing all this I now understand that I could find the culprit by its integrity checksum in package-lock.json, and replace it with the other one from the error message. But back then it was totally unclear to me what's going on.

[vince1995]

So, we've tried also a lot of things. The problem was not the npm client but the npm servers. We've switched to GitHub packaged and still using the npm client. Now there are no problems anymore.

[vince1995]

Oh, and I think it's not a good idea to change the checksum in the package-lock.json file.

[x-yuri]

Surely, but it's not clear what exactly happened.

[darcyclarke]

npm v6 is no longer in active development; We will continue to push security releases to v6 at our team's discretion as-per our Support Policy.

If your bug is preproducible on v7, please re-file this issue using our new issue template.

If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo

Closing: This is an automated message.