NPM pulls corrupted packages from Verdaccio instance

[ckkoeber]

Describe the bug
When I perform a basic npm install in an application I am attempting to set up a dev. environment for corrupted packages end up being pulled from my Verdaccio proxy instance.

To Reproduce
Steps to reproduce the behavior:

1. Set up package.json for my project including adding my dependencies, etc.
2. Set my local system to use my Verdaccio instance: npm set registry [[Verdaccio Server URL]]:4873
3. Perform npm install within the directory I have the package.json in.

Results

I get a ton of output similar to the following:

npm http fetch GET 200 [[Proxy NPM Site]]/@angular%2fplatform-browser-dynamic/-/platform-browser-dynamic-5.2.11.tgz 6430ms
	npm WARN tarball tarball data for @angular/router@5.2.11 (sha512-NT8xYl7Vr3qPygisek3PlXqNROEjg48GXOEsDEc7c8lDBo3EB9Tf328fWJD0GbLtXZNhmmNNxwIe+qqPFFhFAA==) seems to be corrupted. Trying one more time.
	npm WARN tarball tarball data for jquery@3.3.1 (sha512-Ubldcmxp5np52/ENotGxlLe6aGMvmF4R8S6tZjsP6Knsaxd/xp3Zrh50cG93lR6nPXyUFwzN3ZSOQI0wRJNdGg==) seems to be corrupted. Trying one more time.
	npm WARN tarball tarball data for ng-bootstrap@1.6.3 (sha1-1B/UIVTAWTQiy4PEc6OCiqdSW/U=) seems to be corrupted. Trying one more time.

Note the URL encoded name of the package, in this case @angular%2fplatform-browser-dynamic. I do not get this when I set my proxy to be https://registry.npmjs.org/

Expected behavior
I am able to perform an npm install just as if I were connected to the official registry.

Screenshots
N/A

Configuration and Log Files
verdaccio-log.txt
npm-verbose-log.txt
config.yaml.txt

[juanpicado]

You would need to provide more info. npm version. Are you behind a proxy? Environment conditions. I've never heard of this problem before.

[ckkoeber]

Hi, thank you for the quick feedback:

NPM Version: Latest - 6.1.0 but it happens with older versions.
Node Version that Verdaccio is running on: 10.4.0
Node Version that Client is running on: 10.4.0

Environment: Windows Server 2012 (SP2)

The server that Verdaccio is not behind a proxy.

[lgaitan]

Hello @ckkoeber,

I don't think verdaccio stored corrupted packages on its file system,
i've already faced a similar problem and maybe it is the same for you.

In my case it happened because npm was trying to check the package integrity with a wrong hash.
Try deleting your npm-shrinkwrap.json and see if it solves

[ckkoeber]

@lgaitan Thank you for the helpful hint. Forgive me if this is common knowledge: where would I find the npm-shrinkwrap.json file? I assume this is client-side.

[lgaitan]

Yes, it should be located on the project folder that you are trying to install.

it can be either npm-shrinkwrap.json or package-lock.json

[ckkoeber]

OK, so after performing the following everything worked:

rmdir /S /Q node_modules
del package-lock.json
npm set registry [[Verdaccio Instance's IP]]:4873
npm cache clean --force
npm install --force --verbose --no-bin-links

Thank you everyone for the feedback.

[ckkoeber]

Adding *nix variant:

rm -rf node_modules
rm package-lock.json
npm set registry [[Verdaccio Instance's IP]]:4873
npm cache clean --force
npm install --force --verbose --no-bin-links

[checksummaster]

definitively a package-lock.json is involved in this problem.

"rm package-lock.json" should never be done unless you don't care what you will distribute.
package-lock.json make that all the dependency of all your project dependency will never change. It the solution to prevent a crash in case someone bumb a sub sub sub dependency with bad code.
I can talk more longer why we should not do that but ... it not the place here.

When you will make "npm install" a new package-lock.json will be create and the next "npm cache clear -force & npm install --force --verbose --no-bin-links" will give you the error "seems to be corrupted" again.

This bug seem to be only when you use npm (recent version) on windows. verdaccio can run on any type of machine (I try window and mac).
With old version of npm (5.6.0) it give something like
npm WARN tar zlib error: unexpected end of file
npm ERR! cb() never called!

It seem that we don't receive the complete tar file.

If we do "npm install" without clearing the npm cache many time, it work (it mean sometime the tar file is well transfered, sometime not). Also I see that the error is more often when we transfer big tar file, (in may case over 100meg)

note also that I see this error with sinopia 1.4 then it maybe a old bug (or a new one in npm).
note also that yarn don't have this problem (but it dont use package-lock.json anyway)

I have a felling that if the network is slow, we don't have this error ... but for now it just a felling.
(I have the bug more often when I run everything on same PC/virtual PC).

Hope it help

[luguidong]

after performing the following everything still not worked:

rmdir /S /Q node_modules
del package-lock.json
npm set registry [[Verdaccio Instance's IP]]:4873
npm cache clean --force
npm install --force --verbose --no-bin-links

npm version 6.7.0
verdaccio version 3.2.0
node version 10.15.0

[2imagine]

Was able to solve the problem by turning off Verdaccio's 'cache'

uplinks: npmjs: url: https://registry.npmjs.org/ cache: false

Correction: changed the npmrc setting in the project itself to
@company:registry=http:// registry=https://registry.npmjs.org

[juanpicado]

If you turn of the cache, tarballs are not persisted affecting the registry performance. It is ok to do it but be aware of it.

[fuzzybair]

I tried @ckkoeber solution and @2imagine solution both options did not resolve the corrupted package issue for me.

npm install
npm WARN tarball tarball data for cesium@1.57.0 (sha512-EPh38EQdkd9nngbSSB3Yqacqy4MxdnDHLVTP208M+bwQ7Apee5BwDgP4bMp/tKfw2b5Jk1W9w6s9Top5hbo2og==) seems to be corrupted. Trying one more time.
npm WARN tarball tarball data for plotly.js@1.48.1 (sha512-1dar+duEcH0+kVbiwf2xw0oCkm5UNPg5h7Qs9qa5Ak9wRnRq41yGYpjz8myzuLsQ2tEISc8FXiQdCG5wIt2UFQ==) seems to be corrupted. Trying one more time.
npm WARN tarball tarball data for typescript@3.2.4 (sha512-0RNDbSdEokBeEAkgNbxJ+BLwSManFy9TeXz8uW+48j/xhEXv1ePME60olyzw2XzUqUBNAYFeJadIqAgNqIACwg==) seems to be corrupted. Trying one more time.

PS > node --version
v10.16.0
PS > npm -v
6.9.0

using docker version verdaccio/verdaccio@latest
http address - http://0.0.0.0:4873/ - verdaccio/4.0.1

[lock]

🤖This thread has been automatically locked 🔒 since there has not been any recent activity after it was closed.
We lock tickets after 90 days with the idea to encourage you to open a ticket with new fresh data and to provide you better feedback 🤝and better visibility 👀.
If you consider, you can attach this ticket 📨 to the new one as a reference for better context.
Thanks for being a part of the Verdaccio community! 💘