Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Local cache can become corrupted #4652

Closed
seldo opened this Issue Feb 11, 2014 · 3 comments

Comments

Projects
None yet
5 participants
Contributor

seldo commented Feb 11, 2014

Inspired by https://twitter.com/fivetanley/status/433305038165786624 and per conversation with @isaacs, it's possible for local registry cache to become corrupted if a package download fails or the client crashes for some other reason.

A simple fix is to add hash-checking to the client's local-fetch step so that it verifies the local package and re-fetches if it fails.

In the meantime, the workaround is to use cache clean: https://www.npmjs.org/doc/cli/npm-cache.html

Member

domenic commented Feb 11, 2014

The cache in general is due for an overhaul. #4159. Perhaps this can be a simple patch to the existing cache.js monstrosity though.

In either case, it should be pretty easy to write a test for, e.g. with an addition to npm-registry-mocks that includes a bad tarball.

A simple fix is to add hash-checking to the client's local-fetch step so that it verifies the local package and re-fetches if it fails.

To clarify, you'd be checking the hash from the presumed-non-corrupt package.json downloaded from the registry, against the possibly-corrupt tarball?

Contributor

glasser commented Mar 25, 2014

See also #3265.

@othiym23 othiym23 added this to the cache rewrite milestone Sep 26, 2014

@othiym23 othiym23 added the bug label Sep 26, 2014

@othiym23 othiym23 self-assigned this Sep 26, 2014

@zkat zkat added a commit that referenced this issue Apr 17, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
2fecc4a

@zkat zkat added a commit that referenced this issue Apr 18, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
6394384

@zkat zkat added a commit that referenced this issue Apr 18, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
576d441

@zkat zkat added a commit that referenced this issue Apr 18, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
e73f895

@zkat zkat added a commit that referenced this issue Apr 18, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
2d69994

@zkat zkat added a commit that referenced this issue Apr 19, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
e61a3fd

@zkat zkat added a commit that referenced this issue Apr 19, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
71396ca

@zkat zkat added a commit that referenced this issue Apr 20, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
2042018

@zkat zkat added a commit that referenced this issue Apr 20, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
9ce43d8

@zkat zkat added a commit that referenced this issue Apr 22, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
aa7d46e

@zkat zkat added a commit that referenced this issue Apr 22, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
412f1f5

@zkat zkat added a commit that referenced this issue Apr 22, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
86e22ee

@zkat zkat added a commit that referenced this issue Apr 22, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
78bf204

@zkat zkat added a commit that referenced this issue Apr 23, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.
d02e229

@zkat zkat added a commit that referenced this issue Apr 26, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.

fix(doctor): updated doctor command and its tests
8cf4c66

@zkat zkat added a commit that referenced this issue Apr 27, 2017

@zkat zkat feat(cache): rewrite package fetching and caching on top of pacote
Fixes: #2568
Fixes: #2649
Fixes: #3141
Fixes: #4042
Fixes: #4652
Fixes: #5357
Fixes: #5509
Fixes: #5622
Fixes: #5941

All fetching-related networking is now done through pacote, and
the old cache has been entirely replaced by a cacache-based one.

Features:

* npm now supports a variety of hash algorithms for tarball storage. On registries that support it, npm is able to use sha512sum for verification.

* An `integrity` field has been added to `npm-shrinkwrap.json`.

* Package integrity will be fully verified on both cache insert and extraction -- if npm installs something, it's going to be exactly what you downloaded, byte-for-byte, or it will fail.

* If `npm-shrinkwrap.json` is used, npm will bypass checking package manifests and go straight to the tarball, fetching it by content address if locally cached.

* Checksum integrity failures will now retry downloading on error, instead of failing on a single check.

* A new npm command, `npm cache verify`, can now be used to verify and garbage collect your local cache.

* npm now supports arbitrarily large tarball downloads: tarballs will no longer be loaded entirely into memory before extraction.

* packages whose names only differ in casing, and packages from different sources/registries/etc will now correctly be cached separately from each other.

* Some performance improvements.

* Improved fetch retry logic will try harder to download your packages.

BREAKING CHANGE: many shrinkwrap and cache-related things have changed.

* Previously-created caches will no longer be used. They will be left in place, but data will need to be re-cached. There is no facility for rebuilding a cache based on an existing one.

* `npm cache ls` has been removed for now

* `npm cache rm` now always removes the entire cache. There is no granular removal available for now.

* git dependencies can now use semver resolution using `#semver:^1.2.3`

* `--cache-min` and `--cache-max` have been deprecated. Use `--offline`, `--prefer-offline`, and `--prefer-online instead. `--cache-min=9999+` and `--cache-max=0` have been aliased to `--prefer-offline` and `--prefer-online`, respectively.

* npm will now obey HTTP caching headers sent from registries and other remote HTTP hosts, and will use standard HTTP caching rules for its local cache.

* `prepublishOnly` now runs *before* packing the tarball.

* npm no longer supports node@<4.

fix(doctor): updated doctor command and its tests
c63d8cd
Owner

iarna commented Jul 6, 2017

npm@5 directly addresses this issue—it should no longer be possible for the local cache to be corrupted. If anyone encounters an issue where that does happen now, please open a new issue.!

@iarna iarna closed this Jul 6, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment