Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] support multiple integrity algorithms #25

Closed
isaacs opened this issue Oct 23, 2019 · 3 comments · Fixed by #200
Closed

[FEATURE] support multiple integrity algorithms #25

isaacs opened this issue Oct 23, 2019 · 3 comments · Fixed by #200
Labels
Enhancement new feature or improvement

Comments

@isaacs
Copy link
Contributor

isaacs commented Oct 23, 2019

What / Why

Right now, if you provide multiple algorithms to content.write(), it'll error out with:

opts.algorithms only supports a single algorithm for now

It's said that for a long time. Let's support multiple algorithms!

This causes some suboptimal caching in make-fetch-happen, because we may have an integrity value that is a sha512, but it always caches as sha1, so we can never have a cache hit.

How

In lib/content/write.js, we always place the content in a single location based on the integrity and algorithm.

  • Create content locations for all the hashes provided.
  • In lib/content/write.js, move the written content into place, and then hard-link to the other locations.
  • Even if one of the entries is deleted, the others will still be available.

Current Behavior

  • Throws an error if multiple algorithms are provided
  • Content can only ever be cached at a single integrity location
@isaacs
Copy link
Contributor Author

isaacs commented Oct 23, 2019

To investigate: the cache index stores a single integrity string, but would have to store multiple. Ie, instead of "integrity":"sha512-deadbeef" it might be "integrity":"sha512-deadbeef sha1-bad1dea". Fall back to secondary locations on ENOENT errors, if multiple possible content addresses are available.

@isaacs
Copy link
Contributor Author

isaacs commented Oct 25, 2019

Might be good to do this along with the flat folders PR, since it's a change to the content/index layout algorithm, so we don't have to bump that twice.

@mjsir911
Copy link

The cache index files seem to support multiple integrity lines per file eg:

4577d1102461b247feed642939cfaba3270dc6e6	{"key":"pacote:tarball:file:/var/tmp/portage/www-plugins/musescore-downloader-0.11.2/distdir/browserify-rsa-4.0.1.tgz","integrity":"sha512-+YpEyaLDDvvdzIxQ+cCx73r5YEhS3ANGOkiHdyWqW4t3gdeoNEYjSiQwntbU4Uo2/9yRkpYX3SRFeH+7jc2Duw==","time":1605029163847,"size":5999}
98128ac4ec588767de288fb67d666ea9c249ac5c	{"key":"pacote:tarball:file:/var/tmp/portage/www-plugins/musescore-downloader-0.11.2/distdir/browserify-rsa-4.0.1.tgz","integrity":"sha1-IeCr+vbyApzy+vsTNWenAdQTVSQ=","time":1604617124075,"size":5999}

which I've been using as a workaround for this issue, and doesn't require changing the integrity string requirements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement new feature or improvement
Projects
None yet
3 participants