[BUG] Inconsistent SHA-1 and SHA-512 integrity values in registry response

[hubgit]

When running npm install we've observed that sometimes the integrity value for a dependency in package-lock.json (and the _integrity value in the dependency's package.json in node_modules) can change for seemingly no reason: sometimes it starts with sha1- and sometimes it starts with sha512-.

By trying npm install several times while removing node_modules and running npm cache clean --force between each run (and possibly something else), I've managed to capture two requests in ~/.npm/_cacache which exhibit this behaviour:

SHA-512:

db/c0/37002f9ad0a4679e0f0266ec01808bed69f7750a20466b5f0e5c5d4f484a:d247acba954e5cb9677b6aaff7ddc3390c216575

{
    "key": "make-fetch-happen:request-cache:https://registry.npmjs.org/babel-code-frame/-/babel-code-frame-6.26.0.tgz",
    "integrity": "sha512-XqYMR2dfdGMW+hd0IUZ2PwK+fGeFkOxZJ0wY+JaQAHzt1Zx8LcvpiZD2NiGkEG8qx0CfkAOr5xt76d1e8vG90g==",
    "time": 1608640925191,
    "size": 2928,
    "metadata": {
        "url": "https://registry.npmjs.org/babel-code-frame/-/babel-code-frame-6.26.0.tgz",
        "reqHeaders": {
            "connection": [
                "keep-alive"
            ],
            "user-agent": [
                "npm/6.14.9 node/v14.15.3 darwin x64"
            ],
            "npm-in-ci": [
                "false"
            ],
            "npm-scope": [
                ""
            ],
            "npm-session": [
                "e22b38ffdf132baa"
            ],
            "referer": [
                "install"
            ],
            "pacote-req-type": [
                "tarball"
            ],
            "pacote-pkg-id": [
                "registry:babel-code-frame@https://registry.npmjs.org/babel-code-frame/-/babel-code-frame-6.26.0.tgz"
            ]
        },
        "resHeaders": {
            "date": [
                "Tue, 22 Dec 2020 12:42:05 GMT"
            ],
            "content-type": [
                "application/octet-stream"
            ],
            "content-length": [
                "2928"
            ],
            "connection": [
                "keep-alive"
            ],
            "set-cookie": [
                "__cfduid=dba1ba0d5679d97e7751862658a8b2bdc1608640925; expires=Thu, 21-Jan-21 12:42:05 GMT; path=/; domain=.npmjs.org; HttpOnly; SameSite=Lax"
            ],
            "cf-ray": [
                "6059ebb65c772d07-LHR"
            ],
            "accept-ranges": [
                "bytes"
            ],
            "age": [
                "3756173"
            ],
            "cache-control": [
                "public, immutable, max-age=31557600"
            ],
            "etag": [
                "\"fdca204ce9b0158bcc65745baa896e4c\""
            ],
            "last-modified": [
                "Sat, 26 May 2018 17:54:46 GMT"
            ],
            "vary": [
                "Accept-Encoding"
            ],
            "cf-cache-status": [
                "HIT"
            ],
            "cf-request-id": [
                "072c11a5f800002d075825f000000001"
            ],
            "expect-ct": [
                "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\""
            ],
            "server": [
                "cloudflare"
            ],
            "x-fetch-attempts": [
                "1"
            ]
        }
    }
}

SHA-1:

db/c0/37002f9ad0a4679e0f0266ec01808bed69f7750a20466b5f0e5c5d4f484a:f84ae26bb03fa2c70ff409926ad12c559b9faa28

{
    "key": "make-fetch-happen:request-cache:https://registry.npmjs.org/babel-code-frame/-/babel-code-frame-6.26.0.tgz",
    "integrity": "sha1-Y/1D99weO7fONZR9uP42mj9Yx0s=",
    "time": 1608641172290,
    "size": 2928,
    "metadata": {
        "url": "https://registry.npmjs.org/babel-code-frame/-/babel-code-frame-6.26.0.tgz",
        "reqHeaders": {
            "connection": [
                "keep-alive"
            ],
            "user-agent": [
                "npm/6.14.9 node/v14.15.3 darwin x64"
            ],
            "npm-in-ci": [
                "false"
            ],
            "npm-scope": [
                ""
            ],
            "npm-session": [
                "bb3ee71178300e0b"
            ],
            "referer": [
                "install"
            ],
            "pacote-req-type": [
                "tarball"
            ],
            "pacote-pkg-id": [
                "registry:babel-code-frame@https://registry.npmjs.org/babel-code-frame/-/babel-code-frame-6.26.0.tgz"
            ]
        },
        "resHeaders": {
            "date": [
                "Tue, 22 Dec 2020 12:46:12 GMT"
            ],
            "content-type": [
                "application/octet-stream"
            ],
            "content-length": [
                "2928"
            ],
            "connection": [
                "keep-alive"
            ],
            "set-cookie": [
                "__cfduid=dbf197e7846a9f430cd318935e4dc78031608641172; expires=Thu, 21-Jan-21 12:46:12 GMT; path=/; domain=.npmjs.org; HttpOnly; SameSite=Lax"
            ],
            "cf-ray": [
                "6059f1bdeb45ce23-LHR"
            ],
            "accept-ranges": [
                "bytes"
            ],
            "age": [
                "3756420"
            ],
            "cache-control": [
                "public, immutable, max-age=31557600"
            ],
            "etag": [
                "\"fdca204ce9b0158bcc65745baa896e4c\""
            ],
            "last-modified": [
                "Sat, 26 May 2018 17:54:46 GMT"
            ],
            "vary": [
                "Accept-Encoding"
            ],
            "cf-cache-status": [
                "HIT"
            ],
            "cf-request-id": [
                "072c156aad0000ce2324b40000000001"
            ],
            "expect-ct": [
                "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\""
            ],
            "server": [
                "cloudflare"
            ],
            "x-fetch-attempts": [
                "1"
            ]
        }
    }
}

The diff between these two responses is minimal, so I'm not sure why the registry might be returning a different integrity value each time:

❯ diff sha1.json sha512.json 
1c1
< db/c0/37002f9ad0a4679e0f0266ec01808bed69f7750a20466b5f0e5c5d4f484a:f84ae26bb03fa2c70ff409926ad12c559b9faa28	{
---
> db/c0/37002f9ad0a4679e0f0266ec01808bed69f7750a20466b5f0e5c5d4f484a:d247acba954e5cb9677b6aaff7ddc3390c216575	{
3,4c3,4
<     "integrity": "sha1-Y/1D99weO7fONZR9uP42mj9Yx0s=",
<     "time": 1608641172290,
---
>     "integrity": "sha512-XqYMR2dfdGMW+hd0IUZ2PwK+fGeFkOxZJ0wY+JaQAHzt1Zx8LcvpiZD2NiGkEG8qx0CfkAOr5xt76d1e8vG90g==",
>     "time": 1608640925191,
22c22
<                 "bb3ee71178300e0b"
---
>                 "e22b38ffdf132baa"
36c36
<                 "Tue, 22 Dec 2020 12:46:12 GMT"
---
>                 "Tue, 22 Dec 2020 12:42:05 GMT"
48c48
<                 "__cfduid=dbf197e7846a9f430cd318935e4dc78031608641172; expires=Thu, 21-Jan-21 12:46:12 GMT; path=/; domain=.npmjs.org; HttpOnly; SameSite=Lax"
---
>                 "__cfduid=dba1ba0d5679d97e7751862658a8b2bdc1608640925; expires=Thu, 21-Jan-21 12:42:05 GMT; path=/; domain=.npmjs.org; HttpOnly; SameSite=Lax"
51c51
<                 "6059f1bdeb45ce23-LHR"
---
>                 "6059ebb65c772d07-LHR"
57c57
<                 "3756420"
---
>                 "3756173"
75c75
<                 "072c156aad0000ce2324b40000000001"
---
>                 "072c11a5f800002d075825f000000001"

[hubgit]

It seems that perhaps this integrity value is generated locally by the npm cli/cacache/pacote/ssri, but the package-lock.json docs say the following, and I'm assuming this dependency is from a "registry source".

This is a Standard Subresource Integrity for this resource.

• For bundled dependencies this is not included, regardless of source.
• For registry sources, this is the integrity that the registry provided, or if one wasn't provided the SHA1 in shasum.
• For git sources this is the specific commit hash we cloned from.
• For remote tarball sources this is an integrity based on a SHA512 of the file.
• For local tarball sources: This is an integrity field based on the SHA512 of the file.

[hubgit]

@isaacs Is it possible that it could be caused by the issue fixed in this commit, which was first included in pacote v10.2.0, which was brought into cli in this commit, which is only included in npm v7, not in npm v6?

[hubgit]

My guess, also, is that this is the downstream fix, which stops the integrity value set by pacote from being passed on to the options for the next package:

2c305e8#diff-eabad0b45ec780b8b57ae6e3d47fd7d063680bea3acdc6b98a4b45ba3b808cccR24

[darcyclarke]

npm v6 is no longer in active development; We will continue to push security releases to v6 at our team's discretion as-per our Support Policy.

If your bug is preproducible on v7, please re-file this issue using our new issue template.

If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo

[ryanpcmcquen]

This is a security issue given this CVE against SHA1:
https://sha-mbles.github.io/

[cammytang]

@hubgit Hi, did you find the root reason finally about why the registry returns a different integrity value some times? My team encountered same issue.