[BUG] Organization packages installed from the registry are failing to execute the lifecycle scripts

[un-focused]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

Scoped packages (own organization) installed from a registry (GitLab) do not execute the postinstall scripts or any lifecycle scripts. I have tested this behaviour in npm 7 & npm 8 & am aware that npm 8 suppress the output from scripts so I used the following command to test my theory:

npm i --foreground-scripts --loglevel verbose

Other packages execute their postinstall but my package does not run. If I use a local path to the tar file, it does run, but setting the version does not.

If I download the tar manually from the registry & set the path to it, it will run the scripts!

Expected Behavior

Scoped packages installed from a registry exhibit the same behaviour as if I were using a local path to tar file. They should execute the lifecycle scripts.

Steps To Reproduce

None of my scripts are working after publishing an npm package. I will try to give a simple example here to demonstrate the point.

The package.json scripts object (relevant parts) looks like this:

"scripts": {
  "postinstall": "node scripts/postinstall",
}

The scripts/postinstall looks like this:

function main() {
    console.log('hello world')
    process.exit(0);
}

main();

This works when I execute:

$ npm I

inside the package that I am publishing.

When I install it locally it works too:

$ npm i
$ npm run build
$ npm pack --pack-destination

After the pack, in another project, I specify the path to the tar file generated & everything works!

If I run:

$ npm publish

& then install it inside the same project that consumes the node_module, it does not execute the install or postinstall script.

To debug this, I run the command:

npm i --foreground-scripts

and have tried npm i --foreground-scripts --loglevel verbose as well to better understand the problem.

The documentation for the npm lifecycle is lacking & I could not find any information online. Please note that this package is publish in a private gitlab registry and I am able to download & "install" the package as I see it in my node_modules but it does run the lifecycle scripts when installing remotely from npm.

I do have an .npmrc file specified.

Environment

• npm: v8.8.0
• Node.js: v18.1.0
• OS Name: Mac OS Monterey
• System Model Name: Macbook Pro
• npm config:

; "user" config from /Users/unfocused/.npmrc

ignore-script = false 
unsafe-perm = true 

; "project" config from /Users/unfocused/Documents/Git Folders/Projects/example/.npmrc

@org:registry = "https://gitlab.com/api/v4/packages/npm/" 
//gitlab.com/api/v4/packages/npm/:_authToken = (protected) 

; node bin location = /Users/unfocused/.nvm/versions/node/v18.1.0/bin/node
; node version = v18.1.0
; npm local prefix = /Users/unfocused/Documents/Git Folders/Projects/example
; npm version = 8.18.0
; cwd = /Users/unfocused/Documents/Git Folders/Projects/example
; HOME = /Users/unfocused

[un-focused]

This works on npm 6.14.2. It looks like this issue started in npm 7 & persists in npm 8.

[un-focused]

To test this idea, I installed https://www.npmjs.com/package/npm-lifecycle-hooks! & it ran the lifecycle methods! Then I copied the package contents to a new folder & published it as a scoped package in the GitLab NPM Registry & then it did not run any lifecycle methods!

[Janrupf]

Can confirm this behavior too - lifecycle scripts are not running when installing from a custom registry. If a binding.gyp exists node-gyp is always invoked, even if an install script is present.

[pdt-ayidi]

Found the issue with our registry (JFrog) is not sending scripts or hasInstallScript when fetching the package manifest. Without one of these fields npm v7 or higher will generate an inaccurate package-lock. Npm v7 and higher does not inspect node_modules and the package-lock is missing hasInstallScript, causing npm install to skip the life cycles.

This is the relevant code:

cli/workspaces/arborist/lib/node.js

Line 317 in b397774

return !!(hasInstallScript || install || preinstall || postinstall)

Here's the documentation for npm's registry api: https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md

Check with your registry provider if there are discrepancies.

@un-focused what is the response GitLab is sending you for the npm-lifecycle-hooks package manifest? You should be able to get the http request by using NODE_DEBUG=http and --loglevel verbose. Will look something like this:

curl -v \
-H "user-agent: npm/8.15.0 node/v16.17.0 darwin x64 workspaces/false" \
-H "pacote-version: 13.6.1" \
-H "pacote-req-type: packument" \
-H "pacote-pkg-id: registry:npm-lifecycle-hooks" \
-H "accept: application/vnd.npm.install-v1+json; q=1.0, application/json; q=0.8, */*" \
-H "npm-auth-type: legacy" \
-H "npm-command: install" \
-H "authorization: Basic <base64 username:password>" \
-H "Accept-Encoding: gzip,deflate" \
"https://<registry url>/npm-lifecycle-hooks"

[techmunk]

Just weighing in here:
gitlab does not currently support hasInstallScript or _hasShrinkwrap fields. There was a patch written some time ago that seems to have stalled that aimed to address this. https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83807.

The patch has stalled because in order for gitlab to support _hasShrinkwrap, gitlab needs to unpack and inspect the package to add the metadata that gets stored with the package and the team has concerns over the workload this might cause on their infrastructure.

This could be greatly simplified if npm publish included this metadata as part of the PUT request to the registry when publishing the package,, then registries would not need to do any inspection of the packages or have possibly flawed or incorrect logic for determining fields such as hasInstallScript or _hasShrinkwrap.

Is there a reason the publish PUT request does not include all the metadata required by npm install?

Assuming such a change to NPM, Gitlab would still need to be patched for PACKAGE_JSON_ALLOWED_FIELDS to add hasInstallScript, but its less likely to block/stall, and more likely to get in to a release in a reasonable time-frame.