[BUG] Preinstall script runs after installing dependencies

[t1m0thyj]

Current Behavior:

In NPM v7 the preinstall script runs after dependencies are installed, which breaks backwards compatibility with NPM v6.

This may have been on purpose, but I do not see the behavior change listed under breaking changes.
https://blog.npmjs.org/post/626173315965468672/npm-v7-series-beta-release-and-semver-major

Expected Behavior:

The preinstall script should run before dependencies are installed.

• I'm aware that a similar issue ([BUG] npm 7.x executes package.json preinstall script after installing dependencies  #2253) was closed with an explanation of "works as intended". I'm requesting for this behavior of NPM v7 to please be reconsidered.
• This same issue appeared in npm@3 and was treated as a bug and fixed: preinstall execution order in npm@3.x npm#10379
• The ability to hook into NPM before dependencies are installed may not be best practice but is sometimes required. This is the primary reason why I use preinstall scripts.
• If preinstall scripts run after dependencies are installed, they seem identical to postinstall scripts and there is no longer a reason for me to use them.

Steps To Reproduce:

Clone this minimal sample repo which contains 2 packages:

• package1 - This package is designed to fail if installed standalone. It expects the file "temp.txt" to exist that has been generated by a parent package before install.
• package2 - This package installs package1 as a dependency, and has a preinstall script that creates the file "temp.txt".

Run cd package2 && npm install.

With NPM v6, the install succeeds because the preinstall script creates "temp.txt" before package1 is installed.
With NPM v7, the install fails because the preinstall script creates "temp.txt" after package1 is installed.

Environment:

• Ubuntu 18.04
• Node: 15.8.0
• npm: 7.5.3

[darcyclarke]

@t1m0thyj thanks for filing this. We have a consolidated issue tracking work/investigating this: npm/statusboard#267

@wraithgar is looking into it.

[t1m0thyj]

@darcyclarke @wraithgar Thanks for investigating this and updating the documentation. Can this issue please be reopened? I see the linked issue was closed, but the preinstall script is still implemented improperly in npm@7.6.1.

[nlf]

today in npm 7 the preinstall script is working as intended for the majority of use cases. what you're asking for isn't unreasonable, but would create a new class of problem wherein a preinstall script cannot leverage any dependencies. resolving for that problem is why preinstall behaves the way it does.

i would suggest opening an rfc where we can start a discussion about potentially adding a new lifecycle script that runs before anything else. it would have to be a net new hook, though, maybe preunpack or something similar.

[ljharb]

By its name, "preinstall" should not have any dependencies available. imo that it runs after they're installed is just as much of a naming blunder as "prepublish" running on "not just before publish".

[nlf]

you're not wrong, to be honest a lot of our lifecycle scripts are poorly named and don't run exactly when you expect. that's why there's both a prepublish and a prepublishOnly. we have a lot of technical debt here, but changing the expectations about how lifecycle scripts work is a glacier of a breaking change.

the best we can do right now is clearly document when scripts run and what's available to them

[nlf]

a lot of the confusion here is that install is an event as well as a script, and in npm terms today pre and post mean before and after the script not the event. since we run the install script after we've already installed the package, that means that preinstall also runs after the package has been installed, immediately before the install script.

i agree it's not very intuitive, and we'd love to fix it so that the pre and post refer to the actual events and not the accompanying scripts, but it's a long road to get there.

adding these more intuitive hooks could potentially be done as a net new feature though, using a separate top level field in the package.json like "events", that would allow us to keep the current behavior of "scripts" while providing a better implementation elsewhere. eventually we deprecate the usage of "scripts" for anything other than npm run-script and "events" becomes the canonical place you tell us what to run during the various processes. that combined with a clear chart of the lifecycle that details what takes place before and after each event and where event scripts will be run would be a huge win

[jfrancos]

I'm also noticing that for npm v6, I know how to run a script prior to installation of dependencies, and for npm v7, I don't. I'm not too attached to the specific methodology of using a preinstall hook, as long as I have some way of running a script before dependencies are installed. Can someone please let me know how to accomplish this for npm v7?

I've already looked here but couldn't find anything that seemed to help.

[davidgoitia]

I also think this should be treated as a bug as previous versions of npm and all versions of yarn support this.

For example we use preinstall script for generating workspaces from gRPC .proto files before workspaces and dependencies get evaluated. This works perfectly with yarn workspaces but doesn't with npm v7

[ryansonshine]

Agreed on treating this as a bug. Another example is users who use the preinstall hook to authenticate with private registries on their cloud platforms and/or any other registries with short lived tokens.

Azure: https://github.com/gsoft-inc/azure-devops-npm-auth
AWS: https://aws.amazon.com/blogs/devops/publishing-private-npm-packages-aws-codeartifact/

[kmturley]

We were also using preinstall lifecycle script to set the registry and ensure we were logged in:

{
  "name": "private-npm",
  "version": "1.0.0",
  "description": "Private npm demo",
  "main": "index.js",
  "scripts": {
    "login": "npm config set registry https://registry.your-registry.npme.io/ && npm login --registry=https://registry.company-name.npme.io",
    "preinstall": "npm run login",
    "prepare": "npm run login"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {}
}

The change to npm v7 broke this functionality.

[GregVes]

Agreed on treating this as a bug. Another example is users who use the preinstall hook to authenticate with private registries on their cloud platforms and/or any other registries with short lived tokens.

Azure: https://github.com/gsoft-inc/azure-devops-npm-auth
AWS: https://aws.amazon.com/blogs/devops/publishing-private-npm-packages-aws-codeartifact/

Facing the same issue right now with CodeArtifact

[darcyclarke]

Re-opening as per: https://github.com/npm/rfcs/blob/da73d3fd50a40e742f2d3755238ecdb8a6758d7f/meetings/2021-07-21.md#pr-403-rfc-add-preunpack-life-cycle-script---ryansonshine

[wenfangdu]

It's been more than two years now, any progress?

[electrofelix]

It appears that many were using this mechanism as a way of automating login into private registries to reduce developer friction. Which sounds like there is also another underlying issue in that there is no way to register a credential helper tool in user config that can handle perform an auto login when needed for a specific registry similar to how git credentials helpers or docker credsHelper work. Perhaps that would be worth looking at?

[ljharb]

Logging in to registries needs to happen before any installation is attempted, at the app level - there’s no way to avoid that.

[beratbayram]

Another legitimate use case is to force people to use a specific package manager.

  "scripts": {
    "preinstall": "npx -y only-allow pnpm"
  },

[ljharb]

@beratbayram that doesn't seem like something a dependency should be capable of forcing.

[aelders-ncino]

Per the documentation for npm ci: preinstall ... These all run after the actual installation of modules into node_modules, in order, with no internal actions happening in between

This isn't explicitly listed in the npm install, but I assume it may still apply.

Additionally, under Best Practices, it states that:
Don't use install. Use a .gyp file for compilation, and prepare for anything else. You should almost never have to explicitly set a preinstall or install script. If you are doing this, please consider if there is another option. The only valid use of install or preinstall scripts is for compilation which must be done on the target architecture.

What are we supposed to do here? .gyp files seem to only be tied to modifying the node-gyp process. This means that using preinstall is the only way to ensure that a script gets run before someone runs npm install. Having us explicitly run a script before running npm install isn't always possible in all situations.

---

I also attempted a workaround by adding an install script that would be run instead of npm install, but that has very interesting results! Here's an example of my scripts:

{
  "scripts": {
    "preinstall": "echo 'preinstall is running!'",
    "install": "npm install"
  }
}

If I run npm run install, attempting to bypass this issue, the command will infinitely loop!

$ npm run install

> test-npm@1.0.0 preinstall
> echo 'preinstall is running!'

preinstall is running!

> test-npm@1.0.0 install
> npm install

> test-npm@1.0.0 preinstall
> echo 'preinstall is running!'

preinstall is running!

> test-npm@1.0.0 install
> npm install

So when running npm install from a script seems to call preinstall properly. What is happening here?!
Versions: Node v18.15.0, npm v9.6.7

[electrofelix]

Logging in to registries needs to happen before any installation is attempted, at the app level - there’s no way to avoid that.

@ljharb I presume this was directed at my comment?

I'm not suggesting that preinstall be restored to the previous behaviour for login workflows. I'm pointing out that since there is no ability to set a config option at the user level to specify a credential helper to be called by npm when it needs an auth token for a registry, that the preinstall mechanism was misused to facilitate registry services with short lived tokens.

There appears to be an increasing desire from a security perspective to limit exposure by only having tokens that live for a couple of hours, reducing the risk when one leaks.

What I think many would be happy is having some way to make it possible to provide a workflow that can work across multiple private registries. This may mean having the ability to register a helper for each or multiple registries (by pattern) that will handle returning the token needed. The helper may hook into existing enterprise SSO, or 2 Factor auth, but the key is that when a token is needed it triggers creation or reuses a cached one stored securely in a keychain. This also removes the need for tokens to be stored directly in files. Equally important is that npm shouldn't care how the tools works, only that it conforms to specific input/output behaviour. If the tool needs user interaction, approval, etc, it's on the tool to work out how to support, npm need only provide an interface definition that the tool must conform to.

I'll happily log a separate issue for this, but I think it's worth looking at this as one use case that was driving the misuse of preinstall die to the absence of an alternative.

[neuro-sys]

today in npm 7 the preinstall script is working as intended for the majority of use cases.

"As intended" by what line of thinking? I am curious where such things are stated, discussed and confirmed as to be changed?

what you're asking for isn't unreasonable, but would create a new class of problem wherein a preinstall script cannot leverage any dependencies. resolving for that problem is why preinstall behaves the way it does.

Did those "problems" not exist before the change was introduced as late as version 7? Assuming they did, then the course of action to address such "already existing" problems is not by breaking existing behaviour that a large number of existing code is relying on, and change it in a way to contradict the literal meaning of "pre-install" to mean "post-install-and-before-compilation-scripts-execute", but instead, introduce a new solution (e.g. lifecycle phase) that can be leveraged by those who had a problem to begin with. Alternatively, if anything, provide an option to enable the originally intended behaviour (and possibly as a differently named lifecycle phase).

[rubengmurray]

Yeah. This is just weird? Completely breaks the principle of least surprise and even worse - as far as I can tell - there's no alternative that npm provides...

From the comments it seems pnpm functions as expected? I guess I'll go and investigate that 👀

[miketalley]

Bump on making a decision to fix this or implement a preunpack/prepreinstall hook!

I was very surprised to find out this is how the preinstall script, which I believed would execute as named: pre-install, worked.

[voltuer]

so how do i force a user to use pnpm? i need that

[ljharb]

You don’t? people can use any package manager they want.

[prinsss]

so how do i force a user to use pnpm? i need that

Hi @sebfindling ,

To force users to use pnpm, please have a look at this package: just-pnpm

Hope it helps! :)

[SourceCipher]

Another reason why people need preinstall to work before npm install gets executed is so we could output to the console this:

  _    _            _    _ 
 | |  | |          | |  | |
 | |  | |__      __| |  | |
 | |  | |\ \ /\ / /| |  | |
 | |__| | \ V  V / | |__| |
  \____/   \_/\_/   \____/ 
                           

[ljharb]

@SourceCipher preinstallOnly scripts would still have output suppressed, to prevent people polluting installers' terminals with ads and whatnot.

[apoorvkk]

Any potential update on this?

[ajiho]

Any news?