Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] npx cannot run bin script of github dependency when “build“ script included in package.json #4003

Closed
2 tasks done
mbtts opened this issue Nov 6, 2021 · 4 comments
Labels
Bug thing that needs fixing Needs Triage needs review for next steps Release 8.x work is associated with a specific npm 8 release

Comments

@mbtts
Copy link

mbtts commented Nov 6, 2021

Is there an existing issue for this?

  • I have searched the existing issues

This issue exists in the latest npm version

  • I am using the latest npm

Current Behavior

I am attempting to run a command from a github package containing a build script:

npx -y mbtts/npx-build-script

The exit code is 1.

Output with loglevel of silly.:

npx -y --loglevel silly mbtts/npx-build-script
… [truncated] …
npm timing command:exec Completed in 4346ms
npm verb exit 1
npm timing npm Completed in 4817ms
npm verb unfinished npm timer reify 1636174689958
npm verb unfinished npm timer reify:unpack 1636174691674
npm verb unfinished npm timer reifyNode:node_modules/npx-build-script 1636174691674
npm verb code 1

When attempting an almost identical package (with a build- script instead of a build script):

% npx mbtts/npx-build-script#fixed
hello

The exit code is 0.

Output with loglevel of silly.:

% npx -y --loglevel silly mbtts/npx-build-script#fixed
… [truncated] …
hello
npm timing command:exec Completed in 1236ms
npm verb exit 0
npm timing npm Completed in 1511ms
npm info ok 

Expected Behavior

I expect the behaviour to be the same regardless of whether the script is named build or build-.

I also expect the same behaviour from earlier versions of npx (for example npm v6.14.15 on Node.js v14.18.1).

The bin script should run and print “hello“:

% npx mbtts/npx-build-script 
npx: installed 1 in 3.153s
hello

If it is not possible to use a build script with a github dependency run via npx then I would expect this to be clearly documented.

Steps To Reproduce

  1. In this environment: macOS 11.5.1 (20G80), nvm managed Node.js and npm install
  2. With this config: Defaults/no custom .npmrc
  3. Run 'npx -y mbtts/npx-build-script'
  4. See error: exit code 1

To reproduce the working example (same environment/config):

  1. Run 'npx -y mbtts/npx-build-script#fixed'
  2. See error: exit code 0

You can compare the original and fixed versions to prove the only difference is the script name.

Environment

  • npm: 8.1.3
  • Node: v16.13.0
  • OS: macOS 11.5.1 (20G80)
  • platform: Macbook Pro (Intel 64 bit)
  • npm config:
; node bin location = /Users/bettsm/.nvm/versions/node/v16.13.0/bin/node
; cwd = /Users/bettsm
; HOME = /Users/bettsm
; Run `npm config ls -l` to show all defaults.
@mbtts mbtts added Bug thing that needs fixing Needs Triage needs review for next steps Release 8.x work is associated with a specific npm 8 release labels Nov 6, 2021
Siegrift added a commit to api3dao/old-api3-docs that referenced this issue Feb 10, 2022
The `npx` command seems to fail because of the github dependencies that are used in services.
I am saying "seems", because I am not sure, but running `npx --loglevel verbose `

Also, there are some issues on the NPM repo mentioning problems with github dependencies:
npm/cli#4137
npm/cli#4003

I don't think many people use them in production, so that's why I expect
I didn't find the exact issue. Gladly, I empirically found out that
replacing `npx` with `npm exec` works. Also `npm i` works. Also, after
doing `npm i` the `npx` starts to work (likely because it uses the
cached package downloaded by npm - also this was probably a reason why
the package worked for people testing
https://api3dao.atlassian.net/browse/BEC-138).
Siegrift added a commit to api3dao/old-api3-docs that referenced this issue Feb 10, 2022
The `npx` command seems to fail because of the github dependencies that are used in services.
I am saying "seems", because I am not sure, but running `npx --loglevel verbose --package @api3/services --call "create-beacon-reader-app --help"` fails after downloading the utility contracts package.

Also, there are some issues on the NPM repo mentioning problems with github dependencies:
npm/cli#4137
npm/cli#4003

I don't think many people use them in production, so that's why I expect
I didn't find the exact issue. Gladly, I empirically found out that
replacing `npx` with `npm exec` works. Also `npm i @api3/services`
works. Also, after doing `npm i` the `npx` starts to work (likely
because it uses the cached package downloaded by npm - also this was
probably a reason why the package worked for people testing
https://api3dao.atlassian.net/browse/BEC-138).
@pmunin
Copy link

pmunin commented Mar 29, 2022

Confirm - issues is still here with versions:

  npm: '7.24.0',
  node: '16.10.0',
  v8: '9.3.345.19-node.14',
  uv: '1.42.0',
  zlib: '1.2.11',
  brotli: '1.0.9',
  ares: '1.17.2',
  modules: '93',
  nghttp2: '1.42.0',
  napi: '8',
  llhttp: '6.0.2',
  openssl: '1.1.1l+quic',
  cldr: '39.0',
  icu: '69.1',
  tz: '2021a',
  unicode: '13.0',
  ngtcp2: '0.1.0-DEV',
  nghttp3: '0.1.0-DEV'

@nlf
Copy link
Contributor

nlf commented May 31, 2022

this is expected and intentional behavior. if a git dependency defines a build script then npm determines while installing it, we need to build it first. so we clone the repo to a temp directory, we install of the dependencies, we run the build script, then we package up the result and install that into npx's cache.

if you have pre-built artifacts (like you do in your build directory), then renaming the script to something other than build is the correct way to work around this.

@nlf nlf closed this as not planned Won't fix, can't repro, duplicate, stale May 31, 2022
@mbtts
Copy link
Author

mbtts commented Jul 20, 2022

Thank you for the explanation:

  1. Was this conscious change in behaviour between 6.x. and 7.x: “if a git dependency defines a build script then npm determines while installing it, we need to build it first”?
  2. Where is/was this documented?

Essentially:

If it is not possible to use a build script with a github dependency run via npx then I would expect this to be clearly documented.

I am more than happy/willing to draft suggested documentation updates… but trying to establish the facts first.

There is also a CLI consideration here: Aside from the exit code there is no message to indicate the problem (even with a loglevel of silly). Originally it was only by process of elimination which revealed the build script was the problem.

@thejustinwalsh
Copy link

thejustinwalsh commented Jun 29, 2023

I just ran into this issue with a package I haven't tried since npm 6.x and am now running npm 9.x.

I could not find any documentation describing that a script named build is a built-in script or a lifecycle event of any other built-in script, and there is no command named build either.

Thanks to @mbtts for documenting this unexpected behavior, as this bug is a very frustrating time sink, and some documentation or a hint in the output would be appreciated.

@nlf, I wonder if these 4M npm packages know that build has a special undocumented meaning in npm > v6?
https://github.com/search?q=path%3A**%2Fpackage.json+%5C%22build%5C%22&type=code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug thing that needs fixing Needs Triage needs review for next steps Release 8.x work is associated with a specific npm 8 release
Projects
None yet
Development

No branches or pull requests

4 participants