Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert "yarn" executable to a shell script that runs either "node" or "nodejs" #1180

Merged
merged 2 commits into from
Oct 18, 2016

Conversation

Daniel15
Copy link
Member

Summary
Converted yarn to be a shell script that checks for both node or nodejs. If neither is found, shows an error stating that Node.js needs to be installed.

Test plan
Tested in the following environments:

  • Ubuntu 16.04, Node.js v6.7.0 installed from NodeSource repo
  • Debian Testing, Node.js v4.3.1 installed from Debian repo
  • Cygwin on Windows 10
  • WSL (Bash on Ubuntu on Windows) on Windows 10

Built Debian package with ./scripts/build-deb.sh and tested it on Ubuntu.

Background
Yarn always executes node (due to the shebang of #!/usr/bin/env node), and always executes nodejs on Debian/Ubuntu (we sed the shebang when building the package: https://github.com/yarnpkg/yarn/blob/master/scripts/build-deb.sh#L74).

The tricky/unfortunate thing with Node.js on Debian is that /usr/bin/node conflicted with another package. See:
https://lists.debian.org/debian-devel-announce/2012/07/msg00002.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=614907

In the end, both packages that used the node binary name were renamed - Node.js became nodejs, and the other Node (some ham radio app) became ax25-node. This means that when installing Node.js via Debian package, the binary is called nodejs. However, for other installation methods (such as nvm), the binary is called node.

Installing both Yarn and Node.js via Debian package is generally the preferred method, and works fine. However, this naming does have several implications when mixing and matching different installation styles:

  • Yarn installed via npm will fail with Node.js installed via Debian package (it'll try to run node which doesn't exist)
  • Yarn installed via Debian package will fail with Node.js installed via nvm (it'll try to run nodejs which doesn't exist)

Often people work around this by symlinking /usr/bin/node to /usr/bin/nodejs or vice-versa, but that's hacky.

Closes #1142
Closes #819

@@ -1,2 +1,27 @@
#!/usr/bin/env node
require('./yarn.js');
#!/bin/sh
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mostly copied from what npm does, which uses @ForbesLindesay's cmd-shim: https://github.com/ForbesLindesay/cmd-shim/blob/master/index.js#L115

esac

command_exists() {
command -v "$1" >/dev/null 2>&1;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

command -v is a built-in POSIX command that returns the path to the specified command, or exits with an error code if the command doesn't exist.

daniel@debian:~/src/yarn$ command -v nodejs; echo $?
/usr/bin/nodejs
0
daniel@debian:~/src/yarn$ command -v does_not_exist; echo $?
1

It's built-in to the shell so it's very fast.

@Daniel15
Copy link
Member Author

cc @cpojer @bestander - Would be good to get this into the next release, we just need to test it on Mac OS to ensure the script works there (I don't have a Mac to test it on)

@wmertens
Copy link

wmertens commented Oct 18, 2016

I have to question the sanity of this, by doing this you are basically saying that node can be called whatever and that all programs out there have to implement this logic, even in rarely-used scripts in some dusty corner of the code.

Debian has an alternatives mechanism iirc, which lets you choose which program maps to a given name. Why not use that instead?

And NixOS makes wrappers that set up the PATH so that node points to the exact version you specify. That is also a valid mechanism.

I believe that by renaming node to nodejs you cause a maintenance nightmare for all projects using Node. 👎

@ForbesLindesay
Copy link

I've found that people who end up in the situation of having node called nodejs run into a lot of issues with other command line scripts (e.g. create-react-app, post install scripts for modules etc.).

Although we can make yarn resilient to this issue, it just pushes the problem further down the line. I think the right solution here is just to provide clear messaging on how to correctly alias nodejs to node. In the long run, this will save people a lot of pain.

@Daniel15
Copy link
Member Author

I didn't want to break the out-of-the-box experience for Ubuntu and Debian
users, but maybe you're right Forbes. Still, this is strictly better than
the current situation, where it only looks for a "nodejs" (not "node")
binary on Debian and Ubuntu.

I believe that by renaming node to nodejs you cause a maintenance
nightmare for all projects using Node

That's not my choice though; I'm not the maintainer of Node.js. That's the
current situation today.

Maybe I'll email the Node.js maintainers and see if they can use the
alternatives mechanism to alias Node. For now, I think we do need to check
for both binary names.

Sent from my phone.

On Oct 18, 2016 3:15 AM, "Forbes Lindesay" notifications@github.com wrote:

I've found that people who end up in the situation of having node called
nodejs run into a lot of issues with other command line scripts (e.g.
create-react-app, post install scripts for modules etc.).

Although we can make yarn resilient to this issue, it just pushes the
problem further down the line. I think the right solution here is just to
provide clear messaging on how to correctly alias nodejs to node. In the
long run, this will save people a lot of pain.


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1180 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFnHebdIdwhvmPLzwnemzsuKudvXBWAks5q1JvxgaJpZM4KZY64
.

@bestander
Copy link
Member

Let's merge this in as it solves the current problem.
But in the mean time let's keep the discussion going.

@bestander bestander merged commit e2f5c26 into yarnpkg:master Oct 18, 2016
@bouk
Copy link
Contributor

bouk commented Oct 20, 2016

@Daniel15 this PR broke symlinking to the yarn executable, because you're not taking into account that $0 might be a symlink

@Daniel15
Copy link
Member Author

Yeah I noticed that :(. This is how Node does it though - you'd see the
same issue with npm itself, along with any other npm packages that have
executables.

We can use readlink to resolve the symlink, but I don't know if that's
POSIX compliant so it might not work in all environments.

Sent from my phone.

On Oct 20, 2016 8:54 AM, "Bouke van der Bijl" notifications@github.com
wrote:

@Daniel15 https://github.com/Daniel15 this PR broke symlinking to the
yarn executable, because you're not taking into account that $0 might be
a symlink


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1180 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFnHR4hnCR-Aw0gT4xa2VBUS8VBUuVfks5q147OgaJpZM4KZY64
.

@wmertens
Copy link

So now we are piling code on the hole made by supporting non-standard practices… this way madness lies.

@Daniel15
Copy link
Member Author

I'm going to email the maintainers of Node.js packaging and see how they feel about using the update-alternatives mechanism for symlinking /usr/bin/node to /usr/bin/nodejs, or something similar.

So now we are piling code on the hole made by supporting non-standard practices

We need to support it as long as the Linux distribution with majority marketshare (Ubuntu) continues to do it this way. I'm reasonably certain that more people are using a Debian/Ubuntu Node.js package compared to something like nvm.

@Daniel15 Daniel15 deleted the exec-nodejs branch October 20, 2016 20:16
@Daniel15
Copy link
Member Author

Alternatively, we could symlink /usr/bin/node to /usr/bin/nodejs as part of installing Yarn? It feels a bit intrusive though. 😕

@ljharb
Copy link

ljharb commented Oct 20, 2016

Please don't proliferate nodejs usages - only apt calls it that, the rest of us do the right thing :-/

@wmertens
Copy link

Well, you could test if yarn.js is where you think it is, and otherwise use readlink, which most likely is installed. If it is not installed, give up :)

@Daniel15
Copy link
Member Author

Please don't proliferate nodejs usages

@ljharb - The new version is strictly better than the old one - The Debian/Ubuntu package used to only look for nodejs (the shebang was sed-ded as part of the build script), now it looks for both node and nodejs.

The naming of the binary on Ubuntu/Debian is up to the Node.js maintainers, not me. We need to support whatever they do, particularly given Ubuntu's prevalence.

@Daniel15
Copy link
Member Author

Well, you could test if yarn.js is where you think it is, and otherwise use readlink, which most likely is installed. If it is not installed, give up

sure, we can create a separate Github issue for following up on that :)

@Daniel15
Copy link
Member Author

@wmertens @ForbesLindesay et. al - I emailed the Debian JavaScript maintainers and cc'd the maintainer of Node.js:

Hello Debian JavaScript maintainers!

I'm one of the developers on Yarn (https://yarnpkg.com/), a new package manager for JavaScript designed to replace the "npm" client. As part of our launch, I created Debian packages and am hosting them in our own repository.

For the initial release, I used #!/usr/bin/nodejs as the shebang, due to the fact that the Debian Node.js release calls its binary "nodejs" instead of "node". However, this caused issues on systems where people have installed Node.js through other installation methods. In an ideal world, all dependencies would be installed through dpkg/apt-get, but unfortunately that's not the case in the Node.js ecosystem. Some people install Node.js through an application called "nvm" (https://github.com/creationix/nvm), which is designed to allow the user to toggle between different Node.js versions. This is useful for developers to easily test their code on multiple different Node.js versions, as using the Debian Node.js package does not allow multiple versions to be installed concurrently.

Since nvm uses the upstream Node.js releases directly, its binaries are named "node" instead of "nodejs". This means that I needed to modify Yarn to use a shell script that checks for both "node" (for users of nvm and Node.js on non-Debian platforms) and "nodejs" (for users of the Debian node.js packages). This works for Yarn itself, but there's many many scripts and packages on npm that rely on the binary being called "node" rather than "nodejs" and totally fall apart on Debian/Ubuntu. A lot of users end up symlinking /usr/local/bin/node to /usr/bin/nodejs to get a usable environment.

Given the amount of complexity stemming from Debian renaming this binary and the fact that it breaks parts of the Node.js ecosystem, is there any chance it will be renamed back to "node" in the future, or for /usr/bin/node to be symlinked to /usr/bin/node (perhaps using the update-alternatives system to allow the user to choose whether they want to symlink nodejs or ax25-node, in case they have both installed)?

Thank you!

Kind Regards,
Daniel

Feel free to reply to the mailing list post if there's anything you'd like to add 😄

https://lists.alioth.debian.org/pipermail/pkg-javascript-devel/2016-October/014444.html

@ForbesLindesay
Copy link

Doesn't look like their view on this has changed much. Sadly this is a triumph of rules and technical correctness over pragmatism. We should push for:

c) Change ruling through the tech-ctte or a general solution vote

even if that will take until 2019, at least that way this nonsense won't be permanent. I expect node (and I hope yarn) will still be in common usage in 2019 so this will still be a huge win.

In the mean time, questions I would have are:

  1. Can someone rename node-legacy to something more accurate. It makes it look like the legacy version, when in reality it's the correct/recommended version.
  2. Can we depend on nodejs-legacy for now, this may prevent yarn being part of official Debian but I don't know how much of a problem that is?

@Daniel15
Copy link
Member Author

Daniel15 commented Oct 27, 2016

Can we depend on nodejs-legacy for now

This is risky since the user might already have a node binary (eg. if they installed node via nvm). nodejs-legacy creates a symlink from /usr/bin/node to /usr/bin/nodejs. I don't think we can install nodejs-legacy unless the user explicitly wants it.

thomasjm added a commit to thomasjm/stackage that referenced this pull request Feb 25, 2018
Fixes a problem on commercialhaskell/lts-haskell#94
See yarnpkg/yarn#1180 for more context on the
underlying problem
thomasjm added a commit to thomasjm/stackage that referenced this pull request Feb 25, 2018
Fixes a problem on commercialhaskell/lts-haskell#94
See yarnpkg/yarn#1180 for more context on the
underlying problem
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants