npm doesn't gracefully handle EPERM errors that are non-fatal #3565

Closed
kevinastone opened this Issue Jun 15, 2013 · 45 comments

Projects

None yet
@kevinastone

Okay, this is a bit of an obscure case, but I develop using a virtual machine that provides a shared file system between guest and host (VMWare Fusion specifically). The shared filesystem doesn't support file permissions changes by the guest OS.

So, if I try to run npm install on the shared mount, it fails with permissions errors.

It would be preferrable that npm generates warnings about the failure to set permissions but still complete the install. In the case of this filesystem, it ignores the file permissions anyways and would be a workable installation.

Here's an example:

root@ubuntu:/mnt/hgfs/example# npm --prefix `pwd`/prefix install -g coffee-script
npm http GET https://registry.npmjs.org/coffee-script
npm http 304 https://registry.npmjs.org/coffee-script
npm ERR! Error: EPERM, chown '/mnt/hgfs/example/prefix/lib/node_modules/coffee-script/package.json'
npm ERR!  { [Error: EPERM, chown '/mnt/hgfs/example/prefix/lib/node_modules/coffee-script/package.json']
npm ERR!   errno: 50,
npm ERR!   code: 'EPERM',
npm ERR!   path: '/mnt/hgfs/example/prefix/lib/node_modules/coffee-script/package.json',
npm ERR!   fstream_finish_call: 'chown',
npm ERR!   fstream_type: 'File',
npm ERR!   fstream_path: '/mnt/hgfs/example/prefix/lib/node_modules/coffee-script/package.json',
npm ERR!   fstream_class: 'FileWriter',
npm ERR!   fstream_stack: 
npm ERR!    [ '/usr/lib/node_modules/npm/node_modules/fstream/lib/writer.js:305:19',
npm ERR!      '/usr/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:222:7',
npm ERR!      'Object.oncomplete (fs.js:107:15)' ] }
npm ERR! 
npm ERR! Please try running this command again as root/Administrator.

By comparision, installing on the native guest OS filesystem and copying to the shared FS works fine:


root@ubuntu:/mnt/hgfs/example# npm --prefix /tmp/example install -g coffee-script
npm http GET https://registry.npmjs.org/coffee-script
npm http 304 https://registry.npmjs.org/coffee-script
/tmp/example/bin/coffee -> /tmp/example/lib/node_modules/coffee-script/bin/coffee
/tmp/example/bin/cake -> /tmp/example/lib/node_modules/coffee-script/bin/cake
coffee-script@1.6.3 /tmp/example/lib/node_modules/coffee-script

root@ubuntu:/mnt/hgfs/example# cp -r /tmp/example prefix

root@ubuntu:/mnt/hgfs/example# ./prefix/bin/coffee --version
CoffeeScript version 1.6.3
@isaacs
Member
isaacs commented Jun 17, 2013

If that file system was actually a typical Linux fs, then the failure to chown the file would be a fatal error.

Sorry, this can't be reasonably fixed. There's no way to know that the fs is a guest system rather than a "regular" filesystem. It would be nice to make that kind of thing configurable, but it'd be a pretty significant undertaking.

@isaacs isaacs closed this Jun 17, 2013
@benissimo

+1 for the request for an option to skip running chown. I have a similar case using shared folder mounted on NFS: host is Mac OS, guest is Ubuntu VM. Using virtualbox via vagrant (with chef). My workaround is to have the chef recipe ignore the failure, but that's just a bandaid...

p.s. another workaround is to, within your guest, copy your local dir under /tmp (off of the shared folder), run npm install from there, and then copy the node_modules back into your shared folder. Again, a hack, but better than nothing until there's an option to skip running chown.

@DanielHudson

+1 too... I also have a shared folder... (sshfs)

@psi-4ward

+1

@broql
broql commented Nov 10, 2013

+1 (vagrant + ubuntu + shared folder on Windows 7)

@Nagib
Nagib commented Dec 19, 2013

+1 (vagrant + ubuntu + shared folder over NFS)

@DanielHudson

In Vagrantfile...

config.vm.synced_folder ".", "/vagrant", :mount_options => ["dmode=777","fmode=777","uid=33","gid=33"]

Worked for me... 33 is www-data in my case...

@Nagib
Nagib commented Dec 19, 2013

For me it does not work because I cant rely on user id (uid). I fixed this issue using sudo on provision file like this:

sudo su vagrant -c "npm install"
@kevinastone

I wrote a wrapper script that works around this file permissions issue by executing npm with package.json in a temp directory, and then rsyncs back the node_modules folder to your shared filesystem folder.

https://gist.github.com/kevinastone/8790717

@jsalvatier

+1 Thanks @kevinastone that worked for me.

@DanielHudson

+1 @kevinastone perfect!

@kiwe
kiwe commented Jul 1, 2014

My setup: Windows 7 as host, running Ubuntu 14.04LTS as guest under Vagrant + VirtualBox.

I tried @kevinastone's script, but that didn't solve all my problems. I was trying to install gulp. One problem was with the permissions, another seemed to be with the path length created by npm's dependency structure, which samba didn't like. I resolved this by creating a temporary node_modules directory outside the vagrant shared folder and create a symlink to it in my project folder. I will not be able to access the npm installed modules from Windows this way, but that is not a problem for me at this point.

So, to clarify: /vagrant is my shared folder and /vagrant/project is where my web app resides:

cd /vagrant/project
mkdir /tmp/node_modules
ln -sf /tmp/node_modules node_modules
@isaacs
Member
isaacs commented Jul 1, 2014

Can you please upgrade to the latest npm and check if this is still a problem? It should be fixed in the latest graceful-fs.

npm i -g npm

and then try it.

@kevinsimper

I can make it work with packages like "express", but "casperjs" fails if you do "sudo npm install casperjs --save".

https://gist.github.com/kevinsimper/e55daf861ce11514b6e8

@kiwe
kiwe commented Jul 9, 2014

I installed gulp again in a smb-shared folder from inside vagrant+ubuntu system, to reproduce some errors. Posted the npm log and error output on gist. In the npm-stderr.log you can see some EPERM-errors.

https://gist.github.com/kiwe/65364304c300e668424f

@dignifiedquire

Running Vagrant VM Ubuntu 14.04, and host system OS X 10.9.4 results in the same EPERM errors.
NPM version is 2.0.0-alpha-5, but I tried most of the npm versions released in the last 6 month with the same result.

@zajca
zajca commented Aug 12, 2014

I run in to same problem. I have build server for projects and to this server is attached production server disk using sshfs. For now I'm using rsync script in JenkinsCI to sync data from local disk to sshfs disk. Because of npm changing ownership. But this makes my build about 4times longer since I have to rsync data from sshfs to local disk and back.

What I don't understand is to what and why npm changing ownership. In my environment everything is runs under user "jenkins", default group for this user is "jenkins". This user and group is only one allowed for sshfs disk, so when jenkins user creates something it's always have permissions jenkins:jenkins 664, so there is no point in using chown for this files.

Am I missing something?? Can somebody explain me why this is happening and why there is no switch like --disable-chown??

@benpatterson

Seeing the same issue as @zajca on npm set config registry. npm is installed on the system via sudo, and we have the 'jenkins' user running npm installs. For us, the chown error shows up when setting the registry configuration. I see the .npmrc change taking place under /home/jenkins; however, the chown fails and the command exits 50.

No shared disks, but FWIW it is running on vagrant in our current example.

@benpatterson

FWIW it might be line 276 here: https://github.com/npm/npmconf/blob/016b2988e52f1dc1f95a19f124ac76c714228b78/npmconf.js

Or perhaps more accurately, iterating through the contents of 'where' on line 236, which eventually gets you to whoever installed npm.

Still only an educated guess but thought I'd mention it.

@othiym23 othiym23 changed the title from NPM Doesn't Gracefully Handle EPERM Errors that are Non-Fatal to npm doesn't gracefully handle EPERM errors that are non-fatal Sep 21, 2014
@othiym23 othiym23 added the support label Sep 21, 2014
@ilkkao
ilkkao commented Nov 12, 2014

Our team's setup is Mac OS + VMWare Ubuntu + docker. In dev mode Mac OS directory is mounted inside docker container using VMWare shared folder. Docker tools pretty much hide the whole VMWare setup except you get the permission error while running npm install

+1 for for having --disable-chown

@alynurly

+1 for benissimo, it worked thank you very much

@larrybotha

I was having trouble with permissions in Vagrant + Ubuntu and npm install attempting to create symlinks.

The following prevents the creation of symlinks when packages are installed, installing packages without issue from within the VM:

$ npm install --no-bin-links
@lamosty
lamosty commented Feb 20, 2015

If you are using vagrant, you can install vagrant-bindfs and set --chown-ignore and --chmod-ignore to true. npm seems to be working fine after that.

@kevindeleon

@lamosty Where/how did you set this? I've set it in my Vagrantfile like so:

config.bindfs.bind_folder "/mnt/vagrant-#{i}", "#{folder['target']}", owner: sync_owner, group: sync_group, perms: "u=rwX:g=rwX:o=rD", chown_ignore: true, chmod_ignore: true and am still having this exact same permissions issue.

@lamosty
lamosty commented May 13, 2015

Hey @kevindeleon,

I haven't worked with Vagrant for a while, but here's the line I used in my Vagrantfile:

config.bindfs.bind_folder "/vagrant-nfs", remote_www_root, u: 'www-data', g: 'vagrant', p: 'u=rwx:g=rwx:o=rD', chown_ignore: true, chmod_ignore: true, m: 'vagrant,www-data'

-m is for mirror: "Comma-separated list of users who will see themselves as the owners of all files." http://bindfs.org/docs/bindfs-help.txt

The whole Vagrant file: https://github.com/lamosty/wp-dev-vagrant/blob/master/Vagrantfile

@kevindeleon

Hey! @lamosty

Thanks so much for the reply...I'll give that a shot later. Currently I am just using the "default" sync mode instead of nfs, and all is well.

@kevindeleon

@lamosty Just a bit of an update...I added the m: 'vagrant, www-data' to my file, and that seems to have done the trick for me (along with the chmod_ignore, etc...). Back to using NFS!

@lamosty
lamosty commented May 13, 2015

@kevindeleon Happy to hear that!

@iyn
iyn commented May 14, 2015

I also use Vagrant with NFS, but can't use bindfs with vagrant-bindfs (CoreOS won't let you install anything and vagrant-bindfs needs to install bindfs inside a VM). I use Docker containers for my apps, so it means that every command run inside container is run by root. Yet, npm still doesn't work.

My current workaround is to use npm commands on the host system, it'll normally install packages to node_modules/ and update package.json, when run with --save. It works ok, but I'd prefer to run everything inside Vagrant/Docker, hope this bug will be resolved.

@kdemarest

+1 this is significant and absolutely should be fixed. The more general case of npm failures over nfs should not require a work-around. It should simply work.

@nicolag
nicolag commented Jul 21, 2015

+1 I'm using fusion VM, npm v 2.11.3, node v 0.12.7, when I run npm install gave me error

....
npm ERR! errno: -4048,
npm ERR! code: 'EPERM',
npm ERR! path:'Z:\\path_to_folder\\node_modules\\.bin\\module_name
....
npm ERR! Please try running this command again as root/Administrator

anyone able to solve?

@khromov
khromov commented Jul 24, 2015

๐Ÿ‘ Looking for a fix as well.

@ptica
ptica commented Jul 31, 2015
I'm @ {"node":"0.12.7","npm":"2.13.3"} vagrant via nfs share
npm ERR! code EPERM
npm ERR! errno -1
npm ERR! Error: EPERM, chown '/vagrant/node_modules/grunt-jspot/package.json'
@panuhorsmalahti

For me, running "node" as administrator once fixed the issue on Windows 2012 VM (with Vagrant).

@friday
friday commented Aug 2, 2015

I had this problem (running as sudo) and was able to fix it with bindfs, as suggested by @lamosty.

Installing the plugin: vagrant plugin install vagrant-bindfs

Then I changed my Vagrantfile from:
config.vm.synced_folder ".", "/vagrant/src", create: true, nfs: true

To:
config.vm.synced_folder ".", "/vagrant/nfs", create: true, nfs: true config.bindfs.bind_folder "/vagrant/nfs", "/vagrant/src"

I don't really know what I'm doing. I didn't have to add --chown-ignore or --chmod-ignore, but it worked for me.

@deletosh

@kiwe Perfect solution!

@ericsaboia

@friday solution worked to me

@Prizzler
Prizzler commented Sep 7, 2015

same problem on vbox

Vagrant + VBOX + Ubuntu + MacOS

the solution is don't npm install as root

roots/sage#1207

@keatz55
keatz55 commented Sep 12, 2015

@kevinastone Where did you put the script?

@keatz55
keatz55 commented Sep 12, 2015

@kevinastone Thanks for the response!

@gmarchand gmarchand referenced this issue in puphpet/puphpet Oct 22, 2015
Closed

Cannot Utime in nfs shared folder #1953

@clefru
clefru commented Mar 28, 2016

I am with the comment from zajca on Aug 12, 2014. I never understood npm's business of calling chown. That's seems like a broken design, and graceful-fs seems like a wrong fix. But, hey what do I know, I looked at this problem for 10 minutes, and as I didn't know how to make npm use graceful-fs, I did:

echo "int chown() { return 0; }" > preload.c && gcc -shared -o preload.so preload.c LD_PRELOAD=$PWD/preload.so npm install whateveryouwant

@trakos trakos referenced this issue in codekitchen/dinghy Apr 13, 2016
Closed

npm throws Operation not permitted #17

@develar
develar commented May 21, 2016

Solution โ€” since NPM is so slow and unreliable, use named data volumes. https://madcoda.com/2016/03/docker-named-volume-explained

Currently, NPM is broken in mostly all docker drivers โ€” except Parallels (but parallels has another issue โ€” Parallels/docker-machine-parallels#52) It is broken in the private Docker beta as well. I really, really hope that NPM will be fixed some day. But for now named data volumes is a rescue.

@mafrosis

@clefru EPIC hack dude.. And it works ๐Ÿ˜†

@jonnyt886

Ping. This is still an issue for me (npm 3.10.9 / node 6.9.2 on Arch). @clefru's hack further up is what I use at the moment to get around this but I've never come across any other development tools that fail in this way. It's quite irritating that I use npm for everything but need to run a wrapper script to run npm install (and on every machine using the code base, not just mine).

Could someone please explain why using chmod() is needed? Could you use umask() instead, or perform other validation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment