EINVAL: invalid argument, symlink

[atrauzzi]

So I'm getting this error, which I suspect I already know the reason for:

Arguments: 
  /usr/bin/nodejs /usr/bin/yarn add react-redux

PATH: 
  /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Yarn version: 
  0.17.6

Node version: 
  6.9.1

Platform: 
  linux x64

Trace: 
  Error: EINVAL: invalid argument, symlink '../../../loose-envify/cli.js' -> '/MYPROJECT/node_modules/invariant/node_modules/.bin/loose-envify'
      at Error (native)

The reason this is happening is because I'm using Docker for Windows and the path it's looking to create a symlink on is actually backed by an NTFS filesystem. This is going to be fairly common for anyone trying to use yarn through a Docker container on Windows.

I'm not sure what the best course of action would be here, but I don't mind even just having to tell yarn to skip doing symlinks. You might however have an opportunity here to make the user experience a little nicer by detecting when this happens and either raising a warning, or improving the error feedback.

Anyway, this is a thing :)

[meta-meta]

bump

[adiachenko]

Interestingly, npm install works just fine on the latest Beta version of Docker for Windows. I am not a big fan of falling back to npm though.

[meta-meta]

I'm currently working around this with yarn install --no-bin-links then yarn global add for things needed on the command line like webpack and webpack-dev-server since that puts the modules on the containers filesystem.

[adiachenko]

@meta-meta Well, you could do that, but it requires changes to docker images that are tightly coupled with your codebase.

[meta-meta]

@adiachenko I don't love it, but it works for now. Hoping this goes away soon.

[mlgamble]

bump - Exact same situation as OP

[cascornelissen]

I'm running into the same issue... The workaround provided by @meta-meta seems to work but that doesn't really work for my use case either.

NPM seems to handle this perfectly.

[kvahuja]

I am running into the same issue when installing create-react-app. any clue what to do to fix this?

[FDiskas]

🍝 same problem

[novascreen]

I've created a small script that writes the bin links for all dependencies in package.json.
So i run yarn install with --no-bin-links and then run the script afterwards.
https://gist.github.com/novascreen/5e733e90050a0a9e2d8cbae5f188288c
Hopefully this is fixed soon and all i need to do is remove the file and the flag.

[roedit]

+1

[usmanatron]

Thanks @novascreen - this worked beautifully for me! Plus I still don't need to use npm :-)
Hope this gets fixed soon

[wclr]

The same errors in docker

[bbrala]

Yeah, docker uses smb on windows. Yarn isn't handeling mfsymlinks properly which breaks this.

[bbrala]

The weird thing is, npm is creating symlinks fine. I've tried to look into the difference of implementation for the two but haven't found a reason yet unfortunately.

[wclr]

I have this error on Docker for windows, modules get installed in node_modules that is on docker volume

/app - is attached volume
/app/node_modules - is docker data volume

Trying to install for example yarn add localforage, get the error:

error An unexpected error occurred: "EINVAL: invalid argument, symlink '../../../acorn/bin/acorn' -> '/app/node_modules/falafel/node_modules/.bin/acorn'".

But I go to /node_modulesl/node_modules/.bin/

I have no problem doing ln -s ../../../acorn/bin/acorn acorn, So I don't see this as docker-for-windows issue, not clear why yarn gets error doing this.

[FDiskas]

⏰ If you run yarn within a docker - the docker will inform the yarn as he is the Linux OS. Windows by default do not have a symbolic link. You should try run yarn within PC not in docker.

p.s. You can't run ln -s command on mounted volume.
Probably by running npm install you wouldn't be noticed about any errors. And probably command npm install --no-bin-links will get the same results? Does npm if he can't create links acts same as --no-bin-links parameter?

[adiachenko]

You should try run yarn within PC not in docker.

@FDiskas We get it. What we're wondering about at this point is the discrepancy between npm and yarn, because it seems like the former can handle this situation at least to some degree.

[wclr]

I found the problem, this happens because of concurrency in creating symlinks which fails in this case with docker volumes.

https://github.com/yarnpkg/yarn/blob/master/src/package-linker.js#L270

Changing concurrency to 1 fixes the issue.

I've tried just to test if it fails with making many symlinks in parallel inside this node_modules (in which yarn fails), but simple case seem doesn't show any problems, that means that something related to specific yarn's linking flow and need to be investigated further.

cc @kittens @bestander

[atrauzzi]

Nicely done!

[bestander]

We would appreciate a bit more debugging and repro steps.
Bonus points for a PR with a fix :)

[wclr]

Ok added a PR with repro steps.

[bbrala]

Wow this actually fixes it. I tried this fix on master on our vagrant box, which also uses SMB for sharing project folders and now when i install gulp through yarn and npm it doesn't break on invlaid symlink when using the "mfsymlink" mount option. Good find!

vagrant@dev:/home/localcopy/yarntest $ find -L ./ -xtype l

./npm/node_modules/.bin/gulp
./npm/node_modules/.bin/mkdirp
./npm/node_modules/.bin/semver
./npm/node_modules/.bin/strip-bom
./npm/node_modules/.bin/user-home
./npm/node_modules/.bin/which
./yarn/node_modules/.bin/gulp
./yarn/node_modules/global-prefix/node_modules/.bin/which
./yarn/node_modules/gulp/node_modules/.bin/semver
./yarn/node_modules/v8flags/node_modules/.bin/user-home
./yarn/node_modules/vinyl-fs/node_modules/.bin/mkdirp
./yarn/node_modules/vinyl-fs/node_modules/.bin/strip-bom

[brazitech]

hello
new to docker just installed latest version and am getting the symlink error. where do i find the
package-linker.js file to change from 4 to 1.

[wclr]

@brazitech look at the PR #3097.

[Vanuan]

Docker is intended to isolate from the filesystem differences.
Unfortunately when you mount a folder from the host you break that isolation.
To seal the hole you have to use a docker volume in the folder where you intend to create symlinks:

    volumes:
      - ./:/src
      - node_modules:/src/node_modules
      - yarn_cache:/root/.cache

This way yarn wouldn't fail.

P.S. I'm not saying yarn doesn't have an issue when dealing with SMB-mounted volume.

[atrauzzi]

To @Vanuan's point, I like to put an entry for node_modules inside of a .dockerignore file. This permanently protects against ever accidentally snagging the native deps.

[Vanuan]

@atrauzzi Well, .dockerignore file doesn't protect against mounted volume. It only protects the production docker use case: when you use yarn install natively and include the whole source folder into an image.

The docker use case discussed here is the development one: you mount a source folder in runtime and run yarn install inside the container, not during the image being built. Effectively you mount SMB folder (windows localhost) into the linux OS (linux virtual machine).

[atrauzzi]

Ah yeah, definitely gotta watch for that :D

[ghost]

Problem still happens

[wclr]

@Aios what version?
Probably PR is not landed in stable?

[ghost]

@whitecolor Yep verison was 0.12.0
Looking for commit latest stable was 14 and 22 days ago. Commit was 9 days ago

[bestander]

Should be fixed in 0.26

[ghost]

@bestander Great)
So can i use nightly build to development environment?

[bestander]

Yep

…

On 26 May 2017 at 12:09, Dave ***@***.***> wrote: @bestander <https://github.com/bestander> Great) So can i use nightly build to development environment? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1961 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACBdWPlMEIvBGBgisYPosOToT1-j-94fks5r9rLNgaJpZM4K3tPb> .

[kingjerod]

This should be fixed right? Getting this error now in a node:11 docker image, running yarn install --no-bin-links inside the Docker container (Windows host).

[kaytotes]

Same using note:latest wasn't having the issue as of a few days ago 🤔.

[djbingham]

I'm seeing this issue running yarn install in a node:11-alpine Docker container (Windows host). No problem running the same in a node:10-alpine container.

[kaytotes]

Can confirm that falling back to a node:10 image works for me too.