Fixed concurrent unpacks into the same folder

[bestander]

Summary

This is a mitigation for issue #2629.
In some cases two different packages could end up unpacking into the same cache folder.
Example #2629 (comment), package typescript is downloaded twice because yarn.lock has inconsistent entries.
This change prevents more than one package to be unpacked into the same cache folder.

This is not a final fix for #2629, there is a more specific bug in resolver that causes this internal inconsistency but I think fetcher should check that every package will be extracted in a unique folder anyway.

Test plan

• I'll add a unit test in the next PR when I'll be fixing the reason for the same package having 2 downloads
• Manual test plan

1. Check out yarn install fails with ENOENT: no such file or directory occasionally #2629 (comment)
2. yarn install

yarn install v0.24.0-0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
warning Pattern ["@angular/core@^2.0.0"] is trying to unpack in the same destination "/Users/bestander/Library/Caches/Yarn/v1/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3" as pattern ["@angular/core@4.0.0-rc.1","@angular/core@>=2.3.1 <5.0.0 || >=4.0.0-beta <5.0.0"]. This could result in a non deterministic behavior, skipping.
warning Pattern ["typescript@>=2.0.0 <2.2.0"] is trying to unpack in the same destination "/Users/bestander/Library/Caches/Yarn/v1/npm-typescript-2.2.1-4862b662b988a4c8ff691cc7969622d24db76ae9" as pattern ["typescript@2.2.1","typescript@>=2.1.4"]. This could result in a non deterministic behavior, skipping.
[3/4] 🔗  Linking dependencies...
warning "@angular/flex-layout@2.0.0-rc.1" has incorrect peer dependency "@angular/common@^2.2.3".
warning "@angular/flex-layout@2.0.0-rc.1" has incorrect peer dependency "@angular/core@^2.2.3".
warning "@angular/material@2.0.0-beta.2" has incorrect peer dependency "@angular/core@^2.3.0".
warning "@angular/material@2.0.0-beta.2" has incorrect peer dependency "@angular/common@^2.3.0".
warning "@angular/material@2.0.0-beta.2" has incorrect peer dependency "@angular/http@^2.3.0".
warning "@ngrx/effects@2.0.0" has incorrect peer dependency "@angular/core@^2.0.0".
warning "@ngrx/store@2.2.1" has incorrect peer dependency "@angular/core@^2.0.0".
warning "angular2-notifications@0.4.53" has incorrect peer dependency "@angular/core@^2.1.2".
warning "angular2-notifications@0.4.53" has incorrect peer dependency "@angular/common@^2.1.2".
warning "angular2-notifications@0.4.53" has incorrect peer dependency "@angular/platform-browser@^2.1.1".
warning "ng2-translate@5.0.0" has incorrect peer dependency "@angular/core@^2.0.0".
warning "ng2-translate@5.0.0" has incorrect peer dependency "@angular/http@^2.0.0".
warning "redux-batched-actions@0.1.5" has unmet peer dependency "redux@>=1.0.0".
[4/4] 📃  Building fresh packages...
✨  Done in 6.16s.

[arcanis]

Don't we unpack tarballs into randomly-generated directories, before moving them to their final destination?

[bestander]

If the final destination is not random then it won't help

…

On Mon, 10 Apr 2017 at 19:44, Maël Nison ***@***.***> wrote: Don't we unpack tarballs into randomly-generated directories, before moving them to their final destination? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3090 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACBdWFJFB269Av4hZO8wpS3D6uWLqaoPks5runh-gaJpZM4M5IBi> .

[arcanis]

Hm, I guess the rename would fail if both the source and the destination are located on the same filesystem, but yeah, it wouldn't help with the resolver issue.

[FezVrasta]

Is this already published somewhere?

[bestander]

Only in master branch now, the plan is to get it into 0.24.0 next week.

[FezVrasta]

Could it solve the error reported here?
lerna/lerna#752 (comment)

[bestander]

It does seem related.
You can try nightly builds when there is a new one available https://yarnpkg.com/en/docs/nightly

[FezVrasta]

I ran:

git clone git@github.com:yarnpkg/yarn.git
cd yarn
yarn
yarn build-dist
cp -r build-dist /my/project/folder/yarn

Then inside the Dockerfile:

[...]
COPY yarn /opt/yarn
RUN ln -s /opt/yarn/bin/yarn /bin/yarn
[...]

But I get the same error, should I open a new issue?

[bestander]

Yes, go ahead, with a repro steps please.
Also some instructions how to run the dockerfile from zero would be nice.

[FezVrasta]

I'm creating a repository/repro to make everything easier.
#3104