Skip to content

Commit

Permalink
Fixed concurrent unpacks into the same folder (#3090)
Browse files Browse the repository at this point in the history
This is a mitigation for issue #2629.
In some cases two different packages could end up unpacking into the same cache folder.
Example #2629 (comment), package typescript is downloaded twice because yarn.lock has inconsistent entries.
This change prevents more than one package to be unpacked into the same cache folder.

This is not a final fix for #2629, there is a more specific bug in resolver that causes this internal inconsistency but I think fetcher should check that every package will be extracted in a unique folder anyway.

Test Plan:
- I'll add a unit test in the next PR when I'll be fixing the reason for the same package having 2 downloads
- Manual test plan
1. Check out #2629 (comment)
2. yarn install
```
yarn install v0.24.0-0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
warning Pattern ["@angular/core@^2.0.0"] is trying to unpack in the same destination "/Users/bestander/Library/Caches/Yarn/v1/npm-@angular/core-4.0.0-rc.1-7f87b7696b407476e45d6d3c1880a50d5afbb6e3" as pattern ["@angular/core@4.0.0-rc.1","@angular/core@>=2.3.1 <5.0.0 || >=4.0.0-beta <5.0.0"]. This could result in a non deterministic behavior, skipping.
warning Pattern ["typescript@>=2.0.0 <2.2.0"] is trying to unpack in the same destination "/Users/bestander/Library/Caches/Yarn/v1/npm-typescript-2.2.1-4862b662b988a4c8ff691cc7969622d24db76ae9" as pattern ["typescript@2.2.1","typescript@>=2.1.4"]. This could result in a non deterministic behavior, skipping.
[3/4] 🔗  Linking dependencies...
warning "@angular/flex-layout@2.0.0-rc.1" has incorrect peer dependency "@angular/common@^2.2.3".
warning "@angular/flex-layout@2.0.0-rc.1" has incorrect peer dependency "@angular/core@^2.2.3".
warning "@angular/material@2.0.0-beta.2" has incorrect peer dependency "@angular/core@^2.3.0".
warning "@angular/material@2.0.0-beta.2" has incorrect peer dependency "@angular/common@^2.3.0".
warning "@angular/material@2.0.0-beta.2" has incorrect peer dependency "@angular/http@^2.3.0".
warning "@ngrx/effects@2.0.0" has incorrect peer dependency "@angular/core@^2.0.0".
warning "@ngrx/store@2.2.1" has incorrect peer dependency "@angular/core@^2.0.0".
warning "angular2-notifications@0.4.53" has incorrect peer dependency "@angular/core@^2.1.2".
warning "angular2-notifications@0.4.53" has incorrect peer dependency "@angular/common@^2.1.2".
warning "angular2-notifications@0.4.53" has incorrect peer dependency "@angular/platform-browser@^2.1.1".
warning "ng2-translate@5.0.0" has incorrect peer dependency "@angular/core@^2.0.0".
warning "ng2-translate@5.0.0" has incorrect peer dependency "@angular/http@^2.0.0".
warning "redux-batched-actions@0.1.5" has unmet peer dependency "redux@>=1.0.0".
[4/4] 📃  Building fresh packages...
✨  Done in 6.16s.
```
  • Loading branch information
bestander committed Apr 10, 2017
1 parent 45af656 commit fb40251
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 1 deletion.
14 changes: 13 additions & 1 deletion src/package-fetcher.js
Expand Up @@ -76,7 +76,19 @@ export default class PackageFetcher {
}

async init(): Promise<void> {
const pkgs = this.resolver.getPackageReferences();
let pkgs = this.resolver.getPackageReferences();
const pkgsPerDest: Map<string, PackageReference> = new Map();
pkgs = pkgs.filter((ref) => {
const dest = this.config.generateHardModulePath(ref);
const otherPkg = pkgsPerDest.get(dest);
if (otherPkg) {
this.reporter.warn(this.reporter.lang('multiplePackagesCantUnpackInSameDestination',
ref.patterns, dest, otherPkg.patterns));
return false;
}
pkgsPerDest.set(dest, ref);
return true;
});
const tick = this.reporter.progress(pkgs.length);

await promise.queue(pkgs, async (ref) => {
Expand Down
1 change: 1 addition & 0 deletions src/reporters/lang/en.js
Expand Up @@ -93,6 +93,7 @@ const messages = {
allDependenciesUpToDate: 'All of your dependencies are up to date.',
frozenLockfileError: 'Your lockfile needs to be updated, but yarn was run with `--frozen-lockfile`.',
fileWriteError: 'Could not write file $0: $1',
multiplePackagesCantUnpackInSameDestination: 'Pattern $0 is trying to unpack in the same destination $1 as pattern $2. This could result in a non deterministic behavior, skipping.',

yarnOutdated: "Your current version of Yarn is out of date. The latest version is $0 while you're on $1.",
yarnOutdatedInstaller: 'To upgrade, download the latest installer at $0.',
Expand Down

0 comments on commit fb40251

Please sign in to comment.