New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Yarn does not correctly use an offline mirror cache and saved resolutions for flat installations #3910

Open
markerikson opened this Issue Jul 11, 2017 · 4 comments

Comments

Projects
None yet
3 participants
@markerikson

markerikson commented Jul 11, 2017

Do you want to request a feature or report a bug?

A potential bug, although it's entirely possible that I'm misunderstanding intended behavior. Yarn does not seem to make use of the designated yarn-offline-mirror folder and/or flattened "resolutions" values together, particularly if told to be in offline mode.

What is the current behavior?

I've got an in-progress app repo that has previously used NPM. I'm switching over to Yarn, and I want to have a reproducible offline install with the smallest possible node_modules size.

Right now, if I run yarn with no yarn.lock or existing node_modules, the resulting node_modules size is about 120K+ files on disk. If I run yarn --flat and manually pick resolutions for several dozen libs as prompted, the resulting node_modules size is about 22K files, and my package.json is updated with a "resolutions" section containing the chosen values. So far, so good - obviously "flat" is better size-wise, even if I have to deal with all those conflict resolutions myself.

If I then attempt to clone the repo and install it in a VM or other machine where I have not previously downloaded those packages (and thus have an empty Yarn machine-global cache), and I run yarn --flat --offline, I see multiple warnings similar to:

warning Lockfile has incorrect entry for "lodash@4.17.2". Ignoring it.
warning Lockfile has incorrect entry for "babel-core@6.9.1". Ignoring it.
warning Lockfile has incorrect entry for "babel-runtime@6.9.2". Ignoring it.
warning Lockfile has incorrect entry for "fs-extra@0.26.5". Ignoring it.
warning Lockfile has incorrect entry for "object-assign@4.1.0". Ignoring it.
warning Lockfile has incorrect entry for "symbol-observable@^0.2.3". Ignoring it.

Finally, Yarn reports:

error Couldn't find any versions for "lodash" that matches "4.17.2" in our cache. Possible versions: ""

The behavior is not isolated to Lodash - I've seen it fail on other libs as well as I've been trying to make this work, including minimatch.

If the current behavior is a bug, please provide the steps to reproduce.
I don't have complete repro steps or a separate demo repo at the moment, but in general:

  1. Start with a package.json that defines assorted dependencies that have conflicting transitive dependencies, such as multiple requested versions of Lodash. Also have a .yarnrc file with values of yarn-offline-mirror "./offline-mirror" and yarn-offline-mirror-pruning true.
  2. Install the dependencies using yarn --flat, and generally resolve conflicts by choosing the latest versions
  3. Commit the ./offline-mirror folder, package.json, and yarn.lock
  4. Prepare to install in a clean environment without a node_modules or Yarn machine-global cache. This can be done by either deleting the repo's node_modules and the system's Yarn cache folder, or cloning the repo onto another machine entirely.
  5. In the clean repo environment, run yarn --flat --offline.
  6. Observe the warnings about incorrect lockfile entries, and an error stating that one of the libraries does not match what's in cache.

What is the expected behavior?

Yarn would correctly use a combination of the designated per-repo offline mirror folder, yarn.lock, and the "resolutions" section in package.json to successfully install flattened dependencies based on the offline mirror folder.

Please mention your node.js, yarn and operating system version.

Node 8.1.3.
Yarn: I've tried 0.26.1, 0.27.5, and 0.28.2, with the same results.

@mbluecoder

This comment has been minimized.

Show comment
Hide comment
@mbluecoder

mbluecoder Oct 31, 2017

I thought this was well written. AND, it is exactly my problem.

Why hasn't anyone answered here yet?

mbluecoder commented Oct 31, 2017

I thought this was well written. AND, it is exactly my problem.

Why hasn't anyone answered here yet?

@BYK

This comment has been minimized.

Show comment
Hide comment
@BYK

BYK Nov 1, 2017

Member

I thought this was well written. AND, it is exactly my problem.
Why hasn't anyone answered here yet?

It is some good prose with no way of reliably reproducing the issue easily. Maybe that's why? :)

Member

BYK commented Nov 1, 2017

I thought this was well written. AND, it is exactly my problem.
Why hasn't anyone answered here yet?

It is some good prose with no way of reliably reproducing the issue easily. Maybe that's why? :)

@BYK

This comment has been minimized.

Show comment
Hide comment
@BYK

BYK Nov 1, 2017

Member

@markerikson thank you for the detailed explanation but it is very hard to figure out how to reproduce your issue for investigation so we need a sample repo to reproduce the issue reliably.

Also, the 0.x line for Yarn is quite old, the current version is 1.2.1 with 1.3.0 around the corner so I suggest you try the newest version of Yarn first.

Member

BYK commented Nov 1, 2017

@markerikson thank you for the detailed explanation but it is very hard to figure out how to reproduce your issue for investigation so we need a sample repo to reproduce the issue reliably.

Also, the 0.x line for Yarn is quite old, the current version is 1.2.1 with 1.3.0 around the corner so I suggest you try the newest version of Yarn first.

@markerikson

This comment has been minimized.

Show comment
Hide comment
@markerikson

markerikson Nov 1, 2017

Yeah, I haven't tried this again since I filed this. Thought the repro instructions were pretty straightforward, though.

Don't have time to retry it right now, but I'll see if I can generate a repro in the near future.

markerikson commented Nov 1, 2017

Yeah, I haven't tried this again since I filed this. Thought the repro instructions were pretty straightforward, though.

Don't have time to retry it right now, but I'll see if I can generate a repro in the near future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment