Gatsby breaks with yarn/lerna workspaces

[AndyMoreland]

I am using yarn 1.0.2 and lerna to work with many projects and share library code between them.

This combination of tools hoists non-conflicting dependencies to the project root's node_modules folder. Consequently, gatsby gets hoisted up one level higher than the sub-package's node_modules folder, which seems to make gatsby choke.

$ gatsby --version
error A local install of Gatsby was not found.

gatsby is in ../node_modules/.bin/gatsby.

[mickeyreiss-visor]

I'm seeing this issue as well. Here's my manifestation:

» yarn start
yarn run v1.2.1
$ gatsby develop --port 4567
success delete html files from previous builds — 0.029 s
success open and validate gatsby-config.js — 0.005 s
error UNHANDLED REJECTION


  Error: Unable to find plugin "gatsby-plugin-segment"

  - load-plugins.js:96 resolvePlugin
    [Glue]/[gatsby]/dist/bootstrap/load-plugins.js:96:11

  - load-plugins.js:147 processPlugin
    [Glue]/[gatsby]/dist/bootstrap/load-plugins.js:147:27

  - load-plugins.js:168
    [Glue]/[gatsby]/dist/bootstrap/load-plugins.js:168:28

  - Array.forEach

  - load-plugins.js:167 _callee$
    [Glue]/[gatsby]/dist/bootstrap/load-plugins.js:167:28

  - index.js:122 _callee$
    [Glue]/[gatsby]/dist/bootstrap/index.js:122:20


error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[mickeyreiss-visor]

It looks like this code loads plugins via require.resolve. With yarn workspaces, the dependencies are hoisted, so they are located in ../node_modules/. I'm not sure why require.resolve isn't locating those workspace dependencies correctly.

[mickeyreiss-visor]

I have some more details now.

Gatsby uses node's require.resolve to load plugins. This function follows the standard node module resolution algorithm, which recursively checks node_modules directories from the current directory up to the root.

The problem is that with yarn workspaces (and hoisted dependencies in general), gatsby may be in /<PROJECTROOT>/node_modules, while plugins may be in /<PROJECTROOT>/subproject/node_modules. Therefore, gatsby's modules cannot locate the plugins.

ember-cli/ember-cli#7401 seems to be the fix implemented by ember-cli, which encountered a very similar issue. It seems that the resolve module was built to solve this problem.

And here are some nice docs from lerna that describe caveats of yarn's workspace hoisting technique.

[KyleAMathews]

Super interesting @mickeyreiss-visor! Thanks for the research.

Also related is #2831

Let's see if we can get things right for v2!

[geoffreydhuyvetters]

there is a nohoist option WIP / RFC for yarn workspaces, that could help I guess yarnpkg/rfcs#86

[barraq]

@KyleAMathews I tried to extend the webpack config like this:

exports.modifyWebpackConfig = ({ config }) => {
  config.merge({
    resolveLoader: {
      root: [path.resolve(__dirname, '../../node_modules')],
    },
    resolve: {
      // should use `modules` in next version of Webpack
      modulesDirectories: [path.resolve(__dirname, '../../node_modules')],
    }
  });
  return config;
};

where the root of my monorepo (using yarn workspace) is at ../../.

Yet it did not solve the issue, which is weird as both the resolveLoader and resolve now also includes the root node_modules.

[levino]

Any news on this?

[levino]

Fix for me

[KyleAMathews]

Due to the high volume of issues, we're closing out older ones without recent activity. Please open a new issue if you need help!