Gatsby

[arcanis]

What package is covered by this investigations?

Gatsby

Describe the goal of the investigation

I'd like Gatsby to support PnP painlessly (note that this issue being opened doesn't necessarily mean that they aren't compatible, just that they could work better together).

Investigation report

Some problems encoutered:

Gatsby is missing various peer dependencies in its packages

This got fixed in gatsbyjs/gatsby#11994, gatsbyjs/gatsby#11972, and gatsbyjs/gatsby#11971. Noone should have this problem again.

Gatsby uses the copyFile function to generate its cache folder, and this function wasn't implemented in the fslib (so Gatsby could copy the files from its zip archive to the cache folder)

This got fixed within Yarn itself, in arcanis@10b18f5. Noone should have this problem again.

Gatsby needs to be aware of PnP in order to build its files.

I've added a gatsby-node.js file with the following content:

const PnpWebpackPlugin = require(`pnp-webpack-plugin`);

module.exports = {
  onCreateWebpackConfig: ({actions}) => {
    actions.setWebpackConfig({
      resolveLoader: {
        plugins: [
          PnpWebpackPlugin.moduleLoader(module),
        ],
      },
      resolve: {
        plugins: [
          PnpWebpackPlugin.bind(`${__dirname}/.cache`, module, `gatsby`),
          PnpWebpackPlugin.bind(`${__dirname}/public`, module, `gatsby`),
          PnpWebpackPlugin,
        ],
      },
    });
  },
};

Gatsby generates a cache folder inside the project directory and, more annoyingly, uses it to store JS files that aren't compiled yet. This is problematic because since PnP is not aware that this cache folder belongs to the gatsby package it doesn't allow the files it contains to require the dependencies from its source package.

Q: Why is this folder needed? Why can't Gatsby just reference the files directly from their location within the gatsby package? Why do they need to be copied?

I fixed that by adding a new feature in the pnp-webpack-plugin package. It makes it possible to map a location on the disk to a specific locator (in this case the gatsby package), which solves the problem.

A similar issue happens with the public folder, which contains generated files from the gatsby package. Unfortunately those dependencies are loaded by Node, so the webpack plugin doesn't help here.

I fixed this issue simply by adding the referenced dependencies (core-js, lodash, react, and react-dom) in my own package.json, along with gatsby itself. Since there isn't a lot of them it's relatively manageable - unfortunately it will have to be part of the regular usage directives when using Gatsby with PnP.

The ESLint problem occurred.

I fixed this by adding gatsby in the list of allowed fallbacks. It's not great, but better than failing. We'll be able to deprecate this behavior starting from ESLint 6.

The Gatsby plugin loader loads plugin using require.resolve, but doesn't use the right paths option. That causes the PnP loader to reject the call because the plugins aren't listed in Gatsby's dependencies.

Not solved yet.

---

So far with the changes described above it seems to work (I can access the website at localhost:8000). I now have to figure out how to add themes and such and see whether other problems arise. I've put my current progress in the gatsby branch, feedback welcome!

To test:

$> git clone <this repository>
$> cd packages/website
$> ../packages/berry-cli/bin/berry gatsby develop