Use cross-env and esm together for gatsby-develop on windows powershell #28

wesbos · 2020-09-09T16:47:50Z

Here is a tricky one I can't get working. I'm using es modules for my gatsby-config.js and gatsby-node.js files. Currently they only support common JS, so I am using the esm package to use import/export syntax.

The way we require esm is by setting NODE_OPTIONS="-r esm" env variable, which requires the esm package first before running gatsby develop. To get it working on windows, we use the cross-env package.

So the command looks like this:

"develop": "cross-env NODE_OPTIONS=\"-r esm\" gatsby develop",

Works great on a mac, but with windows powershell I get Error 10123 We encountered an error while trying to load your site's gatsby-config. Please fix the error and try again.. This only happens when there is a plugins array in gatsby-config.js. Taking it out solves the problem.

So to reproduce the error:

Make sure you are on Node >=14
Clone this repo
Go to finished-files/gatsby
run npm install
run npm run develop

If you see GraphQL / Sanity errors, that is fine and means it worked. Don't worry about the site building, I just want to get past the above error.

Not acceptable solutions:

Just use a .env file - that is introduced later in the course
Just use WSL. I need this to work on powershell
Just use docker - see above
LOL BUY A MAC

Ideally npm run develop will work on all platforms, but I'm okay with a npm run develop:win command.

Things I've tried:

Using set NODE_OPTIONS="-r esm" && gatsby develop - gatsby runs in a sub-process and it doesn't seem to transfer the variable down
using cross-env-shell - it didn't work but not sure if I'm using it correctly.

Happy to pay $250 USD to anyone who can get it running.

The text was updated successfully, but these errors were encountered:

Daniel-Griffiths · 2020-09-09T17:22:26Z

This is not a solution, but one thing to note is that that it tries to find a file that doesnt exist \node_modules\\gatsby-plugin-react-helmet\\gatsby-config

It's got me curious now too 😆

askkaz · 2020-09-09T17:24:38Z

yeah, the comments around similar issues mention that mac is more graceful at handling missing files than windows. I'm guessing there is some filename weirdness between windows/mac, but I'm still waiting for my old windows machine to boot/update

wesbos · 2020-09-09T17:25:57Z

yes! and if you take out that plugin, it just errors on the next plugin. I'm bettering esm is doing something weird with the require path lookup

Asjas · 2020-09-09T17:28:45Z

Removing the plugins in their entirety makes it boot up to the Gatsby GraphQL errors 👀

wesbos · 2020-09-09T17:29:48Z

Yeah that is noted above, if you get those graphql errors, it means it's working! So what about the plugins array is making it do that..

Daniel-Griffiths · 2020-09-09T17:35:19Z

For reference running

$env:NODE_OPTIONS='-r esm'; npm run develop

with the develop command set to

"develop": "gatsby develop",

Results in the same error as using cross env, so I don't think the issue lies with the cross env package.

wesbos · 2020-09-09T17:37:19Z

Here is the whole stack trace from the error:

 ERROR #10123  CONFIG

We encountered an error while trying to load your site's gatsby-config. Please fix the err
or and try again.



  Error: C:\Users\wes\Desktop\master-gatsby-master\finished-files\gatsby\node_modules\gats
  by\dist\bootstrap\get-config-file.js:1
  Error: Cannot find module 'C:\\Users\\wes\\Desktop\\master-gatsby-master\\finished-files
  \\gatsby\\node_modules\\gatsby-plugin-styled-components\\gatsby-config'
  Require stack:
  - C:\Users\wes\Desktop\master-gatsby-master\finished-files\gatsby\node_modules\gatsby\di
  st\bootstrap\get-config-file.js
  - C:\Users\wes\Desktop\master-gatsby-master\finished-files\gatsby\node_modules\gatsby\di
  st\bootstrap\index.js
  - C:\Users\wes\Desktop\master-gatsby-master\finished-files\gatsby\node_modules\gatsby\di
  st\commands\develop-process.js
  - C:\Users\wes\Desktop\master-gatsby-master\finished-files\gatsby\.cache\tmp-16336-XwaIQ
  L5eWhQb

Daniel-Griffiths · 2020-09-09T17:39:14Z

Not sure if it helps but I get a slightly longer stack trace

 ERROR #10123  CONFIG

We encountered an error while trying to load your site's gatsby-config. Please fix the error and try again.



  Error: C:\Users\Daniel\master-gatsby\finished-files\gatsby\node_modules\gatsby\dist\bootstrap\get-config-file.js:1
  Error: Cannot find module 'C:\\Users\\Daniel\\master-gatsby\\finished-files\\gatsby\\node_modules\\gatsby-plugin-react-helmet\\gatsby-config'
  Require stack:
  - C:\Users\Daniel\master-gatsby\finished-files\gatsby\node_modules\gatsby\dist\bootstrap\get-config-file.js
  - C:\Users\Daniel\master-gatsby\finished-files\gatsby\node_modules\gatsby\dist\bootstrap\index.js
  - C:\Users\Daniel\master-gatsby\finished-files\gatsby\node_modules\gatsby\dist\commands\develop-process.js
  - C:\Users\Daniel\master-gatsby\finished-files\gatsby\.cache\tmp-6348-M4LsCDK3VOYP

  - v8-compile-cache.js:162 Function.require.resolve
    [gatsby]/[v8-compile-cache]/v8-compile-cache.js:162:23

  - get-config-file.js:32 getConfigFile
    [gatsby]/[gatsby]/dist/bootstrap/get-config-file.js:32:30

  - index.js:72 resolveTheme
    [gatsby]/[gatsby]/dist/bootstrap/load-themes/index.js:72:46

  - index.js:146
    [gatsby]/[gatsby]/dist/bootstrap/load-themes/index.js:146:28

  - util.js:16 tryCatcher
    [gatsby]/[bluebird]/js/release/util.js:16:23

  - reduce.js:166 Object.gotValue
    [gatsby]/[bluebird]/js/release/reduce.js:166:18

  - reduce.js:155 Object.gotAccum
    [gatsby]/[bluebird]/js/release/reduce.js:155:25

  - util.js:16 Object.tryCatcher
    [gatsby]/[bluebird]/js/release/util.js:16:23

  - promise.js:547 Promise._settlePromiseFromHandler
    [gatsby]/[bluebird]/js/release/promise.js:547:31

  - promise.js:604 Promise._settlePromise
    [gatsby]/[bluebird]/js/release/promise.js:604:18

  - promise.js:641 Promise._settlePromiseCtx
    [gatsby]/[bluebird]/js/release/promise.js:641:10

  - async.js:97 _drainQueueStep
    [gatsby]/[bluebird]/js/release/async.js:97:12

  - async.js:86 _drainQueue
    [gatsby]/[bluebird]/js/release/async.js:86:9

  - async.js:102 Async._drainQueues
    [gatsby]/[bluebird]/js/release/async.js:102:5


not finished open and validate gatsby-configs - 0.077s
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! slicks-slices-frontend@0.1.0 develop: `cross-env NODE_OPTIONS="-r esm" gatsby develop`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the slicks-slices-frontend@0.1.0 develop script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Daniel\AppData\Roaming\npm-cache\_logs\2020-09-09T16_38_29_809Z-debug.log
PS C:\Users\Daniel\master-gatsby\finished-files\gatsby>

Asjas · 2020-09-09T17:41:13Z

@Daniel-Griffiths It's the same stack-trace 🙂 - Wes just shortened it when he posted it here

Daniel-Griffiths · 2020-09-09T17:41:25Z

Woops 😆

louisjrdev · 2020-09-09T17:45:59Z

Looks like this might be the issue, might need a PR to gatsby.
nodejs/node#31710

Daniel-Griffiths · 2020-09-09T17:48:49Z

@louisjrdev Yeah I noticed the same thing, the closest thing I have gotten this to work is to touch a .env and remove it later. But I know that doesnt meet the requirements 🤔

wesbos · 2020-09-09T17:49:45Z

does putting it in a .env make it run?

louisjrdev · 2020-09-09T17:51:47Z

See also: gatsbyjs/gatsby#23705

Daniel-Griffiths · 2020-09-09T17:54:21Z

@wesbos It changes the error

 ERROR #10123  CONFIG

We encountered an error while trying to load your site's gatsby-config. Please fix the error and try again.



  Error: C:\Users\Daniel\master-gatsby\finished-files\gatsby\gatsby-config.js:1
  (function (exports, require, module, __filename, __dirname) { import dotenv from 'dotenv';
                                                                                                    ^^^^^^
  SyntaxError: Cannot use import statement outside a module

Looks the error shifts from gatbsy config and it looks like esm is not even enabled when using a .env file. Hmm.

louisjrdev · 2020-09-09T17:54:43Z

I think it's less a "make ESM work with gatsby issue" than it is a "gatsby doesn't support esm fully yet".

wesbos · 2020-09-09T17:55:22Z

That error means -r esm isn't being used at all

@louisjrdev to be clear, this is using the esm package, not the built in node.js esm experimental esm

Daniel-Griffiths · 2020-09-09T17:55:55Z

@louisjrdev I am sure I got it working in the past by using Typescript but that's waaay out of scope for this 😅

wesbos · 2020-09-09T17:57:56Z

Someone on twitter mentioned it works if you create an empty gatsby-config.js file in each plugin.

So - why is gatsby looking for a config file in the plugin that doesn't exist?

ejdigby · 2020-09-09T18:00:30Z

FYI you also get to the same point as @Daniel-Griffiths by replacing NODE_OPTIONS=\"-r esm\" with NODE_OPTIONS=$(\"-r esm\").

Potential gets you slightly further without having to use a .env file

wesbos · 2020-09-09T18:02:23Z

That point isn't any further, I think that method doesn't expose the NODE_OPTIONS value to the child process and thus esm package is completely ignored.

ejdigby · 2020-09-09T18:05:35Z

So if you run the following you can see (I think) that the child process does get the env variable, its just a problem with how handled after that:

test.js

console.log(process.env.NODE_OPTIONS)

package.json

"develop": "cross-env NODE_OPTIONS=\"-r esm\" node test.js",

output

-r esm

Apologies if it doesn't show what I think it shows

Daniel-Griffiths · 2020-09-09T18:05:44Z

Ok I got it working but the solution is a little nasty. Gimme a sec to write the solution...

louisjrdev · 2020-09-09T18:09:25Z

Oooh I have it working too

Daniel-Griffiths · 2020-09-09T18:09:34Z

So in the path

node_modules/gatsby/dist/bootstrap/get-config-file.js

Go to line 40

And remove the code in that if statement so it looks like this:

Then I used patch-package to persists the fix: https://www.npmjs.com/package/patch-package

wesbos · 2020-09-09T18:10:40Z

Ah! I was just looking at this: gatsbyjs/gatsby#17008

I wonder if Gatsby can be patched then?

louisjrdev · 2020-09-09T18:11:48Z

@wesbos Okay so i think I have solved this, without modifying any node modules.

So in the gatsby-config.js make it:

module.exports = {
  pathPrefix: '/pizza',
  siteMetadata: {
    title: `Slicks Slices`,
    siteUrl: 'https://gatsby.pizza',
    description: 'The best pizza place in Hamilton!',
    twitter: '@slicksSlices',
  },
  plugins: [
    'gatsby-plugin-react-helmet',
    'gatsby-plugin-styled-components',
    {
      // this is the name of the plugin you are adding
      resolve: 'gatsby-source-sanity',
      options: {
        projectId: '0jfvvkkd',
        dataset: 'production',
        watchMode: true,
        token: process.env.SANITY_TOKEN,
      },
    },
  ],
};

Follow the instructions here:
gatsbyjs/gatsby#7810 (comment)

change the develop script in the package.json to just gatsby develop

louisjrdev · 2020-09-09T18:12:49Z

A bit hacky but it works, not sure if it does on mac / linux

wesbos · 2020-09-09T18:13:03Z

@louisjrdev thanks but that is removing all the ES Modules and just reverting it back to commonJS. So it gets rid of the problem, but also the feature :)

Daniel-Griffiths · 2020-09-09T18:15:33Z

Besides the above I can't find any other way to get this working. I think someone needs to submit a patch to Gatsby and find a real fix instead of deleting the problem 😆

patch-package has worked great for me in the past but I understand it's not the kind of thing you want to use on a course.

wesbos · 2020-09-09T18:16:05Z

Got it running based on @Daniel-Griffiths and this thread: gatsbyjs/gatsby#17008

If I crack open node_modules/gatsby/dist/utils/test-require-error.js and replace this:

const firstLine = err.toString().split(`\n`)[0];

with this:

const firstLine = err.toString().replace(/\\\\/g, '\\').split(`\n`)[0];

It builds! So it's an issue with gatsby.. Perhaps they would fancy a PR

Asjas · 2020-09-09T18:19:26Z

Yeah, If I do the same it works on Windows so I can confirm that seems to be the fix to build it, and I tested on MacOS and it's still working there as well after I change it there. 👍

Daniel-Griffiths · 2020-09-09T18:20:49Z

I will submit a PR to gatsby shortly, then we wait... 👀

wesbos · 2020-09-09T18:21:11Z

gatsbyjs/gatsby#26840 PR

Daniel-Griffiths · 2020-09-09T18:21:17Z

Beat me to it 😆

askkaz · 2020-09-09T18:24:56Z

its surprising that this isn't a more common issue, right?

wesbos · 2020-09-09T18:27:14Z

Yeah - very bizarre. It might have something to do with esm changing how paths reported? Not sure what it does under the hood. There were a few reports of it happening in gatsby, and none in esm though.

Daniel-Griffiths · 2020-09-09T18:30:36Z

@askkaz I guess most people end up sticking with commonjs as it's the default. 🤷‍♂️

I saw various articles and github threads trying to find a solution, but a lot of the fixes seem outdated and no longer work.

wesbos · 2020-09-10T19:15:29Z

Merged into gatsby. Thanks everyone! @Daniel-Griffiths shoot me an email and ill get you the $250 wesbos@gmail.com

Daniel-Griffiths · 2020-09-10T22:20:27Z

Awesome thanks @wesbos! 🙂

Will shoot you a email from [redacted]

wesbos changed the title ~~Use cross-env and esm together for gatsby-develop on windows powersell~~ Use cross-env and esm together for gatsby-develop on windows powershell Sep 9, 2020

wesbos mentioned this issue Sep 9, 2020

Cannot find gatsby-config.js gatsbyjs/gatsby#15565

Closed

wesbos mentioned this issue Sep 9, 2020

Allow for double slashes on windows gatsbyjs/gatsby#26840

Merged

wesbos closed this as completed Sep 10, 2020

Use cross-env and esm together for gatsby-develop on windows powershell #28

Use cross-env and esm together for gatsby-develop on windows powershell #28

Comments

wesbos commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020

askkaz commented Sep 9, 2020

wesbos commented Sep 9, 2020

Asjas commented Sep 9, 2020

wesbos commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020 • edited

wesbos commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020

Asjas commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020

louisjrdev commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020

wesbos commented Sep 9, 2020

louisjrdev commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020 • edited

louisjrdev commented Sep 9, 2020

wesbos commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020 • edited

wesbos commented Sep 9, 2020

ejdigby commented Sep 9, 2020

wesbos commented Sep 9, 2020

ejdigby commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020

louisjrdev commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020 • edited

wesbos commented Sep 9, 2020

louisjrdev commented Sep 9, 2020

louisjrdev commented Sep 9, 2020

wesbos commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020

wesbos commented Sep 9, 2020

Asjas commented Sep 9, 2020 • edited

Daniel-Griffiths commented Sep 9, 2020

wesbos commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020

askkaz commented Sep 9, 2020 • edited

wesbos commented Sep 9, 2020

Daniel-Griffiths commented Sep 9, 2020

wesbos commented Sep 10, 2020

Daniel-Griffiths commented Sep 10, 2020 • edited

Daniel-Griffiths commented Sep 9, 2020 •

edited

Daniel-Griffiths commented Sep 9, 2020 •

edited

Daniel-Griffiths commented Sep 9, 2020 •

edited

Daniel-Griffiths commented Sep 9, 2020 •

edited

Asjas commented Sep 9, 2020 •

edited

askkaz commented Sep 9, 2020 •

edited

Daniel-Griffiths commented Sep 10, 2020 •

edited