Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermittent/inconsistent build/develop errors #11109

Open
patricksimonian opened this issue Jan 16, 2019 · 19 comments

Comments

@patricksimonian
Copy link

@patricksimonian patricksimonian commented Jan 16, 2019

Firstly really loving Gatsby, it's awesome!

Description

There a 2 different error messages I have been getting on my local development/production builds which appear to be intermittent and inconsistent. Firstly, if there is a preference to open an issue for each separate one I will be happy to do so. Just let me know:)

Steps to reproduce

Unfortunately I have no clue how to reproduce these issues for Error 1 and Error 2 , they seem to occur between code changes and re running gatsby develop or gatsby build

Here is the gatsby project for reference if need be.

Errors

On gatsby develop or gatsby build these errors may occur from time to time.

Error 1

  TypeError: Cannot read property 'children' of undefined

  - actions.js:50 children.concat.children.map.child
    [app-web]/[gatsby]/dist/redux/actions.js:50:39

  - Array.map

  - actions.js:49 findChildrenRecursively
    [app-web]/[gatsby]/dist/redux/actions.js:49:42

  - actions.js:301 actions.deleteNode.args
    [app-web]/[gatsby]/dist/redux/actions.js:301:29

  - redux.js:468
    [app-web]/[gatsby]/[redux]/lib/redux.js:468:35

  - source-nodes.js:77 staleNodes.forEach.node
    [app-web]/[gatsby]/dist/utils/source-nodes.js:77:34

  - Array.forEach

  - source-nodes.js:77
    [app-web]/[gatsby]/dist/utils/source-nodes.js:77:18

  - Generator.next

  - util.js:16 tryCatcher
    [app-web]/[bluebird]/js/release/util.js:16:23

  - promise.js:512 Promise._settlePromiseFromHandler
    [app-web]/[bluebird]/js/release/promise.js:512:31

  - promise.js:569 Promise._settlePromise
    [app-web]/[bluebird]/js/release/promise.js:569:18

  - promise.js:614 Promise._settlePromise0
    [app-web]/[bluebird]/js/release/promise.js:614:10

  - promise.js:694 Promise._settlePromises
    [app-web]/[bluebird]/js/release/promise.js:694:18

  - async.js:138 _drainQueueStep
    [app-web]/[bluebird]/js/release/async.js:138:12

  - async.js:131 _drainQueue
    [app-web]/[bluebird]/js/release/async.js:131:9

  - async.js:147 Async._drainQueues
    [app-web]/[bluebird]/js/release/async.js:147:5

  - async.js:17 Immediate.Async.drainQueues
    [app-web]/[bluebird]/js/release/async.js:17:14

Work Around

Deleting the .cache folder and rebuilding/ rerunning dev appears to fix this issue.

Error 2

error UNHANDLED REJECTION


  TypeError: Cannot read property 'internal' of undefined

  - build-node-types.js:67 _.flatMap.groupBy.node
    [app-web]/[gatsby]/dist/schema/build-node-types.js:67:84

  - lodash.js:497 arrayAggregator
    [app-web]/[lodash]/lodash.js:497:34

  - lodash.js:4829 Function.groupBy
    [app-web]/[lodash]/lodash.js:4829:16

  - lodash.js:4374
    [app-web]/[lodash]/lodash.js:4374:28

  - lodash.js:683 arrayReduce
    [app-web]/[lodash]/lodash.js:683:21

  - lodash.js:4373 baseWrapperValue
    [app-web]/[lodash]/lodash.js:4373:14

  - lodash.js:9052 LodashWrapper.wrapperValue
    [app-web]/[lodash]/lodash.js:9052:14

  - build-node-types.js:67 groupChildNodesByType
    [app-web]/[gatsby]/dist/schema/build-node-types.js:67:140

  - build-node-types.js:123 inferChildFields
    [app-web]/[gatsby]/dist/schema/build-node-types.js:123:28

  - build-node-types.js:137 inferFields
    [app-web]/[gatsby]/dist/schema/build-node-types.js:137:23

  - build-node-types.js:187 fields
    [app-web]/[gatsby]/dist/schema/build-node-types.js:187:19

  - Array.forEach

  - Array.forEach

Work Around

Again deleting the .cache folder and running the development server or rebuilding fixes this issue.

Expected result

To be able to run the development server consistently and or run a build

Actual result

Not those things
What happened.
Things broke :(

Environment

We run our app on openshift in production. For local dev, we run on docker which is pretty close to the openshift image.

local development

System:
    OS: Linux 4.9 Debian GNU/Linux 8 (jessie) 8 (jessie)
    CPU: (6) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Shell: 4.3.30 - /bin/bash
  Binaries:
    Node: 8.12.0 - /usr/local/bin/node
    Yarn: 1.9.4 - /usr/local/bin/yarn
    npm: 6.4.1 - /usr/local/bin/npm
  Languages:
    Python: 2.7.9 - /usr/bin/python
  npmPackages:
    gatsby: ^2.0.64 => 2.0.64
    gatsby-plugin-emotion: ^3.0.1 => 3.0.1
    gatsby-plugin-favicon: ^3.1.4 => 3.1.4
    gatsby-plugin-google-analytics: ^2.0.8 => 2.0.8
    gatsby-plugin-lunr: ^1.3.0 => 1.3.0
    gatsby-plugin-offline: ^2.0.18 => 2.0.19
    gatsby-plugin-postcss: ^2.0.2 => 2.0.2
    gatsby-plugin-react-helmet: ^3.0.4 => 3.0.4
    gatsby-plugin-sharp: ^2.0.14 => 2.0.15
    gatsby-plugin-styled-components: ^3.0.4 => 3.0.4
    gatsby-remark-autolink-headers: ^2.0.12 => 2.0.12
    gatsby-remark-images: ^3.0.1 => 3.0.1
    gatsby-remark-prismjs: ^3.1.4 => 3.1.4
    gatsby-source-filesystem: ^2.0.11 => 2.0.11
    gatsby-source-github-api: 0.0.3 => 0.0.3
    gatsby-transformer-json: ^2.1.6 => 2.1.6
    gatsby-transformer-remark: ^2.1.15 => 2.1.15
    gatsby-transformer-sharp: ^2.1.9 => 2.1.9
    gatsby-transformer-yaml: ^2.1.6 => 2.1.6
  npmGlobalPackages:
    gatsby-cli: 2.4.8

Even running on my local machine i still get the same issue

System:
    OS: macOS 10.14.2
    CPU: x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 8.12.0 - /usr/local/bin/node
    npm: 6.5.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 71.0.3578.98
    Safari: 12.0.2
  npmPackages:
    gatsby: ^2.0.64 => 2.0.91
    gatsby-plugin-emotion: ^3.0.1 => 3.0.1
    gatsby-plugin-favicon: ^3.1.4 => 3.1.5
    gatsby-plugin-google-analytics: ^2.0.8 => 2.0.9
    gatsby-plugin-lunr: ^1.3.0 => 1.3.0
    gatsby-plugin-offline: ^2.0.18 => 2.0.21
    gatsby-plugin-postcss: ^2.0.2 => 2.0.2
    gatsby-plugin-react-helmet: ^3.0.4 => 3.0.5
    gatsby-plugin-sharp: ^2.0.14 => 2.0.17
    gatsby-plugin-styled-components: ^3.0.4 => 3.0.4
    gatsby-remark-autolink-headers: ^2.0.12 => 2.0.12
    gatsby-remark-images: ^3.0.1 => 3.0.1
    gatsby-remark-prismjs: ^3.1.4 => 3.2.0
    gatsby-source-filesystem: ^2.0.11 => 2.0.16
    gatsby-source-github-api: 0.0.3 => 0.0.3
    gatsby-transformer-json: ^2.1.6 => 2.1.7
    gatsby-transformer-remark: ^2.1.15 => 2.2.0
    gatsby-transformer-sharp: ^2.1.9 => 2.1.10
    gatsby-transformer-yaml: ^2.1.6 => 2.1.7
  npmGlobalPackages:
    gatsby-cli: 2.4.1
    gatsby-remark-code-titles: 1.0.2

I'd think that this would have to be a caching issue somewhere. I know for a fact in production, in our openshift environment we create a new image for each PR (as well as any new code changes pushed to the PR) and so the call to gatsby build in production never fails. I don't claim to be an expert, it could very well be something in our codebase but I don't think I am modifying any of the Gatsby build/develop objects in anyway.

And again to clarify this issue has been only happening in development when running gatsby build or gatsby develop

@polarathene

This comment has been minimized.

Copy link
Contributor

@polarathene polarathene commented Jan 17, 2019

Are you using remote assets at all? I've been using docker for development with Gatsby and get similar errors, these tend to be due to my poor/unstable connection in China+VPN where downloading my remotely sourced images can fail(either no image at all, or partially downloaded image(which avoids the error).

In my code, , I'm using the graphql data result in my props, if the image failed to download, it's childImageSharp data is empty/null, thus errors occur, I'm able to avoid it by filtering these out prior, though obviously not ideal.

Could it be that you're experiencing an issue similar to me with local development due to connection failing to fetch some remote assets? (I deploy to production with netlify which works fine) If so, then perhaps Gatsby needs to throw some sort of error message in CLI output about these processing/query failures upfront(and perhaps try again). For my data on a project it can take about 10 minutes or longer to reprocess again just because of a failure like this(prior to my filter hack to avoid the issue in development).

@skrapkam

This comment has been minimized.

Copy link

@skrapkam skrapkam commented Jan 17, 2019

I'm getting error 2 as well! Just started randomly happening...

@patricksimonian

This comment has been minimized.

Copy link
Author

@patricksimonian patricksimonian commented Jan 17, 2019

Are you using remote assets at all? I've been using docker for development with Gatsby and get similar errors, these tend to be due to my poor/unstable connection in China+VPN where downloading my remotely sourced images can fail(either no image at all, or partially downloaded image(which avoids the error).

In my code, , I'm using the graphql data result in my props, if the image failed to download, it's childImageSharp data is empty/null, thus errors occur, I'm able to avoid it by filtering these out prior, though obviously not ideal.

Could it be that you're experiencing an issue similar to me with local development due to connection failing to fetch some remote assets? (I deploy to production with netlify which works fine) If so, then perhaps Gatsby needs to throw some sort of error message in CLI output about these processing/query failures upfront(and perhaps try again). For my data on a project it can take about 10 minutes or longer to reprocess again just because of a failure like this(prior to my filter hack to avoid the issue in development).

I am using remotely sourced assets via a locally authored source plugin, however these two errors quit gatsby at build time and not run time. They both appear to be happening during node creation. Which comes to mind... perhaps I should try to reproduce the results while not running my local source plugin I'll come back with my results on that.

@skrapkam

This comment has been minimized.

Copy link

@skrapkam skrapkam commented Jan 17, 2019

I am using remotely sourced assets via a locally authored source plugin, however these two errors quit gatsby at build time and not run time. They both appear to be happening during node creation. Which comes to mind... perhaps I should try to reproduce the results while not running my local source plugin I'll come back with my results on that.

Disabling my locally source plugin seems to resolve the error

@patricksimonian

This comment has been minimized.

Copy link
Author

@patricksimonian patricksimonian commented Jan 17, 2019

I am using remotely sourced assets via a locally authored source plugin, however these two errors quit gatsby at build time and not run time. They both appear to be happening during node creation. Which comes to mind... perhaps I should try to reproduce the results while not running my local source plugin I'll come back with my results on that.

Disabling my locally source plugin seems to resolve the error

can you provide a link to your plugin so i can check it out? Maybe I'm doing too much magic in mine...

@skrapkam

This comment has been minimized.

Copy link

@skrapkam skrapkam commented Jan 17, 2019

I am using remotely sourced assets via a locally authored source plugin, however these two errors quit gatsby at build time and not run time. They both appear to be happening during node creation. Which comes to mind... perhaps I should try to reproduce the results while not running my local source plugin I'll come back with my results on that.

Disabling my locally source plugin seems to resolve the error

can you provide a link to your plugin so i can check it out? Maybe I'm doing too much magic in mine...

Yeah, here's the plugin.

@patricksimonian

This comment has been minimized.

Copy link
Author

@patricksimonian patricksimonian commented Jan 17, 2019

took a quick look. The only suspect i'd see from your code is https://github.com/MantasMikal/gatsby-source-instagram-all/blob/master/src/sourceNodes.js#L13

where you delete a property from a configuration object. Other than that it looks pretty normal. My local plugin is quite the behemoth but at the end of the day it does the same thing as all source plugins which is generate nodes by calling the createNode method.

@polarathene

This comment has been minimized.

Copy link
Contributor

@polarathene polarathene commented Jan 18, 2019

I think I have a similar plugin to what you two are referring as local plugin? I added offline/local support to gatsby-source-contentful via createRemoteNode(), iirc I'd get some build errors sometimes that were fixed by clearing out the cache.

@patricksimonian

This comment has been minimized.

Copy link
Author

@patricksimonian patricksimonian commented Jan 22, 2019

yeah that's my solution to my build errors.

@skrapkam

This comment has been minimized.

Copy link

@skrapkam skrapkam commented Jan 30, 2019

Any updates or ideas on how to fix this?

@patricksimonian

This comment has been minimized.

Copy link
Author

@patricksimonian patricksimonian commented Feb 15, 2019

Nothing so far, i still get it inconsistently. One thought is that it is perhaps tied to switching branches in git. The .cache folder is ignored and so perhaps mixing those up is causing this issue. As i mentioned before, however, my work around of deleting the cache has been working great so far.

@danoc danoc closed this Feb 15, 2019
@danoc danoc reopened this Feb 15, 2019
@danoc

This comment has been minimized.

Copy link
Contributor

@danoc danoc commented Feb 15, 2019

(Closed by accident—Sorry!)

@danoc

This comment has been minimized.

Copy link
Contributor

@danoc danoc commented Feb 15, 2019

Hi! I'm also running into this issue.

In my case it fails when calling findChildrenRecursively (actions.js) on the children of this node:

{ id: 'e7b3d69d-4834-54d1-9edb-52c18ea6a40c',
  parent: '1671edea-3cd1-5d88-ac40-e58084c964ef',
  children: [ '2cbff358-3084-58b5-98cd-32480f02c8f2' ],
  text: 'Text that describes the current status and is only visible to\nscreenreaders.',
  internal:
   { type: 'ComponentDescription',
     mediaType: 'text/markdown',
     content: 'Text that describes the current status and is only visible to\nscreenreaders.',
     contentDigest: 'f6d656b03e614d805f851c70135e935c',
     owner: 'gatsby-transformer-react-docgen' } }

The children of that node is from gatsby-mdx.

{ id: '2cbff358-3084-58b5-98cd-32480f02c8f2',
  children: [],
  parent: 'e7b3d69d-4834-54d1-9edb-52c18ea6a40c',
  internal:
   { content: 'Text that describes the current status and is only visible to\nscreenreaders.',
     type: 'Mdx',
     contentDigest: '8b6d3163f67bce5882615be6d2b1aaf2',
     owner: 'gatsby-mdx' },
  frontmatter: { title: '', _PARENT: 'e7b3d69d-4834-54d1-9edb-52c18ea6a40c' },
  exports: {},
  rawBody: 'Text that describes the current status and is only visible to\nscreenreaders.' }

Gatsby fails on getNode(child).children on the gatsby-mdx node (2cbff358-3084-58b5-98cd-32480f02c8f2) because getNode(child) is undefined. It is undefined because it is deleted from the Redux store.

I'm not sure if this is normal, but I noticed that deleteNode (actions.js) and getNode (nodes.js) are called a bunch of times on that same node:

success open and validate gatsby-configs — 0.154 s
success load plugins — 0.180 s
success onPreInit — 0.384 s
success delete html and css files from previous builds — 0.034 s
success initialize cache — 0.004 s
success copy gatsby files — 0.103 s
success onPreBootstrap — 0.013 s
⠠ source and transform nodes
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
error Cannot read property 'children' of undefined

  TypeError: Cannot read property 'children' of undefined

  - actions.js:57 children.concat.children.map.child
    [thumbprint]/[gatsby]/dist/redux/actions.js:57:47

  - Array.map

  - actions.js:52 findChildrenRecursively
    [thumbprint]/[gatsby]/dist/redux/actions.js:52:21

  - actions.js:338 actions.deleteNode.args
    [thumbprint]/[gatsby]/dist/redux/actions.js:338:33

Can't debug any longer and will do rm -rf .cache to workaround. I've seen this bug pop up a few times (over the course of months) although I'm not sure how to reliably reproduce.

@danoc

This comment has been minimized.

Copy link
Contributor

@danoc danoc commented Feb 19, 2019

Was able to reproduce this again today. I'm not 100% sure about this, but it seems to happen after I go a certain amount of time (over a day?) without working in the codebase.

Do nodes have some sort of time-based expiration that causes Gatsby to mark them as "stale"?

Never-mind, happened again after just a few minutes of not using the repo. 🤷‍♂️

@gatsbot

This comment has been minimized.

Copy link

@gatsbot gatsbot bot commented Mar 13, 2019

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

@gvocale

This comment has been minimized.

Copy link

@gvocale gvocale commented Mar 22, 2019

I ran into the same issue as well. I did delete a folder within /pages (a page and a css file), while gatsby develop was running. When I restarted gatbsy develop, I got TypeError: Cannot read property 'children' of undefined. Deleting rm -rf .cache and restarting gatsby develop solved the issue.

@polarathene polarathene added not stale and removed stale? labels Mar 23, 2019
@cbetta

This comment has been minimized.

Copy link

@cbetta cbetta commented May 5, 2019

I keep running into this issue as well. No matter of debugging has gotten me anywhere. It seems related to trying to create a child-parent link on something that's undefined, but no amount of guarding seems to resolve the issue.

@rikakomoe

This comment has been minimized.

Copy link

@rikakomoe rikakomoe commented Aug 21, 2019

I have this issue too. Deleting rm -rf .cache and restarting gatsby develop solved the issue. But almost every time I restarted the dev server I ran into this issue.

@wardpeet

This comment has been minimized.

Copy link
Member

@wardpeet wardpeet commented Sep 4, 2019

Hi!

Sorry to hear you're running into an issue. To help us best begin debugging the underlying cause, it is incredibly helpful if you're able to create a minimal reproduction. This is a simplified example of the issue that makes it clear and obvious what the issue is and how we can begin to debug it.

If you're up for it, we'd very much appreciate if you could provide a minimal reproduction and we'll be able to take another look.

I wasn't able to run the project listed in first comment.

Thanks for using Gatsby! 💜

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.