Parallel query running fails when querying JSON with custom type name

[jabrks]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

Parallel query running is failing on my site whilst trying to query the child JSON field of a file node. This only seems to happen when the type name option is set for gatsby-transformer-json

Reproduction Link

https://codesandbox.io/s/gatsby-json-pqr-wzdv5

Steps to Reproduce

Run gatsby clean followed by gatsby build

Expected Result

The build completes successfully

Actual Result

The build fails with the following errors

ERROR #85923  GRAPHQL

There was an error in your GraphQL query:

Cannot query field "childJson" on type "File".

If you don't expect "childJson" to exist on the type "File" it is most likely a typo.
However, if you expect "childJson" to exist there are a couple of solutions to common problems:

- If you added a new data source and/or changed something inside gatsby-node.js/gatsby-config.js, please try a restart of your development server
- The field might be accessible in another subfield, please try your query in GraphiQL and use the GraphiQL explorer to see which fields you can query and what shape they have
- You want to optionally use your field "childJson" and right now it is not used anywhere. Therefore Gatsby can't infer the type and add it to the GraphQL schema. A quick fix is to add at least one entry with that field ("dummy content")

It is recommended to explicitly type your GraphQL schema if you want to use optional fields. This way you don't have to add the mentioned "dummy content". Visit our docs to learn how you can define the schema for "File": https://www.gatsbyjs.com/docs/reference/graphql-data-layer/schema-customization#creating-type-definitions

ERROR #85928

An error occurred during parallel query running.
Go here for troubleshooting tips: https://gatsby.dev/pqr-feedback



  Error: Worker exited before finishing task

  - index.js:113 ChildProcess.<anonymous>
    [sandbox]/[gatsby-worker]/dist/index.js:113:45

  - child_process.js:282 Process.ChildProcess._handle.onexit
    internal/child_process.js:282:12

Environment

System:
  OS: macOS 11.6.2
  CPU: (12) x64 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
  Shell: 5.8 - /bin/zsh
Binaries:
  Node: 16.13.0 - ~/.nvm/versions/node/v16.13.0/bin/node
  Yarn: 1.22.17 - ~/.nvm/versions/node/v16.13.0/bin/yarn
  npm: 8.3.0 - ~/.nvm/versions/node/v16.13.0/bin/npm
Languages:
  Python: 2.7.16 - /usr/bin/python
Browsers:
  Chrome: 97.0.4692.99
  Firefox: 96.0.1
  Safari: 15.2
npmPackages:
  gatsby: ^4.6.0 => 4.6.0 
  gatsby-plugin-amp: ^0.2.2 => 0.2.2 
  gatsby-plugin-eslint: ^4.0.2 => 4.0.2 
  gatsby-plugin-feed: ^4.6.0 => 4.6.0 
  gatsby-plugin-google-tagmanager: ^4.6.0 => 4.6.0 
  gatsby-plugin-image: ^2.6.0 => 2.6.0 
  gatsby-plugin-loadable-components-ssr: ^4.1.1 => 4.1.1 
  gatsby-plugin-react-helmet: ^5.6.0 => 5.6.0 
  gatsby-plugin-react-svg: ^3.1.0 => 3.1.0 
  gatsby-plugin-sass: ^5.6.0 => 5.6.0 
  gatsby-plugin-sharp: ^4.6.0 => 4.6.0 
  gatsby-plugin-sitemap: ^5.6.0 => 5.6.0 
  gatsby-plugin-twitter: ^4.6.0 => 4.6.0 
  gatsby-plugin-webpack-bundle-analyser-v2: ^1.1.26 => 1.1.26 
  gatsby-remark-copy-linked-files: ^5.6.0 => 5.6.0 
  gatsby-source-filesystem: ^4.6.0 => 4.6.0 
  gatsby-transformer-json: ^4.6.0 => 4.6.0 
  gatsby-transformer-remark: ^5.6.0 => 5.6.0 
  gatsby-transformer-sharp: ^4.6.0 => 4.6.0 
  gatsby-transformer-yaml: ^4.6.0 => 4.6.0 
npmGlobalPackages:
  gatsby-cli: 4.6.0

Config Flags

No response

[jens-duttke]

We have the same problem running Gatby 4 in a GitHub action. Locally on Windows it was only reproduceable once, in all our other tests it worked. I assume it's a race-condition.

[LekoArts]

Hi, thanks for the issue!

Downloading your codesandbox as a zip, running it locally I also see this. At the moment I sadly don't have a hunch why that might happen. We saw this sometimes in our E2E tests, too, so I don't think this is specific to the JSON transformer.

[jabrks]

I spent some time investigating this over the last few days and I think I understand what's going on now.

During schema building, Gatsby takes a snapshot of the schema and saves it to the cache.

gatsby/packages/gatsby/src/schema/schema.js

Lines 156 to 163 in 711a96f

	if (enginePrintConfig) {
	// make sure to print schema that will be used when bundling graphql-engine
	await printTypeDefinitions({
	config: enginePrintConfig,
	schemaComposer,
	parentSpan: activity.span,
	})
	}

It then uses that snapshot to populate the schema that parallel queries are run against.

gatsby/packages/gatsby/src/utils/worker/child/schema.ts

Lines 24 to 33 in 711a96f

	const schemaSnapshotPath = path.join(
	workerStore.program.directory,
	`.cache`,
	`schema.gql`
	)
	if (await fs.pathExists(schemaSnapshotPath)) {
	const schemaSnapshot = await fs.readFile(schemaSnapshotPath, `utf-8`)
	store.dispatch(actions.createTypes(schemaSnapshot))
	}

The function that prints the schema includes a hardcoded list of internal types which are excluded from the output.

gatsby/packages/gatsby/src/schema/print.ts

Lines 344 to 359 in 711a96f

	const internalTypes = [
	`Boolean`,
	`Buffer`,
	`Date`,
	`Float`,
	`ID`,
	`Int`,
	`Internal`,
	`InternalInput`,
	`JSON`,
	`Json`,
	`Node`,
	`NodeInput`,
	`Query`,
	`String`,
	]

That list includes the Json type, and if I look at the snapshot from the reproduction I shared above then I can see that the following type is missing.

type Json implements Node @childOf(types: ["File"]) @dontInfer {
  apples: Int
  oranges: Int
  bananas: Int
}

If I use a different value for the type name option when configuring the transformer plugin then this all works as expected.

[pieh]

Opened #35098 with fix for Json named types. Thank you @jabrks for your research quickly pointing me to a cause!

[jabrks]

Thanks for taking a look @pieh!