Direct-ID lookup for nodes in graphql fails in some instances

[me4502]

Description

When querying a single node by a direct-ID lookup, the query sometimes returns null. Querying for the whole list, and filtering by something such as slug finds the document fine, with the same ID.

Thanks to @DSchau for helping me debug this.

I had a feeling it was related to #20609, but after testing on a version from before that change (2.18.22), the issue still occurs.

Steps to reproduce

I have no idea how to reliably reproduce this sorry. It has randomly occurred on my project. It appears to just be the one node, on a page with >10k pages (so therefore a lot of nodes are in the db)

This query on the site shows it in a simple way, however.

Expected result

The contentfulBlogPost query should be able to find a node, as the above query has proven it exists and provided an ID.

Actual result

A direct-ID lookup for this object does not work. Looking up by other things, such as contentful_id, do work.

Environment

  System:
    OS: macOS 10.15.3
    CPU: (12) x64 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 12.14.1 - ~/.npm/bin/node
    Yarn: 1.22.0 - ~/.npm/bin/yarn
    npm: 6.13.6 - ~/.npm/bin/npm
  Languages:
    Python: 2.7.17 - /usr/local/bin/python
  Browsers:
    Firefox: 70.0.1
    Safari: 13.0.5

[vladar]

Can you clarify the "random" part? I mean - if you run this query twice in GraphiQL - do you get different results? Or maybe some action during the development session changes the behavior?

[me4502]

Sorry, by random I meant that I couldn’t find any reason why this one is the one affected.

It is always the same node, and it always fails

[vladar]

Is it possible to get access to your project so that we could take a look locally? I'll post an email in the next message (will delete later)

[me4502]

I've looked into it and I don't think we're able to provide access to the project sorry. It's in a monorepo with a few other intertwined projects, as well as it uses a few libraries from private NPM repos.

[vladar]

Hi @me4502

I understand but the problem is that it is nearly impossible to debug this without reproduction. To help us best begin debugging the underlying cause, it is incredibly helpful if you're able to create a minimal reproduction. This is a simplified example of the issue that makes it clear and obvious what the issue is and how we can begin to debug it.

Thanks for using Gatsby! 💜

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜

[me4502]

So we've had this occur again in a way that cannot be worked around.

Using contentful, rich text fields are a separate node, accessed through content___NODE. Direct ID lookups are failing for some of these, and therefore the content___NODE field comes up as null.

I understand this is almost impossible to debug without a minimal reproduction, however I have not been able to create one reliably. The best I can do is using the exact same data so that the node store is in the same state. However this is still not minimal and is very intertwined with systems that can't easily be shared.

I'm happy to debug and solve this, my main reason for asking again now is to work out if anyone on the Gatsby team has any ideas about what areas of the codebase could cause something like this. I've been looking into it for a few days and haven't been able to find a concrete cause.

[benrobertsonio]

@me4502 There are some known issues with Contentful's rich text feature, documented below, which might give some direction!

• fix(gatsby-source-contentful): fix missing data with rich text fields #15084
• Embedded content breaks Gatsby hot reloading contentful/rich-text#94

[me4502]

@benrobertsonio Those issues are referring to the partial sync the API does on update, we’re using the option to force a full sync to get around that issue.

This issue is outside of the Contentful plugin from what I can tell. The data is all there correctly in Gatsby’s node store, it just cannot be queried by ID from graphql.

[vladar]

I am curious if it is the same with in operator? i.e.

contentfulBlogPost(id: { in: ["your-mysterious-id"] }) {
  id
}

(at the moment this bypasses id indexes). Also worth trying regex by partial match. Just to collect a bit more data.

And just to clarify - you can't reproduce it on the same contentful space but with some simple starter (using GraphiQL to check the results)?

[me4502]

Using in or regex also fail, they don't find the node.

I haven't specifically tested with just the contentful space, as the issue doesn't only occur with contentful data. It happens to pretty much any type of node, just a very small percentage of them.

I will do a test with just that shortly.

[me4502]

I just confirmed that the same issue occurs with a fresh gatsby-default-starterwith /just/ gatsby-source-contentful setup for this space.

[vladar]

Sounds like a repro? Can you share it? As far as I understand we need a temporary content delivery token only to see it?

[me4502]

Thanks, I've just emailed the key and other required info to the email you posted earlier

[vladar]

Surprisingly, I can't reproduce it locally. Tried this query:

{
  allContentfulBlogPost(filter:{slug:{eq:"download-youtube-videos"}, node_locale:{eq:"en-US"}}) {
    nodes {
      id
      contentful_id
      title
    }
  }
  contentfulBlogPost(id:{eq:"cd0ea374-9b02-5fdd-8294-08fb642c741d"}) {
    id
    title
  }
}

And got the expected result:

{
  "data": {
    "allContentfulBlogPost": {
      "nodes": [
        {
          "id": "cd0ea374-9b02-5fdd-8294-08fb642c741d",
          "contentful_id": "blog-download-youtube-videos",
          "title": "How to Download YouTube Videos, Step-by-Step"
        }
      ]
    },
    "contentfulBlogPost": {
      "id": "cd0ea374-9b02-5fdd-8294-08fb642c741d",
      "title": "How to Download YouTube Videos, Step-by-Step"
    }
  }
}

The screenshot:

Maybe it is some other node now? Can you share your steps?

[me4502]

I'm on my phone so don't have the exact query, but I basically queried for allContentfulBlogPost, with a filter on content equalling null.

This gave me a list of all the ContentfulBlogPost nodes that had a content___NODE that was unable to be resolved

[vladar]

This gave me a list of all the ContentfulBlogPost nodes that had a content___NODE that was unable to be resolved

I see such nodes (where content___NODE is set but content field is null). But this seems to be different from your original issue description as those nodes are not in the store.

They are created and then deleted for some reason (all during sourceNodes step). So when querying they do not show up either in the list or individually.

But I do repro cases with content === null. Trying to figure out why those nodes are being deleted...

[vladar]

Did some digging. A rich text node is a child of a blog post node. And we have a weird sequence of calls in contentful plugin:

createNode(blogPostNode)
createNode(richTextNode)
createNode(blogPostNode)

So the same blog post node is created twice. The problem though that when we re-create existing node Gatsby deletes all of its children (assuming they will be re-created too).

But contentful plugin never re-creates them hence we get the blog post without text node. Still digging why contentful calls createNode twice for the same blog post node.

[vladar]

Ok, it looks like you have identical contentful_id (and hence node id) for blog posts AND related assets. As a result, we face a weird collision that causes this. In reality the sequence of calls is:

createNode(blogPostNode)
createNode(richTextNode)
createNode(blogPostAssetNode)

The blogPostAssetNode has the same id as blogPostNode, so Gatsby confuses it with blogPostNode and clears children of the blog post.

Ideally, ids should be unique for a single contentful space. But contentful source plugin could have namespaced it with content type to avoid collisions like this.

A PR is welcome - it shouldn't be too complex! Basically add type to this function (and adjust all usages):

gatsby/packages/gatsby-source-contentful/src/normalize.js

Lines 100 to 103 in c350f59

	const makeId = ({ spaceId, id, currentLocale, defaultLocale }) =>
	currentLocale === defaultLocale
	? `${spaceId}___${id}`
	: `${spaceId}___${id}___${currentLocale}`

[me4502]

Ah, thanks - I’ll submit a PR on Monday.

When testing this new issue I used getNode to see if it still existed and it retuned a result, which a conflicting ID would explain.

For the original issue, using just the Contentful data made it return different broken pages so I’ll try finding which ones are affected with just that data, thanks

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜