Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(gatsby-source-contentful): fixed id collision in contentful entries #23514

Merged

Conversation

me4502
Copy link
Contributor

@me4502 me4502 commented Apr 27, 2020

Description

This resolves an ID collision in the contentful source plugin, as noted here: #22004 (comment)

Related Issues

Related to #22004

@me4502 me4502 requested a review from vladar April 27, 2020 01:53
@me4502 me4502 requested a review from a team as a code owner April 27, 2020 01:53
@me4502
Copy link
Contributor Author

me4502 commented Apr 27, 2020

If anyone has any ideas as to why those unit tests fail that'd be helpful thanks.

Nothing appears to be wrong in the tests, and the actual code functions correctly.

@vladar
Copy link
Contributor

vladar commented Apr 27, 2020

Failures do seem unrelated. Maybe try merging master to this branch?

@me4502 me4502 force-pushed the fix/contentful-id-collision branch from 08a28a5 to 3995cd0 Compare April 28, 2020 00:37
@me4502
Copy link
Contributor Author

me4502 commented Apr 28, 2020

Seems it still fails, with an unrelated linting issue now too

@wardpeet
Copy link
Contributor

If you enable us to commit to the branch we could try to run our formatter. Currently you have to do it manually by running yarn format

@me4502 me4502 force-pushed the fix/contentful-id-collision branch from 3995cd0 to e80d55a Compare April 29, 2020 00:11
@me4502
Copy link
Contributor Author

me4502 commented Apr 29, 2020

The checkbox to allow committing to the branch is missing so I can't enable it sorry - I could move this to a branch on the official Gatsby project if that'd make it easier.

I've just made some changes that fix the test issues within this PR, and also just fix ones that are in master so that it passes

@me4502 me4502 force-pushed the fix/contentful-id-collision branch from 00ab480 to 81a1957 Compare May 1, 2020 03:52
Copy link
Contributor

@vladar vladar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job and thanks for the PR!

I've added a couple of comments inline. Also, I think we need tests for this use case because there are certainly edge cases (sys.linkType vs sys.type; deleted entries; also I see foreignReferenceMap uses sys.id as a key which can possibly break with identical ids but different types).

@me4502 me4502 force-pushed the fix/contentful-id-collision branch from 81a1957 to 4a9b0d9 Compare May 11, 2020 00:38
@me4502
Copy link
Contributor Author

me4502 commented May 11, 2020

I've just fixed the issues noted

@ascorbic ascorbic added topic: source-contentful Related to Gatsby's integration with Contentful status: needs core review Currently awaiting review from Core team member labels May 12, 2020
Copy link
Contributor

@vladar vladar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, Thanks!

But I need to test it locally before merging. So it will take some time as I am busy with something else at the moment.

Also, it would be great to add some tests for this use-case (identical ids in different types)

Copy link
Collaborator

@axe312ger axe312ger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be rebased on #15084

@wardpeet
Copy link
Contributor

@axe312ger Should we wait for #15084 to get this in? I feel this one is an easier merge than #15084

@wardpeet wardpeet self-assigned this Jun 16, 2020
@axe312ger
Copy link
Collaborator

axe312ger commented Jun 17, 2020

@wardpeet Yes lets make progress and get this merged! I'll rebase #15084 then 😅

@vladar
Copy link
Contributor

vladar commented Jun 17, 2020

I think both PRs are potentially breaking so it might make sense to include them both in a new major version of the plugin? What do you think?

@axe312ger
Copy link
Collaborator

@vladar yes! Totally down for this!

@axe312ger axe312ger added the breaking change If implemented, this proposed work would break functionality for older versions of Gatsby label Jun 23, 2020
@axe312ger axe312ger changed the base branch from master to contentful-next June 23, 2020 11:10
@axe312ger axe312ger merged this pull request into gatsbyjs:contentful-next Jun 23, 2020
axe312ger pushed a commit that referenced this pull request Jun 23, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
@me4502 me4502 deleted the fix/contentful-id-collision branch June 24, 2020 22:12
axe312ger pushed a commit that referenced this pull request Jun 30, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit that referenced this pull request Jul 8, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit that referenced this pull request Jul 8, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit that referenced this pull request Jul 17, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit that referenced this pull request Jul 17, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit that referenced this pull request Jul 29, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit that referenced this pull request Jul 29, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit that referenced this pull request Aug 5, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit that referenced this pull request Aug 5, 2020
…es (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit to axe312ger/gatsby that referenced this pull request Oct 2, 2020
…es (gatsbyjs#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit to axe312ger/gatsby that referenced this pull request Oct 2, 2020
…es (gatsbyjs#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit to axe312ger/gatsby that referenced this pull request Oct 2, 2020
…es (gatsbyjs#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
axe312ger pushed a commit to axe312ger/gatsby that referenced this pull request Oct 2, 2020
…es (gatsbyjs#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries
pvdz pushed a commit that referenced this pull request Oct 5, 2020
…e performance (#27244)

* fix(gatsby-source-contentful): fixed id collision in contentful entries (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries

* fix(gatsby-source-contentful): properly resolve subsequent sync calls   (#15084)

* fix(gatsby-source-contentful): fixed id collision in contentful entries (#23514)

* fix(gatsby-source-contentful): fixed id collision in contentful entries

* improve logging and clean up code (#26238)

* improve comment about contentful-resolve-response usage

* test(unit): remove cross test data pollution

* further improve loggin by replacing console.time with GatsbyReporter

* improve code style

* missing file

* remove logging verbosity

* adjust mocked reporter for tests to pass

* remove unused deep-map dependency

* fix: use content type id by default to create reference gql types (#26102)

* temporary console.time calls for performance measurements

* perf(gatsby-source-contentful): use a map and less loops to merge old and latest sync API response

* log time Contentful API processing  and GraphQL node creation

* upgrade dependencies and use faster node 10 versions

* remove duplicate logging

* allow multiple Contentful sources

fixes #26875

* refactor(imports): fix dependency structure and make sharp optional

fixes #23904

* align tests to match new multi source cache pattern

Co-authored-by: Matthew Miller <mnmiller1@me.com>
Co-authored-by: Khaled Garbaya <khaled@contentful.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change If implemented, this proposed work would break functionality for older versions of Gatsby status: needs core review Currently awaiting review from Core team member topic: source-contentful Related to Gatsby's integration with Contentful
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants