Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gatsby-source-graphcms] Add GraphCMS Plugin #1867

Closed
wants to merge 9 commits into from
Closed

[gatsby-source-graphcms] Add GraphCMS Plugin #1867

wants to merge 9 commits into from

Conversation

@Redmega
Copy link

@Redmega Redmega commented Aug 21, 2017

Adds the gatsby-source-graphcms plugin to fetch all data from a GraphCMS backend.

Need to experiment with more complex data types and relationships to see if they're supported as is or if changes are needed.

As per @jorishermans in Discord:

Let the user decide the mapping in the plugin config in gatsby-config.js
[...] I implemented a mapping feature in gatsby-source-mongodb
it would be nice if the mapping of mediatypes is almost the same in all the other plugins

TODO:

  • Fix relationships/embedded fields
  • Implement mapping feature for transformation plugins, like the MongoDB plugin
  • Write tests
  • Create demo site

We can use the Starwars API for tests and the demo site.

As per @mlukaszczyk from @GraphCMS, Vinylbase is the "official" demo project for GraphCMS, and they'd be delighted to have a Gatsby clone of it using the same API (https://api.graphcms.com/simple/v1/vinylbase). Spike source for reference: https://github.com/GraphCMS/example_04_static_site_generation

Issues:

1. Fields which have sub selections do not work

This one pertains to the introspection metaquery method. I'll work on that later.

Example:

Artist {
  picture {
    url
...

The metaQuery currently used is not capable of finding url from the above query, which will cause the query that fetches all data to fail. It will have to be modified. It looks like it will be a bit more difficult to find that url and add it to the query, as it might require making a __type query for each field of a Type that has subfields and then modifying the final query before firing it.

Errors can be seen here: https://github.com/Redmega/example_05_static_site_generation_with_gatsby

2. Embedded fields aren't found by GraphQL

Example:

query getAllArtists {
  allArtists {
    name
    id
    records {
      id
      title
      tracks {
        id
        title
        length
      }
    }
  }
}

This query fed into the graphcms source plugin produces artists. name, id, records, id, and titles - but no tracks, or the subfields within. They exist in the redux store json, but graphql can't seem to identify them. More investigation is needed.

@gatsbybot
Copy link
Collaborator

@gatsbybot gatsbybot commented Aug 21, 2017

Deploy preview ready!

Built with commit 7f6851c

https://deploy-preview-1867--using-drupal.netlify.com

@gatsbybot
Copy link
Collaborator

@gatsbybot gatsbybot commented Aug 21, 2017

Deploy preview ready!

Built with commit d6de985

https://deploy-preview-1867--gatsbygram.netlify.com

Copy link
Contributor

@KyleAMathews KyleAMathews left a comment

This is looking fantastic!!

Excellent work!

Really love how simple the code you've come up with.

Tests would be great + a simple example site is super helpful for manual testing in the future + as a base for people to try out the plugin.

plugins: [
/*
* Gatsby's data processing layer begins with “source”
* plugins. Here the site sources its data from a graphql endpoint
Copy link
Contributor

@KyleAMathews KyleAMathews Aug 22, 2017

"sources its data from the GraphCMS endpoint."

}
],
```
I suggest either using a `.env` file or setting environment variables to access ala `process.env.GATSBY_GRAPHQL_ENDPOINT`, `process.env.GATSBY_GRAPHQL_TOKEN` so theres no endpoint and token on git ;)
Copy link
Contributor

@KyleAMathews KyleAMathews Aug 22, 2017

Linking to https://www.gatsbyjs.org/docs/environment-variables/ would be better.

That page needs updated to include a reference to using process.env for plugins.

Also you don't want to use "I" when writing docs as this is a community plugin so generally either just use command form "Use either a .env file or set environment variables directly…" or use "We". But "we" is rarely what you want.

Copy link
Author

@Redmega Redmega Aug 22, 2017

Is there a special way to use process.env for plugins? Does the gatsby-config not have access to webpack's definitions at execution? I haven't tried it out yet, admittedly, so I will when I get home tonight.



// If type ends in a non-vowel, we need to append es. Else s.
export const formatTypeName = t => `all${t}${/s$/.test(t) ? `es` : `s`}`
Copy link
Contributor

@KyleAMathews KyleAMathews Aug 22, 2017

perhaps should use this instead? https://www.npmjs.com/package/pluralize

Or is this what GraphCMS says is their process?

Copy link
Author

@Redmega Redmega Aug 22, 2017

I'll use pluralize. Forgot about all the special edge cases!

Copy link
Contributor

@KyleAMathews KyleAMathews Aug 22, 2017

Well… what does GraphCMS do? That's what matters. Unless they use pluralize as well you'll just generate a bunch of invalid queries. Can't you get directly from the available types the connection name?

Copy link
Author

@Redmega Redmega Aug 22, 2017

According to @GraphCMS

[...] the process of pluralizing names is on their side. They use this library to do this: https://github.com/atteo/evo-inflector

I'll implement evo-inflector tonight. Looks like that's only on Java with Maven. I'll implement pluralize instead as it seems to be using the same plural rules.

Can't you get directly from the available types the connection name?

Unfortunately no, that isn't in the data returned by the __type query. It is however returned by the __schema query, so maybe something can be done with that...

@KyleAMathews
Copy link
Contributor

@KyleAMathews KyleAMathews commented Aug 22, 2017

@Redmega Redmega changed the title Add GraphCMS Plugin [gatsby-source-graphcms] Add GraphCMS Plugin Aug 24, 2017
@Redmega
Copy link
Author

@Redmega Redmega commented Sep 28, 2017

For some reason when using the custom query to fetch data, the nodes are not being created properly. It is missing certain fields. I haven't had the chance to test it with different data, I'll do more tests tomorrow.

@Thebigbignooby
Copy link

@Thebigbignooby Thebigbignooby commented Sep 28, 2017

looking forward to using this !

@Thebigbignooby
Copy link

@Thebigbignooby Thebigbignooby commented Oct 10, 2017

what's the status on this ?

@Redmega
Copy link
Author

@Redmega Redmega commented Oct 10, 2017

@Thebigbignooby Been busy with work! Sorry!

Trying to get this out as simply as possible, but passing the query through directly causes some issues with the createNode function. It's not populating nested data more than one or two levels deep. I'll add more context to the main post tonight.

@rafacm
Copy link

@rafacm rafacm commented Oct 13, 2017

I have created PR #2445 since I did not know how to add to this existing one. That PR contains a working example for this plugin in examples/using-graphcms.

/cc @Redmega @KyleAMathews @Thebigbignooby

@tlvenn
Copy link

@tlvenn tlvenn commented Oct 24, 2017

Question, with the recent addition of schema stitching feature, could this be leveraged as a way to more easily plug a graphql endpoint as a data source in Gatsby ?

@KyleAMathews what do you think ?

@rafacm
Copy link

@rafacm rafacm commented Nov 3, 2017

with the recent addition of schema stitching feature, could this be leveraged
as a way to more easily plug a graphql endpoint as a data source in Gatsby ?

@tlvenn: can you elaborate?

@Redmega
Copy link
Author

@Redmega Redmega commented Nov 3, 2017

Schema stitching is a way to stitch schema from multiple sources together into a single query: ardatan/graphql-tools#382

@Redmega
Copy link
Author

@Redmega Redmega commented Nov 9, 2017

Continued in #2445

@Redmega Redmega closed this Nov 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants