New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gatsby-source-graphcms] Add GraphCMS Plugin #1867

Closed
wants to merge 9 commits into
base: master
from

Conversation

Projects
None yet
6 participants
@Redmega
Copy link

Redmega commented Aug 21, 2017

Adds the gatsby-source-graphcms plugin to fetch all data from a GraphCMS backend.

Need to experiment with more complex data types and relationships to see if they're supported as is or if changes are needed.

As per @jorishermans in Discord:

Let the user decide the mapping in the plugin config in gatsby-config.js
[...] I implemented a mapping feature in gatsby-source-mongodb
it would be nice if the mapping of mediatypes is almost the same in all the other plugins

TODO:

  • Fix relationships/embedded fields
  • Implement mapping feature for transformation plugins, like the MongoDB plugin
  • Write tests
  • Create demo site

We can use the Starwars API for tests and the demo site.

As per @mlukaszczyk from @GraphCMS, Vinylbase is the "official" demo project for GraphCMS, and they'd be delighted to have a Gatsby clone of it using the same API (https://api.graphcms.com/simple/v1/vinylbase). Spike source for reference: https://github.com/GraphCMS/example_04_static_site_generation

Issues:

1. Fields which have sub selections do not work

This one pertains to the introspection metaquery method. I'll work on that later.

Example:

Artist {
  picture {
    url
...

The metaQuery currently used is not capable of finding url from the above query, which will cause the query that fetches all data to fail. It will have to be modified. It looks like it will be a bit more difficult to find that url and add it to the query, as it might require making a __type query for each field of a Type that has subfields and then modifying the final query before firing it.

Errors can be seen here: https://github.com/Redmega/example_05_static_site_generation_with_gatsby

2. Embedded fields aren't found by GraphQL

Example:

query getAllArtists {
  allArtists {
    name
    id
    records {
      id
      title
      tracks {
        id
        title
        length
      }
    }
  }
}

This query fed into the graphcms source plugin produces artists. name, id, records, id, and titles - but no tracks, or the subfields within. They exist in the redux store json, but graphql can't seem to identify them. More investigation is needed.

@gatsbybot

This comment has been minimized.

Copy link

gatsbybot commented Aug 21, 2017

Deploy preview ready!

Built with commit 7f6851c

https://deploy-preview-1867--using-drupal.netlify.com

@gatsbybot

This comment has been minimized.

Copy link

gatsbybot commented Aug 21, 2017

Deploy preview ready!

Built with commit d6de985

https://deploy-preview-1867--gatsbygram.netlify.com

@KyleAMathews
Copy link
Contributor

KyleAMathews left a comment

This is looking fantastic!!

Excellent work!

Really love how simple the code you've come up with.

Tests would be great + a simple example site is super helpful for manual testing in the future + as a base for people to try out the plugin.

plugins: [
/*
* Gatsby's data processing layer begins with “source”
* plugins. Here the site sources its data from a graphql endpoint

This comment has been minimized.

@KyleAMathews

KyleAMathews Aug 22, 2017

Contributor

"sources its data from the GraphCMS endpoint."

}
],
```
I suggest either using a `.env` file or setting environment variables to access ala `process.env.GATSBY_GRAPHQL_ENDPOINT`, `process.env.GATSBY_GRAPHQL_TOKEN` so theres no endpoint and token on git ;)

This comment has been minimized.

@KyleAMathews

KyleAMathews Aug 22, 2017

Contributor

Linking to https://www.gatsbyjs.org/docs/environment-variables/ would be better.

That page needs updated to include a reference to using process.env for plugins.

Also you don't want to use "I" when writing docs as this is a community plugin so generally either just use command form "Use either a .env file or set environment variables directly…" or use "We". But "we" is rarely what you want.

This comment has been minimized.

@Redmega

Redmega Aug 22, 2017

Is there a special way to use process.env for plugins? Does the gatsby-config not have access to webpack's definitions at execution? I haven't tried it out yet, admittedly, so I will when I get home tonight.



// If type ends in a non-vowel, we need to append es. Else s.
export const formatTypeName = t => `all${t}${/s$/.test(t) ? `es` : `s`}`

This comment has been minimized.

@KyleAMathews

KyleAMathews Aug 22, 2017

Contributor

perhaps should use this instead? https://www.npmjs.com/package/pluralize

Or is this what GraphCMS says is their process?

This comment has been minimized.

@Redmega

Redmega Aug 22, 2017

I'll use pluralize. Forgot about all the special edge cases!

This comment has been minimized.

@KyleAMathews

KyleAMathews Aug 22, 2017

Contributor

Well… what does GraphCMS do? That's what matters. Unless they use pluralize as well you'll just generate a bunch of invalid queries. Can't you get directly from the available types the connection name?

This comment has been minimized.

@Redmega

Redmega Aug 22, 2017

According to @GraphCMS

[...] the process of pluralizing names is on their side. They use this library to do this: https://github.com/atteo/evo-inflector

I'll implement evo-inflector tonight. Looks like that's only on Java with Maven. I'll implement pluralize instead as it seems to be using the same plural rules.

Can't you get directly from the available types the connection name?

Unfortunately no, that isn't in the data returned by the __type query. It is however returned by the __schema query, so maybe something can be done with that...

Angel Piscola added some commits Aug 22, 2017

@KyleAMathews

This comment has been minimized.

Copy link
Contributor

KyleAMathews commented Aug 22, 2017

@Redmega Redmega changed the title Add GraphCMS Plugin [gatsby-source-graphcms] Add GraphCMS Plugin Aug 24, 2017

@Redmega

This comment has been minimized.

Copy link

Redmega commented Sep 28, 2017

For some reason when using the custom query to fetch data, the nodes are not being created properly. It is missing certain fields. I haven't had the chance to test it with different data, I'll do more tests tomorrow.

@Thebigbignooby

This comment has been minimized.

Copy link

Thebigbignooby commented Sep 28, 2017

looking forward to using this !

@Thebigbignooby

This comment has been minimized.

Copy link

Thebigbignooby commented Oct 10, 2017

what's the status on this ?

@Redmega

This comment has been minimized.

Copy link

Redmega commented Oct 10, 2017

@Thebigbignooby Been busy with work! Sorry!

Trying to get this out as simply as possible, but passing the query through directly causes some issues with the createNode function. It's not populating nested data more than one or two levels deep. I'll add more context to the main post tonight.

@rafacm

This comment has been minimized.

Copy link

rafacm commented Oct 13, 2017

I have created PR #2445 since I did not know how to add to this existing one. That PR contains a working example for this plugin in examples/using-graphcms.

/cc @Redmega @KyleAMathews @Thebigbignooby

@tlvenn

This comment has been minimized.

Copy link

tlvenn commented Oct 24, 2017

Question, with the recent addition of schema stitching feature, could this be leveraged as a way to more easily plug a graphql endpoint as a data source in Gatsby ?

@KyleAMathews what do you think ?

@rafacm

This comment has been minimized.

Copy link

rafacm commented Nov 3, 2017

with the recent addition of schema stitching feature, could this be leveraged
as a way to more easily plug a graphql endpoint as a data source in Gatsby ?

@tlvenn: can you elaborate?

@Redmega

This comment has been minimized.

Copy link

Redmega commented Nov 3, 2017

Schema stitching is a way to stitch schema from multiple sources together into a single query: apollographql/graphql-tools#382

@Redmega

This comment has been minimized.

Copy link

Redmega commented Nov 9, 2017

Continued in #2445

@Redmega Redmega closed this Nov 9, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment