[research] Data Layer: how next.js connects to backends

[rufuspollock]

How should we get (meta)data / content? What is the "data layer" for this system?

The content fetching divides into two parts:

graph LR

remote[Remote e.g. CKAN API] --fetchers--> localdb[Local DB Cache]
localdb --common api--> render[Renderer]

• Want a common API / structure for local DB so that renderers have a simple interface and the complexity of fetchers is hidden from them. Options:
  • redux
  • graphql based
• SUMMARY: atm we are strongly inclining to graphql based approach. (see below for reasoning)
• fetchers will always be somewhat bespoke

Tasks

• Research options and summarize approach DONE - see above for summary
• Try out apollo graphql for local db (put in some fixtures) Try out apollo graphql server for "Local DB" (local state and remote data cache) #24
• Try out connecting up external API e.g. CKAN to local DB GraphQL Experiment - Apollo + Next.js #13

Analysis (in progress)

Next.js data fetching can be done in two ways:

• getStaticProps - fetches data at build time
• getServersideProps - fetches data at runtime

We want to use the getServersideProps method for our use case. There's a repo with helpful examples to see how data fetching works in Next. https://github.com/vercel/next.js/blob/canary/examples

What we want ...

fetch could use graphql, raw queries it doesn't matter ...

graph TD

external[External Data Source e.g. CKAN or Wordpress]
react[React Component]
page[Rendered Page]

external --fetch--> react
external --fetch --> redux
external --fetch --> store[Store - non redux]
redux--> react
store --graphql--> react

react --render--> page

gatsby ...

graph TD

external[External] --> source[Source plugin]
source --> store[Internal Store - nodes]
store --graphql--> component[Component]
component --render--> page

Inbox (misc)

• Official Next.js example using wordpress: https://github.com/shuding/next-cms

Inbox on Graphql approach

• https://medium.com/disdj/next-js-redux-in-an-easy-way-fae083aa037c - short guide to using redux in nextjs
• What about using graqhql
  • apollo
  • https://relay.dev/ - this basically replaces redux.
  • urlq https://github.com/FormidableLabs/urql
• Using GraphQL API in getStaticProps or getServerSideProps vercel/next.js#10946 - Using GraphQL API in getStaticProps or getServerSideProps
• Mentions https://github.com/lfades/next-with-apollo
• Lots of examples
  • https://github.com/zeit/next.js/tree/canary/examples
  • https://github.com/zeit/next.js/tree/canary/examples/with-apollo

Summary

• A local graphql layer can replace redux as local state management solution
  • https://engineering.circle.com/https-medium-com-mattdionis-move-over-redux-apollo-client-as-a-state-management-solution-1f9325f96cdd
  • Relay supports this: https://relay.dev/docs/en/local-state-management
  • Apollo supports this https://www.apollographql.com/docs/react/data/local-state/
• And stuff like apollo has multiple sources out of the box that convert other APIs into local graphql ...
  • https://www.apollographql.com/docs/tutorial/data-source/
• This is an attractive approach
  • Fixes one of long-standing pain points for redux (local vs nonlocal state)
  • Nice query interface to local state
• Apollo is better than relay (?)
  • Jan 2019 - https://blog.bitsrc.io/apollo-and-relay-side-by-side-adb5e3844935

    • I found the Relay documentation to be opaque, incomplete, and downright confusing at times. The docs were often missing the why behind certain decisions; instead, you had to follow along and figure it out yourself. Apollo’s documentation isn’t perfect, but it is much more thorough and beginner-friendly. [ed: i would agree based on brief perusal]

  • https://gist.github.com/nikolasburk/36cae3e7f79074a34c4d64f1428c96ac - https://www.prisma.io/blog/relay-vs-apollo-comparing-graphql-clients-for-react-apps-b40af58c1534 - 3y old and old version of relay
  • Con: gatsby uses relay internally
  • Artsy explaining use of relay: https://artsy.github.io/blog/2019/04/10/omakase-relay/
    • Well engineered. Was not alternative
    • Apollo is now much bigger ecosystem and friendlier ...
• next.js support
  • https://github.com/vercel/next.js/tree/canary/examples/with-relay-modern
  • https://github.com/vercel/next.js/tree/canary/examples/with-apollo

Excellent critique of next.js SSR approach: https://en.paqmind.com/blog/ssr-is-not-the-future and recommendations on how to approach (jan 20 2020)

This section of gatsby doc is useful https://www.gatsbyjs.org/docs/graphql-concepts/#why-is-graphql-so-cool

For a more in-depth look, read why Gatsby uses GraphQL.

• Eliminate frontend data boilerplate — no need to worry about requesting & waiting for data. Just ask for the data you need with a GraphQL query and it’ll show up when you need it
• Push frontend complexity into queries — many data transformations can be done at build-time within your GraphQL queries
• It’s the perfect data querying language for the often complex/nested data dependencies of modern applications
• Improve performance by removing data bloat — GraphQL enables you to select only the data you need, not whatever an API returns

[anuveyatsu]

Hi @rufuspollock should we wait for this to complete before we can start wiring up CKAN API with, e.g., Apollo?

[anuveyatsu]

Closing this since we've implemented #29