feat(gatsby-source-graphql): Query batching

[vladar]

Description

This PR allows gatsby-source-graphql users to improve performance 2-10 times by introducing query batching strategy along with a new environment variable for controlling query concurrency.

By default, gatsby-source-graphql executes each query in a separate network request. Query batching strategy implemented in this PR merges several queries into a single query and saves several network round-trips.

Consider the following queries:

{
  query: `query(id: Int!) {
    node(id: $id) {
      foo
    }
  }`,
  variables: { id: 1 },
}

{
  query: `query(id: Int!) {
    node(id: $id) {
      bar
    }
  }`,
  variables: { id: 2 },
}

They will be merged into a single query:

{
  query: `
    query(
      $gatsby0_id: Int!
      $gatsby1_id: Int!
    ) {
      gatsby0_node: node(id: $gatsby0_id) {
        foo
      }
      gatsby1_node: node(id: $gatsby1_id) {
        bar
      }
    }
  `,
  variables: {
    gatsby0_id: 1,
    gatsby1_id: 2,
  }
}

Then the response will be transformed back to the shape expected for original queries.

Caveat: Batching is only possible for queries starting approximately at the same time. In other words, it is bounded by the number of parallel GraphQL queries executed by Gatsby (by default it is 4).

To work around this limitation this PR also introduces a new environment variable GATSBY_EXPERIMENTAL_QUERY_CONCURRENCY.

By tweaking this concurrency level and number of queries in batch it possible to significantly improve the performance in some cases.

Bench on my test site (queries per second, bigger is better):

	no batching	5 queries per batch	10 queries per batch
Concurrency: 4 (baseline)	5 q/s	4.8 q/s	4.8 q/s
Concurrency: 20	18.95 q/s	19.07 q/s	22.73 q/s
Concurrency: 50	15 q/s	40.3 q/s	45 q/s
Concurrency: 100	server died :)	44.3 q/s	48 q/s

Documentation

The documentation is outlined in the README.md (happy for any advice)

Related Issues

Fixes #13425

[laurieontech]

This is awesome Vlad! I know this is a WIP so did an editing pass on the README and can do a more thorough review when it's ready.

[freiksenet]

Amazing work!

Some minor comments.

[laurieontech]

I'd actually prefer that we remove the object wrapper from those queries to make it a valid graphql query. It'll help clarify things since otherwise everything is a js tag. I'm mixed on the js vs. json. I'll let you pick which you prefer.

[vladar]

@laurieontech So GraphQL request is sent to the server as JSON:

{
  "query": "query ($id: Int!) { foo(id: $id) }",
  "variables": {
    "id": "id",
  }
}

And batching transforms both the query and the variables. So we must somehow reflect it in the example. That's why we have this wrapper vs just graphql query.

P.S. Thanks a lot for your feedback on docs 🌮

[laurieontech]

Ah, that makes sense. Hadn't thought about the transformation piece. In that case I'd go for JS as well. So keep it as is :)

[vladar]

Published in gatsby@2.20.5 and gatsby-source-graphql@2.3.0

[yaacovCR]

@vladar, any chance this batching link can be broken off into a separate package, even included within graphql-tools?

Would be helpful to community, I think.

[vladar]

The problem is that some parts of this code rely on Gatsby validations for simplicity. So it may fail in several edge cases outside of Gatsby. It is possible to make it work everywhere, it just requires time.

I think a good idea would be to wait for feedback from Gatsby users and then move this to a separate package (or graphql-tools-fork).

[gmac]

For posterity, this PR now has a shoutout in the GraphQL Tools documentation here: https://www.graphql-tools.com/docs/batch-execution#batch-execution. The GraphQL Tools implementation is more generic, but is unflappably based on the excellent work in this PR.