Composing public GraphQL APIs #490

Open
KyleAMathews opened this Issue Sep 15, 2016 · 15 comments

Projects

None yet

8 participants

@KyleAMathews

A more relevant question now as of yesterday ;-)

So with Github or any other of what I'm assuming is the coming avalanche of public GraphQL APIs, it'd often be really handy if you could compose all or parts of a public schema with your private one.

For example, what if all of your users had a connected Github account and you could write queries like:

{
  viewer {
    githubViewer {
      repositories { edges { node { name, id } } }

  }
}

Or if there was a Wikipedia GraphQL API (PLEEAASSEEE someone build this!):

{
  viewer {
    hometown { // Jump into Wikipedia here
      population
    }
  }
}

Ideally there'd be some helper code that'd consume the introspection query and auto-construct the schema locally + provide helpers for bridging between schemas. Then there'd need to be some sort of AST slicer to pull out the part of the query that's for the remote API and then send it there to be resolved.

Thoughts? Doable? Someone already explored this further than I have? NPM package I missed ;-)?

/cc @taion who helped develop some of these ideas.

@taion
taion commented Sep 15, 2016

It does seem like this could live in userspace. Something like

gitHubUser: {
  type: remoteType(gitHubSchema, 'User'),
  resolve: (obj, args, context, info) => (
    resolveRemote(
      GITHUB_API_URL,
      fragmentName => `user($user: ${obj.gitHubId}) { ${fragment} }`,
      info,
    )
  ),
}

Perhaps?

@stubailo

I'd love to work on something like this. My first thought would be:

  1. Import schema with introspection
  2. Namespace all types, for example by prepending GitHub_ and Wikipedia_
  3. Write some code or a DSL that adds some fields/resolvers to existing types, to wire together the schema; for example:
extend GitHub_Repository {
  wikipediaPage: Wikipedia_Page
}
resolver: (root, args, context, info) => {
  return <something goes here>
}

I think the main help here would be a way to get the relevant fragment to send to the Wikipedia API from somewhere like info.

@taion
taion commented Sep 15, 2016

For introspection, I think it'd be important to do that as a build step rather than a runtime step. You don't want to be firing off an introspection query at runtime.

@stubailo

Agreed!

@nybble73

I've been messing around with extendSchema - but as with a few other ideas I've played with, it falls apart on interface naming (i.e. the two services both are relay compliant so implement the node interface, but extendSchema of the external services graphql and the actual services node aren't the same so it conflicts - even though the shape of them is identical).

@theorygeek

One important consideration would be how you secure it? That is, if the query against GitHub runs with elevated privileges, you probably care about the contents of the fragment you're sending to the remote server.

Maybe you only import part of the remote schema, and exclude parts that shouldn't be accessible? Or do you look at the AST of the inner fragment and apply security before you execute the remote query or something?

Just don't want something like this:

{
  viewer {
    someRemoteApi {
      node(id: "some-object-i-shouldn't-be-able-to-access") {
        ... on SomeType { sensitiveField }
      }
    }
  }
}
@pluma
pluma commented Sep 27, 2016 edited

This gets a huge ๐Ÿ‘ from me because it ties in with my work on GraphQL at ArangoDB. Basically ArangoDB allows developers to build data-centric microservices in JS that run directly in the database and expose their own HTTP APIs. We've supported GraphQL using graphql-sync (which is a wrapper around graphql-js without promises) since ArangoDB 2.8. The upcoming release (thanks to the great example of express-graphql) will make it even easier to expose GraphQL APIs from those microservices.

With REST APIs these services would typically be put behind a public-facing proxy that takes care of authentication, CORS and so on. There are even dedicated SaaS solutions for this like Kong.

With GraphQL there's currently no way to tie multiple GraphQL APIs together like this. IMO this currently doesn't make for a compelling story for GraphQL as a replacement for REST APIs in microservices.

The security concerns when composing GraphQL APIs would be almost exactly the same as when providing a credentialed proxy to a REST API. A simple whitelist or blacklist would suffice IMO (though even that seems non-trivial to implement).

@taion
taion commented Sep 27, 2016

I don't think exposing a number of GraphQL APIs internally is an effective way to build a unified public-facing GraphQL API when using microservices internally.

The missing piece conceptually is that, in an effective GraphQL API, it's useful for types to form a graph. Suppose you were building a simple blog-like structure with a "users" service and a "posts" service. It's not just the case that these can trivially live in a flat URL-like space โ€“ the User type should have a posts: [Post] field, but the Post type should have an author: User field.

Unlike with a REST API where perhaps the post service can expose an author_id field, an effective GraphQL schema should actually expose relationships โ€“ in which case you need some way to express what they are, and then you're halfway toward actually building out a GraphQL schema.

I think the good pattern here is connecting to federated services in a limited way โ€“ having a nice way to link to the GitHub data for a user is great.

The idea of a GraphQL reverse proxy like you'd use for REST is IMO not a good pattern, though. Just build an actual schema โ€“ the richness available there is one of the main benefits of using GraphQL.

@nybble73

We're moving toward a federated graphql microservices system as well. I agree that this is not a trivial undertaking. But if our internal structure has specific guidelines on naming and global ids, it should be possible to federate to each service requests that they provide and have the central dispatching service understand how to hydrate the links between them.

@KyleAMathews

Drupal's graphql support seems to be getting really good https://youtu.be/yhyoQwuSUWo

@KyleAMathews KyleAMathews referenced this issue in superawesomelabs/leo Oct 12, 2016
Open

leo-plugin-github #146

@nodkz
Contributor
nodkz commented Oct 26, 2016 edited

@KyleAMathews I'm trying implement what you want in https://github.com/nodkz/graphql-compose
graphql-compose is a tool for building graphql types. Still in progress (polish API). So I'm not ready to write docs for it right now.

On top of graphql-compose, I builded this plugins:

All these tools I'm using for our internal project. So for OSS I'm moving not so fast as I want. And compose plugins for 3rd party GraphQL Services and REST API planned at the beginning of next year.

With graphql-compose and its plugins, the building of a graphql schema on the server is not pain anymore. For me ;) Ridiculously redeem from ctrl+ั, ctrl+v. Just see on example app for northwind data, where ~700LoC converted 8 mongoose models to schema with 123 graphql types [live demo] [source code]. Relay app example which works with this northwind graphql server.


PS. Some funny slides about the graphql difference between frontend and backend developers:

screen shot 2016-10-26 at 12 20 30

screen shot 2016-10-26 at 12 24 50

@linonetwo

hahahah that's why we sometimes needs to use client side graphql server @nodkz

@linonetwo

Any progress on this?

@bsr203 bsr203 referenced this issue in neelance/graphql-go Dec 7, 2016
Closed

import types of schema A into schema B #1

@linonetwo
linonetwo commented Dec 10, 2016 edited

@stubailo I have an idea: Using JSON-LD 's concepts.

The JSON-LD Expansion is for transforming local types to global types in schema.org, to make a single source of types (think about the single source of truth). And namespacing a remote schema is about doing something like JSON-LD Expansion.

And JSON-LD Compaction is for using types in schema.org. It will transform long URL-like schema.org types to a short type name. (See https://www.youtube.com/watch?v=Tm3fD89dqRE if you aren't clear about it) . In GraphQL, we can just use introspection to do that, which is costly. But in JSON-LD way, what we need is to expose a ใ€Œ@contentใ€ field. And using a remote type means using something from a type source likes schema.org. we can do costless cascading by this, since cascading GraphQL may need to use tons of remote types.

Just an idea inspired by #271

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment