[Do not merge] 2.0 with schema stitching

[freiksenet]

This is a WIP take on schema-stiching and schema merging. This will be published to a pre-release tag. See src/test/testingSchemas.js for examples..

API

makeRemoteExecutableSchema({ schema: GraphQLSchema, fetcher: Fetcher }): GraphQLSchema

Given a GraphQL schema (can be a non-executable client schema made by buildClientSchema) and a Fetcher, produce a GraphQL Schema that routes all requests to the fetcher.

introspectSchema( fetcher: Fetcher, context?: {[key: string]: any} ): Promise<GraphQLSchema>

Use fetcher to build a client schema using introspection query. For easy of use of makeRemoteExecutableSchema. Provides a client schema, so a non-executable schema. Accepts optional second argument context, which is passed to the fetcher.

Fetcher

type Fetcher = (
  operation: {
    query: string;
    operationName?: string;
    variables?: { [key: string]: any };
    context?: { [key: string]: any };
  },
) => Promise<ExecutionResult>;

Usage with apollo-link

import { HttpLink, makePromise, execute } from 'apollo-link';

const link = new HttpLink({ uri: 'http://api.githunt.com/graphql' });
const fetcher = (operation) => makePromise(execute(link, operation));
const schema = makeRemoteExecutableSchema({
  schema: await introspectSchema(fetcher),
  fetcher,
});

Usage with apollo-fetch

import { createApolloFetch } from 'apollo-fetch';

const apolloFetch = createApolloFetch({ uri: 'http://api.githunt.com/graphql'});
const fetcher = ({ query, variables, operationName, context}) => apolloFetch({
  query, variables, operationName
});
const schema = makeRemoteExecutableSchema({
  schema: await introspectSchema(fetcher),
  fetcher,
});

Usage with a generic HTTP client (like node-fetch)

import fetch from 'node-fetch';

const fetcher = async ({ query, variables, operationName, context }) => {
  const fetchResult = fetch('http://api.githunt.com/graphql', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ query, variables, operationName })
  });
  return fetchResult.json();
};
const schema = makeRemoteExecutableSchema({
  schema: await introspectSchema(fetcher),
  fetcher,
});

mergeSchemas

mergeSchemas({
  schemas: Array<GraphQLSchema | string>,
  resolvers?: (mergeInfo: MergeInfo) => IResolvers,
  onTypeConflict?: (
    left: GraphQLNamedType,
    right: GraphQLNamedType
  ) => GraphQLNamedType
})

type MergeInfo = {
  delegate(
    operation: 'query' | 'mutation',
    rootFieldName: string,
    args: any,
    context: any,
    info: GraphQLResolveInfo
  ) => any
}

schemas

schemas can be both GraphQLSchema (but it has to be an executable schema) or strings. In case they are strings only extensions (extend type) will be used. Passing strings is useful to add fields to existing types to link schemas together.

resolvers

resolvers is an optional a function that takes one argument - mergeInfo and
returns resolvers in makeExecutableSchema format.

mergeInfo and delegate

mergeInfo currenty is an object with one propeprty - delegate.

delegate takes operation and root field names, together with GraphQL context
and resolve info, as well as arguments for the root field. It forwards query to
one of the merged schema and makes sure that only relevant fields are requested.

mergeInfo.delegate(
  'query',
  'propertyById',
  {
    id: parent.id,
  },
  context,
  info,
);

onTypeConflict

onTypeConflict lets you customize type resolving logic. Default logic is to
take the first encountered type of all the types with the same name. This
methods allows customization of this, for example by taking other type or
merging types together.

Example

import {
  makeRemoteExecutableSchema,
  introspectSchema,
  mergeSchemas,
} from 'graphql-tools';
import { HttpLink, execute, makePromise } from 'apollo-link';

async function makeMergedSchema() {
  // Create remote executable schemas
  const PropertyLink = new HttpLink({
    uri: 'https://v7l45qkw3.lp.gql.zone/graphql',
  });
  const PropertyFetcher = operation =>
    makePromise(execute(PropertyLink, operation));
  const PropertySchema = makeRemoteExecutableSchema({
    schema: await introspectSchema(PropertyFetcher),
    fetcher: PropertyFetcher,
  });

  const BookingLink = new HttpLink({
    uri: 'https://41p4j4309.lp.gql.zone/graphql',
  });
  const BookingFetcher = operation =>
    makePromise(execute(BookingLink, operation));
  const PropertySchema = makeRemoteExecutableSchema({
    schema: await introspectSchema(BookingFetcher),
    fetcher: BookingFetcher,
  });

  // A small string schema extensions to add links between schemas
  const LinkSchema = `
    extend type Booking {
      property: Property
    }

    extend type Property {
      bookings(limit: Int): [Booking]
    }
  `;

  // merge actual schema
  const mergedSchema = mergeSchemas({
    schemas: [PropertySchema, BookingSchema, LinkSchema],
    // Define resolvers manually for links
    resolvers: mergeInfo => ({
      Property: {
        bookings: {
          fragment: 'fragment PropertyFragment on Property { id }',
          resolve(parent, args, context, info) {
            return mergeInfo.delegate(
              'query',
              'bookingsByPropertyId',
              {
                propertyId: parent.id,
                limit: args.limit ? args.limit : null,
              },
              context,
              info,
            );
          },
        },
      },
      Booking: {
        property: {
          fragment: 'fragment BookingFragment on Booking { propertyId }',
          resolve(parent, args, context, info) {
            return mergeInfo.delegate(
              'query',
              'propertyById',
              {
                id: parent.propertyId,
              },
              context,
              info,
            );
          },
        },
      },
    }),
  });

  return mergedSchema;
}

Known issues

• Unions and interfaces don't work on remote schemas
• Unions and interfaces don't work with remote schemas unless __typename is passed. However in mergedSchema it is done already, just need to figure out how to port it.
• New lower level API that will allow handling links by just defining resolvers
• Better default solution for remote custom scalars, currently they break eg JSON.
• Make sure that aliases work
• Types that only implement interfaces are missing
• Better error handling - handle non fatal errros, make sure errors propagate from sub schemas, optional imperative interface to deal with errors from delegate.
• Optional arguments in root fields don't work as expected when using delegate (and are basically ignored)
• Generate var names in a more sane way, currently we just do _${argName}.
• Figuring out if we want to add namespacing transform for schema

Production ready checklist

• Documentation
• Example app(s)
• Blog post / announcement
• Get rid of lodash and other extra deps
• Validate usage in production for several people
• Move all tests to not use .to.not.be.undefined, really hard to debug. use .to.deep.equal for full response

[stubailo]

Made the tests pass by adding @types/lodash package!

One idea: What if instead of prefixing field names, we put them in a nested field? Like:

# current
query {
  Booking_bookingById {
    ...
  }
}

# proposed
query {
  Booking {
    bookingById { ... }
  }
}

[stubailo]

Some initial comments, more to come

[stubailo]

This should be "no such schema" - do we even need these getters though? Would it be easier to just make schemas public?

[freiksenet]

Then it won't type check :)

[stubailo]

Wait really? Why not?

[stubailo]

Maybe getLinkByFieldName?

[stubailo]

Yep - I'll want this for the demo. Might write it

[freiksenet]

fixed.

[stubailo]

Hmm, I think it would be better if we took two separate arguments for schema and field name, instead of splitting them on _. This couples it to the serialization we choose in a different part of the code.

[freiksenet]

Yeah, whole namespacing related code is a bit of a hack and will be better once we figure out proper namespaces.

[timbotnik]

RE: prefixing - my initial use-case is to merge an existing in-production schema with a new one. This means that actually I don't want to prefix my existing schema at all, that should be supported shouldn't it?

[stubailo]

@timbotnik say more? Oh you mean you just want to graft a new one onto an existing one?

I think it would make sense to be able to have no prefix at all.

[timbotnik]

@stubailo yes "grafting" is a good word for it ;-) Also, if you knew for sure that there would be no namespace conflicts, it could be useful to support a non-prefixed merge in general.

Thinking through a general case for non-prefixed merges, something that would be amazing would be if there was a way to handle de-duplication. I'm thinking more here about things like custom scalars and "utility" types that could be imagined to resolve exactly the same way on either endpoint.

Maybe more complicated, but also worth considering that an interface type might be shared between two schemas but each schema provides a different implementation.

[stubailo]

Yeah - I think the deduplication is probably going to be a future thing. Do you think it's critical for your initial use case?

[timbotnik]

@stubailo I don't think it's critical at all, however the "custom scalar" thing is probably the thing that will look the strangest - like having Timestamp and Void [NS]_Timestamp and [NS]_Void in the final schema.

[stubailo]

Yeah, that's a great point. We should definitely work towards that. Having a concrete production use case would be super great.

[freiksenet]

@timbotnik We didn't really plan for "grafting" use case, some of the code would probably be shared, but I don't think we will support it in first iteration. However it seems that there are lots of use cases in schema merging, so I'll definitely think on how to allow some customization of how stuff will be merged, maybe providing functions to resolve type conflicts.

(Having said that, we really don't know what is the end goal of what we are doing, it's very much explanatory thing and a way to collect schema use cases. Maybe grafting is a very common one and then we'll certainly support it)

[freiksenet]

@timbotnik Re: de-duplication, our longer term plan involved having namespaces that can depend on other namespaces for shared types. Thus you could have 'common types' as one namespace and use scalars and shared types from there.

[freiksenet]

@stubailo This could work for field namespacing, but we would still need namespaces for types. I'm not sure :)

[stubailo]

Hmmm for the final release it would be super awesome to not include lodash - for client side use that really kills the bundle size.

[freiksenet]

Yeah, we can either inline them or use lodash/function modules.

[timbotnik]

@freiksenet I think perhaps the definition of "grafting" is getting confused. By this, all I mean is that I want to use an existing "local" schema (that I already have clients using in production), and merge a new schema onto it. Since the existing schema is already in use in the wild, I don't want to introduce a prefix on that schema. I think the only feature ask here is to allow me to turn off prefixing for this one schema during the merge.

[stubailo]

Just published 2.0.0-alpha.1! You can install with npm install graphql-tools@next!

[freiksenet]

@timbotnik Ah, now I get it. Yeah, I will separate namespacing from merging today.

…

On Thu, 17 Aug 2017, 5.00 timbotnik ***@***.***> wrote: @freiksenet <https://github.com/freiksenet> I think perhaps the definition of "grafting" is getting confused. By this, all I mean is that I want to use an existing "local" schema (that I already have clients using in production), and merge a new schema onto it. Since the existing schema is already in use in the wild, I don't want to introduce a prefix on that schema. I *think* the only feature ask here is to allow me to turn off prefixing for this one schema during the merge. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#382 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKjiBznbY9X96nZZKYL0Xval4VRzpyxks5sY56ggaJpZM4O3jxQ> .

[trevordmiller]

I'm excited for makeRemoteExecutableSchema! Thank you.

[stubailo]

BTW here's my sample code from the talk at GraphQL NYC! This only works with 2.0.0-alpha.5 - the API for future alphas is a bit different.

import * as express from "express";
import * as bodyParser from "body-parser";
import { graphqlExpress, graphiqlExpress } from "apollo-server-express";
import { makeRemoteExecutableSchema, mergeSchemas } from "graphql-tools";
import { createApolloFetch } from "apollo-fetch";

async function run() {
  // XXX fix typings
  const universeSchema = await makeRemoteExecutableSchema(
    createApolloFetch({
      uri: "https://www.universe.com/graphql/beta"
    }) as any
  );

  const weatherSchema = await makeRemoteExecutableSchema(
    createApolloFetch({
      uri: "https://5rrx10z19.lp.gql.zone/graphql"
    }) as any
  );

  const schema = mergeSchemas({
    schemas: [
      { prefix: "universe", schema: universeSchema },
      { prefix: "weather", schema: weatherSchema }
    ],
    links: [
      {
        name: "location",
        from: "Event",
        to: "weather_location",
        resolveArgs: parent => ({ place: parent.cityName }),
        fragment: `
          fragment WeatherLocationArgs on Event {
            cityName
          }
        `
      }
    ]
  });

  const app = express();

  app.use("/graphql", bodyParser.json(), graphqlExpress({ schema }));

  app.use(
    "/graphiql",
    graphiqlExpress({
      endpointURL: "/graphql"
    })
  );

  app.listen(3000);
  console.log("Listening!");
}

try {
  run();
} catch (e) {
  console.log(e, e.message, e.stack);
}

Here's the query and result:

Also here are my slides for the talk: https://www.slideshare.net/sashko1/modular-graphql-with-schema-stitching

[stubailo]

I added tests that check for local schema merging, remote schema merging, and a hybrid of the two.

Should help us catch bugs with remote schemas! Already found one :]

[nnance]

@freiksenet @stubailo this is amazing stuff. so much progress in such a short amount of time. so I have been very busy decomposing our production graphql server over the last couple of weeks. I have decomposed them into something we call GraphQL modules. It's in the perfect state to use this project to test a production use case. The only problem is that I will not be able to share the project. However, we are currently merging/stitching 6 modules together at build time to make up our GraphQL server including a pretty large set of common types so it should be a good use case. I will be able to share snippets as I find problems.

Would this be helpful?

[stubailo]

Absolutely! Please let us know whatever problems you run into, and also what features are missing for you to be able to use the tool :)

Super excited to see people getting pumped about this.

[timbotnik]

Remote schemas are not working if they have root types that are not called Query or Mutation, such as

schema {
  query: RootQuery
  mutation: RootMutation
}

b/c the schema actually needs to be transformed as well to replace RootQuery => Query etc

[timbotnik]

You get errors like Error: "Query" defined in resolvers, but not in schema

[freiksenet]

Fixed that.

[gilesbradshaw]

How easy to get this working over websockets?

[stubailo]

@gilesbradshaw you could easily get queries and mutations over websockets working with the right fetcher passed in, but for subscriptions it gets a bit more complicated for sure. That would be a big extra feature on top of this.

[stubailo]

This needs a changelog entry 😛

[freiksenet]

@sashko 🖕

This is ready to ship now :)

Fixed!

[lifeiscontent]

@freiksenet where can I find the documentation for this now that this is merged?

[pozylon]

Awesome this seems to be done, but I just tried it and aliases don't work anymore if routed through makeRemoteExecutableSchema.

[martijnwalraven]

@pozylon: Can you elaborate on that and maybe include a code example that shows what doesn't work?

[pozylon]

Of course, sorry, here you go with the query:

{
  orders {
    status
    created
    updated
    ordered
    confirmed
    fullfilled
    currency
    it: total(category: ITEMS) {
      amount
      currency
    }
  }
}

I run it in GraphiQL and that was the result without proxying:

{
  "data": {
    "orders": [
      {
        "status": "CONFIRMED",
        "created": 1506586141020,
        "updated": 1506586183279,
        "ordered": 1506586183294,
        "confirmed": 1506586183294,
        "fullfilled": null,
        "currency": "CHF",
        "it": {
          "amount": 5958,
          "currency": "CHF"
        }
      },...

And that's what happens with proxying through the stitching:

{
  "data": {
    "orders": [
      {
        "status": "CONFIRMED",
        "created": 1506586141020,
        "updated": 1506586183279,
        "ordered": 1506586183294,
        "confirmed": 1506586183294,
        "fullfilled": null,
        "currency": "CHF",
        "it": null
      },...

total is of type "Money" which has a currency and amount (String + Int)

[martijnwalraven]

@pozylon: So are you saying it works if you remove the it alias for total?

[pozylon]

@martijnwalraven: exactly, if i call it without the alias it's all fine.

[martijnwalraven]

@pozylon: Great, thanks for explaining. @freiksenet, do you have any idea what could be causing this?

[gviligvili]

I have a micro service (A) which has no protection.
so I stitch it to my main app server (B) which has protection.

is there a recipe on how to (B) can protect (A) queries? Like to if the user authorized or validate logic first?

[pozylon]

@gviligvili: Use a middleware together with apollo-fetch based Fetcher and cancel requests when appropriate. https://github.com/apollographql/apollo-fetch

[freiksenet]

@lifeiscontent we'll merge in the docs asap, see https://github.com/apollographql/tools-docs/pull/134 for progress.

@pozylon Thanks, i'll check it.

[freiksenet]

@pozylon Fixed in #411

[Techbinator]

Hello again,

I have a small issue stitching a schema that was automatically generated with postgraphql library(https://github.com/postgraphql/postgraphql).

The service works fine on it's own but when i try to mergeSchemas i get "Unhandled promise rejection (rejection id: 1): Error: No such type: Query".

I already stitched 3 other services and it works perfectly.

You can find here the generated schema https://gist.github.com/filtudo/40ffe1bf27863d736e1646f9576dab4d

And here a link with the graphiql schema docs
https://pasteboard.co/GNmvbyz.png

Any help or guidance would be greatly appreciated.

Cheers

[freiksenet]

@filtudo The reason it doesn't work is because it refers to Query type in each payload. We handle Query type specially, tbh I didn't envision Query used this way. I wonder how common such use case is in practice.

Ping @sashko @jbaxleyiii @martijnwalraven

[Techbinator]

@freiksenet

Any chance on "tackling" this issue on your side(since postgraphql i would say is a pretty decently used library) or i should start thinking of alternatives/hacks for this case?

Thank you

[freiksenet]

@filtudo It seems to be a common pattern in many APIs to support Relay, so I will fix it, yes.

[freiksenet]

@filtudo Fixed in #413

[freiksenet]

@filtudo Published as 2.2.1.

@pozylon Your bug is fixed in 2.2.0.

Hey folks, this is part of main release now, please create new issues if you have any problems with schema stitching, it's hard to track this thread cause it's closed.