Relay Global Object Identification support.

[alloy]

This adds the Relay Node ID support that we need to make full use of Relay’s capabilities.

• It adds a Node interface, which identifies an object with the __id field. From the __id value it is able to deduce the actual type and its type-specific ID.
• Article, Artist, Artwork, Partner, and PartnerShow have gained the Node interface. Any other types that will be used with Relay in the future will need to get support added.
• All types that have a id field now also have a __id field, because even though you can’t look all of those up through the Node interface yet, the globally unique IDs does mean that Relay can already cache them efficiently.
• I started using GraphQLID for all id fields instead of GraphQLString. I did not yet add the non-null type to all of those, but I have a feeling that there are lots of places where a value (such as an ID) cannot be null.

[alloy]

Fetch Artwork Global ID

{
  artwork(id: "mr-brainwash-charlie-chaplin-red") {
    __id
  }
}

{
  "data": {
    "artwork": {
      "__id": "QXJ0d29yazptci1icmFpbndhc2gtY2hhcmxpZS1jaGFwbGluLXJlZA=="
    }
  }
}

Fetch Artwork through Node interface

{
  node(__id: "QXJ0d29yazptci1icmFpbndhc2gtY2hhcmxpZS1jaGFwbGluLXJlZA==") {
    __typename
    ... on Artwork {
      __id
      _id
      id
      title
    }
  }
}

{
  "data": {
    "node": {
      "__typename": "Artwork",
      "__id": "QXJ0d29yazptci1icmFpbndhc2gtY2hhcmxpZS1jaGFwbGluLXJlZA==",
      "_id": "575a0dcfcd530e4f0d00022e",
      "id": "mr-brainwash-charlie-chaplin-red",
      "title": "Charlie Chaplin (Red)"
    }
  }
}

[alloy]

The ID is a base64 encoded combination of the type and its real ID, so that it can be deconstructed and resolved regardless of type.

ruby -r base64 -e 'p Base64.decode64("QXJ0d29yazptci1icmFpbndhc2gtY2hhcmxpZS1jaGFwbGluLXJlZA==")'
"Artwork:mr-brainwash-charlie-chaplin-red"

[alloy]

Decided to try to get Relay to use a differently named ID field so we don’t have to update all clients, especially not Eigen, which uses it in 1 place for analytics facebook/relay#1061 (comment)

[alloy]

Well, seems like trying to change that requirement in upstream Relay might take longer than I want, so for now I’ll just continue as-is, but if there’s a hint of things changing in Relay before this PR is finalise I’ll switch it over.

[alloy]

@ashfurrow I see you’re using the id field from MP https://github.com/artsy/eigen/blob/5427294088fdd8c5a51bc7d3cdeb644c9f0f42ef/Artsy/Networking/ARRouter.m#L1029 & https://github.com/artsy/eigen/blob/5427294088fdd8c5a51bc7d3cdeb644c9f0f42ef/Artsy/Networking/ARRouter.m#L1031. Would you be able to say if that ID is used outside of the context of MP? i.e. do you expect it be an ID that you can use to fetch data from other APIs?

[alloy]

If it turns out that there’s more already deployed iOS code that relies on id fields, then I guess it makes more sense to push for adding support for a different field to Relay.

(Although in the case I linked to in the previous comment the model is a Sale and we’re not using that in Eigen atm, I think it would be very confusing API to have id be something different in different places.)

[alloy]

Thoughts on the name of this parameter?

[ashfurrow]

Makes sense.

[dzucconi]

We can't just call it id and it do the right thing?

[dzucconi]

:reads above:

[dzucconi]

gid? key?

[ashfurrow]

We use the id and _id fields of the Sale object in different ways. For _id, we call it the causalityID because it's what we need to connect and authenticate against the causality web socket API. The id field we call the liveSaleID and is used as a more traditional gravity id (basically as a slug) to direct the user to auction registration URLs and to fetch bidders and sale info from gravity directly. The distinction between the two isn't immediately obvious to me, and I regret that in my haste to get causality integration working, I may have misused the fields.

Please advise on next steps – it's possible to disable the native live experience and fall back to a mobile web view (indeed, we're planning on doing so already). If this is an infrastructure change that eigen needs to make, we can roll it into our existing update.

[dzucconi]

Thank youuu

[alloy]

Oh you had wanted this? I reverted it, because it seems like I won’t need it after all :/

Let me know if anyone wants this in anyways and I’ll re-apply the patches.

[alloy]

Ok, re-adding it after all.

[alloy]

It seems like my code is failing on Node 5, I’m using Node 6 locally.

Ok, was using Array.prototype.includes which isn’t available on Node 5 yet.

[alloy]

Paging @broskoski, now that @dzucconi has officially left.

[alloy]

Oops, forgot that @broskoski is out for the week, paging @mzikherman, sir.

[alloy]

Noticed I still need to add support to to Author and Partner.

[mzikherman]

nice makes sense

[alloy]

I changed nothing with regards to the NPM modules in the last commit, yet CI now fails on that 😭

[alloy]

I’m not even sure why it tries to install fsevents 1.0.5, the shrinkwrap file has it pegged to 1.0.12

[alloy]

If I ssh into CI and run npm install it actually does install fsevents 1.0.12. Some stale dependency cache somewhere?

[mzikherman]

Noo :( sorry for making you go back in for minor-ish changes

[alloy]

Making the fsevents 1.0.12 dependency explicit does change something, but now it hits the following:

npm ERR! enoent ENOENT: no such file or directory, rename '/home/runner/metaphysics/node_modules/tar-pack/node_modules/isarray' -> '/home/runner/metaphysics/node_modules/tar-pack/node_modules/isarray'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.

Like… I dunno… it’s trying to rename a path to the same path? What’s up with that?

I give up, this seems very much a CI issue. I have no issues locally and also not when SSH-ing into CI. I’m going to revert the last commit and call it a day.

Noo :( sorry for making you go back in for minor-isa changes

Nah, that change should be totally irrelevant. I would say, if you feel good about the code, go ahead and merge it and give CI some time to chill, if that’s something y’all do in some cases.

[mzikherman]

I'll try to rebuild it tonight (explore Semaphore's dependency cache- maybe can be force updated?), and if not I'll merge and babysit the staging deploy.

Either way you'll have it by (your) tmrw morning :)

Thanks for the awesome PR.

[alloy]

explore Semaphore's dependency cache- maybe can be force updated?

My naive attempt was based on this: https://semaphoreci.com/docs/caching-between-builds.html, but the .semaphore-cache was just empty :-/

Either way you'll have it by (your) tmrw morning :)

Thanks for the awesome PR.

💃🙌

[mzikherman]

I expired the cache and seems to have worked!

https://semaphoreci.com/artsy-it/metaphysics/settings/admin (need to be logged in as it@)

[alloy]

Yay!!

[ermik]

What do you guys think of using UUIDv5 as __id-workaround across different services instead of (just) base64-encoding a string? (Let's just say I can enforce truly unique GUID generation across different data stores, so collision is solved.)

(Oh didn't mean to assign people, sorry.)

[alloy]

@ermik It wouldn’t solve our issue, which is that the id field was already in use (and non-global) and we couldn’t change that without breaking some clients’ assumptions. Aside from that, I do like being able to encode whatever is needed to reach the same node again, which we also do with some complex filter fields that take many params.

[ermik]

@alloy sounds interesting. It's almost like a case of a funky collision turned into an interesting pattern. Thanks for the insight. I'm trying to grasp the ontology of your system to advance my code — learning from mistakes (and smarts) of others.

[ermik]

How do I use the object identification mechanism you have in this project on stitched schemas?

For example, I have two separate services both supporting relay and exposing Node root field, how do I stitch them together and resolve the node query in a metaphysics-like app? I'd really appreciate any guidance or tips on this. Thank you! ❤️

[alloy]

@ermik We have not been doing this yet. My idea was that the different services would encode the IDs themselves but prepend the IDs with their service name. Then the stitched schema could have a resolver that delegates to the responsible backend service by just matching the start of the IDs with base64 encoded versions of the services names (be sure to take into account how base64 encoding works, an example can be found here).

[ermik]

So, since I will need more answers from you guys 😉, I hope this will be interesting for you to see. I actually didn't get your response until just now and hacked it together the best i could.

First off, an I might PR this in the future, but I found that this repo is, as far as I understand is missing something like the following:

/**
 * update-master-schema.js
 * Grabs the stitched schema to allow Relay compiler and other tools
 * dependent on schema definition files to work.
 */
import fs from "fs";
import { printSchema } from "graphql/utilities";
import path from "path";
import { mergeSchemas } from "../server/graphql/lib/stitching/mergedSchemas";

let schema;
async function magic() {
  schema = await mergeSchemas();
  fs.writeFileSync(
    path.join(__dirname, "../server/graphql/master.graphql"),
    printSchema(schema)
  );
}

magic();

Then, mangling the beauty that is your code:

/*
 * megedSchemas.js
 * Prepares the edge layer to resolve graphql requests to the internal APIs by 
 * using schema merge, delegation, and transformation tooling 
 */

/* ... */

  const mergedSchema = _mergeSchemas({
    schemas: [
      lloydSchema,
      localSchema
      /**/
    ],
    resolvers: {
      Query: {
        // delegating directly, no subschemas or mergeInfo
        node: (parent, args, context, info) => {
          return info.mergeInfo.delegateToSchema({
            schema: lloydSchema,
            operation: "query",
            fieldName: "node",
            args: {
              id: args.id
            },
            context,
            info
          });
        }
      }
    }

/* ... */

Which works for a single service, but must not be hard (I assume) to make it work for multiple via the same mapping mechanism you use for loaders for example.

[alloy]

Indeed we’re not persisting the stitched schema, we’ve started using tooling such as graphql-fetch-schema to fetch when the client needs it. Thanks for offering to PR, though!

As for the delegation, my thought was to do something like:

        node: (parent, args, context, info) => {
          let schema
          // Encode schema name prefix in such a way that it takes into account Base64 encoding boundaries.
          if (args.id.startsWith(base64("lloyd"))) {
            schema = lloydSchema
          }
          return info.mergeInfo.delegateToSchema({
            schema,
            operation: "query",
            fieldName: "node",
            args: {
              id: args.id
            },
            context,
            info
          });
        }

[ermik]

There is a need for a contract that adheres to the graphql/relay spec. The node field itself holds the promise of identifying any object that implements the node interface by only it's ID. So when we look upstream, that same promise, for a given type, is the held by the particular service — never two at once (same as in case of eventual consistency needing a master of some sort). Any further linkage is well-served by service-to-service connections e.g. gRPC & Thrift which is how we can pick up the slack of resolving deeply nested/relational queries: get the user from "users" service, but also get his likes from the "likes" service can be resolved by the users service alone. (maybe an antipattern tho, don't know, but it seems to be very much like the illustration in Artsy tech blog) Correct me if I am mistaken, but afaik GraphQL is of no help here, as it doesn't resolve children until parent query comes back.

Based on these considerations I'd keep the decision of picking a schema it in a controlled format of a static map:

/** @flow */

import { fromGlobalId } from "graphql-relay";
import type { GraphQLSchema } from "graphql";

type ObjectType = string;
type Service = string;
type NodeMap = { [ObjectType]: Service };

const nodeResolvers: NodeMap = {
  ["Vendor"]: "lloyd",
  ["ElasticSearch"]: "local"
};

/**
 * @func schemaFromGlobalId retrieves delegate schema for a Relay Global ID
 * @param {string} id — Relay standard base64 encoded tuple of shape "type:id"
 * @desc instead of base64() cost if encoding servicename as part of type
 * we will pay expected base64() cost associated with Relay services; it will
 * also be paid at the destination, but that is also expected.
 */
const schemaFromGlobalId = (id: string): GraphQLSchema | void => {
  try {
    // fromGlobalId is unsafe, and generally this is flaky
    switch (nodeResolvers[fromGlobalId(id).type]) {
      case "lloyd":
        return lloydSchema;
      case "local":
        return localSchema;
      default:
        throw new Error("UnknownService");
    }
  } catch (e) {
    // sentry, etc.
    throw IdentificationFailureError(); // catch and recover thread elsewhere
  }
};

/* ... */
        node: (parent, args, context, info) => {
          const schema = schemaFromGlobalId(args.id);
          return info.mergeInfo.delegateToSchema({
            schema,
            operation: "query",
            fieldName: "node",
            args: {
              id: args.id
            },
            context,
            info
          });
        }

Introducing additional base64 data into an already clunky (imho) system of word followed by colon followed by actual id seems rough around the egdes to me.

Another issue with encoding the service name as part of the nodeGlobalID is that it would have to be an adjustment to either typename itself or the id of the entity.

1. If you look at how graphql-relay package resolves types from ID it is clear to me that I cannot prepend anything to the type: it is defined as "everything until the first colon". This would mean encoding the service name as part of type itself.
2. Prepending the service name to all service-local IDs at the graphql endpoint seems plausible, and provides an additonal check of "don't care about this request cus that's not my name" — but at the same time if we sent it there in the first place why don't we terminate the mess of resolving types during stitching at the point where stitching occurs then working cleanly further upstream.

[alloy]

For Relay, the only requirement is that a Node ID is globally unique and that the GraphQL server can resolve back to a unique entity from it. A strong suggestion is to make this ID opaque to the client.

One way to make the ID opaque but encode the required data, as is used by the graphql-relay package, is to Base64 encode type:id, but it could really be anything. You don’t need to use the functions provided by graphql-relay. My thought for our stitched schema is that the upstream services encode service:type:id; then metaphysics doesn’t even need to decode those IDs when the client requests them, it can just match the first part of the encoded ID with pre-cached Base64 encoded versions of the services.