Use case analysis for multi-entity resolution in federation

This repo exists to serve as an exploration of the possible ways that we can overcome the N+1 problem in entity resolution.

Case study

The front-facing API structure has a query, showProducts, that is meant to be like a shop front, it returns a list of products.

# overall composed schema
type Query {
    showProducts: [Product!]
}

The Product subgraph owns the databases, etc related to products.

# product subgraph
type Product {
    id: ID!
    variations: [Variation!]
}

type Variation {
    id: ID!
    imageUrl: String!
    price: Float!
}

Products can have variations, in that e.g. a wallet might come in blue or black. There's no guarantee that the variations have the same price either. But on the front page we should be showing only one variation.

We therefore have a presentation subgraph that internally is connected to some ML model. The ML model can take a list of variations, and the user ID (from auth token), then choose a particular variation to recommend to the user. E.g. it might choose blue for people who like the colour blue, black otherwise.

The ML model is expensive to run for single inferences and is better in batch. So our aim is to give it a 2D array of [][]Variation, have it choose a particular variation for each, then return []Variation which would be the recommended variations for the users.

# presentation subgraph
type Product @key(fields: "id") {
    id: ID!
    display: Variation! @requires(fields: "variations {price imageUrl id}")
}

type Variation {
    id: ID!
    imageUrl: String!
    price: Float!
}

The website will then make the query to get the variation to render the page:

query shoppage {
  showProducts {
    ... on Product {
      id
      display {
        id
        imageUrl
        price
      }
    }
  }
}

The examples in the folders above will focus on the presentation subgraph and many different attempts to generate code using gqlgen to achieve the desired outcome.

The key requirement here is that some part of the function scope must have access to all of the @required fields from all of the products at once.

The examples will be using the documented recipes for federation on gqlgen@v0.17.89, the latest version at the time of writing.

Running the examples

cd into the example directory and:

go run server.go

In another terminal, run the entity resolution curl:

curl --request POST \
  --url http://localhost:8080/query \
  --header 'content-type: application/json' \
  --data '{"query":"query entityresolution(  $representations: [_Any!]!) {  _entities(representations: $representations) {    ... on Product {      id      display {        id      }    }  }}","variables":{  "representations": [    {      "__typename": "Product",      "id": "1",      "variations": [        {          "id": "1a",          "price": 1.00,          "imageUrl": "1a.png"        },        {          "id": "1b",          "price": 2.00,          "imageUrl": "1b.png"        }      ]    },    {      "__typename": "Product",      "id": "2",      "variations": [        {          "id": "2a",          "price": 3.00,          "imageUrl": "2a.png"        },        {          "id": "2b",          "price": 4.00,          "imageUrl": "2b.png"        }      ]    }  ]}}'