Add the ability for scalars to be powered by sqlTables

join-monster excels at retrieving exactly the fields asked for by a given GraphQL query, but sometimes, there isn't an exact 1-1 correspondence between the desired fields in a GraphQL API and the table structure powering the system. Sometimes, data is normalized in the database or broken out into many tables for performance or reporting reasons or whatever, and join-monster supports these non 1-1 mappings fine with sqlExpr, alwaysFetch, and custom resolvers just fine, except when the "virtual" field is a scalar.

I've ended up having to use a few custom scalars in my application, mostly from the `graphql-scalars` package as well as a couple other random ones, and I have one that is powered by a whole other table in the actual database schema. As an example, I have a `Post` object with normal fields that map to columns, but then a `tags` field on the Post that should return a string of tags, where the Tag is a custom scalar to apply some special parsing and formatting to the tag. The tags are stored in their own tags table in the database. To expose this using `join-monster`, I could create a whole new `GraphQLObjectType` to represent Tags and then have a `{name, id}` field or whatever, but I feel like that is changing the design of the API to accomodate join-monster (and GraphQL to an extent), instead of making the nicest API for developers, which would just be a plain old array of strings.

So, I'd like to be able to do a `GraphQLList` of `Tag`, where `Tag` is a `new GraphQLScalar`, and have `join-monster` traverse that join and fetch my tags from the database. Turns out the only thing preventing this was the whitelist of types that `join-monster` looks for `sqlTable` on, so that's the only actual functional change in this PR.

This is probably a pretty rare thing, but it's super handy for these custom scalars, as well as the JSON scalar escape hatch that folks might use to escape the strict typing shackles of GraphQL when they need to.

docs/CHANGELOG.md

@@ -109,6 +109,26 @@ const User = new GraphQLObjectType({
 
 The old `sortKey` synax (as an object looking like `{key, order}`) continues to work but is no longer recommended. The new syntax allows for reliable sortKey ordering as well as independent directions per sort key, with all the complicated SQL generation handled for you.
 
+- Add support for resolving GraphQL scalars backed by `sqlTable`s. Usually, scalars point to table columns, but this allows them to use the same `extensions` property to declare that the scalar is a whole table. This is often paired with a `resolve` function that takes what `join-monster` returns and turns it into a valid value for the scalar. An example would be a tags field that outputs a list of strings, but where each tag is actually stored as it's own row in a different table in the database, or backing a `JSONScalar` by a table to get around GraphQL's type strictness.
+
+```javascript
+export const Tag = new GraphQLScalarType({
+  name: 'Tag',
+  extensions: {
+    joinMonster: {
+      sqlTable: 'tags',
+      uniqueKey: 'id',
+      alwaysFetch: ['id', 'tag', 'tag_order']
+    }
+  },
+  parseValue: String,
+  serialize: String,
+  parseLiteral(ast) {
+    // ...
+  }
+})
+```
+
 ### v2.1.2 (May 25, 2020)
 
 #### Fixed

docs/map-to-table.md

@@ -88,3 +88,31 @@ const User = new GraphQLObjectType({
   })
 })
 ```
+
+## Using scalars instead of objects
+
+Rarely, you may have a value in your GraphQL API that's best represented as a scalar value instead of an object with fields, like a special string or a JSON scalar. `GraphQLScalar`s can also be extended such that `join-monster` will retrieve them using SQL joins or batches.
+
+As an example, we could set up a `Post` object, powered by a `posts` table, that has a `tags` field which is powered by a whole other `tags` table. The `Tag` scalar might be a custom `GraphQLScalar` like so:
+
+```javascript
+const Tag = new GraphQLScalarType({
+  name: 'Tag',
+  extensions: {
+    joinMonster: {
+      sqlTable: 'tags',
+      uniqueKey: 'id',
+      alwaysFetch: ['id', 'tag_name']
+    }
+  },
+  parseValue: String,
+  serialize: String,
+  parseLiteral(ast) {
+    // ...
+  }
+})
+```
+
+which configures `join-monster` to fetch tags from the `tags` table, and to always fetch the `tag_name` column.
+
+The `Post` object can then join `Tag`s just like any other `join-monster` powered object, using either a connection or a plain `GraphQLList`. See the section on [joins](/start-joins) for more details.

src/index.d.ts

@@ -100,6 +100,12 @@ export interface InterfaceTypeExtension {
   alwaysFetch?: string | string[]
 }
 
+export interface ScalarTypeExtension {
+  sqlTable?: string
+  uniqueKey?: string | string[]
+  alwaysFetch?: string | string[]
+}
+
 declare module 'graphql' {
   interface GraphQLObjectTypeExtensions<TSource = any, TContext = any> {
     joinMonster?: ObjectTypeExtension<TSource, TContext>
@@ -117,6 +123,9 @@ declare module 'graphql' {
   interface GraphQLInterfaceTypeExtensions {
     joinMonster?: InterfaceTypeExtension
   }
+  interface GraphQLScalarTypeExtensions {
+    joinMonster?: ScalarTypeExtension
+  }
 }
 
 // JoinMonster lib interface

src/query-ast-to-sql-ast/index.js

@@ -31,7 +31,8 @@ class SQLASTNode {
 const TABLE_TYPES = [
   'GraphQLObjectType',
   'GraphQLUnionType',
-  'GraphQLInterfaceType'
+  'GraphQLInterfaceType',
+  'GraphQLScalarType'
 ]
 
 function mergeAll(fieldNodes) {

test-api/data/schema/mysql.sql

@@ -53,3 +53,12 @@ CREATE TABLE sponsors (
   created_at DATETIME(3) DEFAULT CURRENT_TIMESTAMP(3)
 );
 
+
+DROP TABLE IF EXISTS tags;
+CREATE TABLE tags (
+  id INT AUTO_INCREMENT PRIMARY KEY,
+  post_id INTEGER NOT NULL,
+  tag VARCHAR(255),
+  tag_order INTEGER DEFAULT 1,
+  created_at DATETIME(3) DEFAULT CURRENT_TIMESTAMP(3)
+);

test-api/data/schema/oracle.sql

@@ -86,3 +86,16 @@ CREATE TABLE "sponsors" (
   "created_at" TIMESTAMP DEFAULT CURRENT_TIMESTAMP
 )
 
+
+begin
+execute immediate 'drop table "tags" purge';
+exception when others then null;
+end;
+
+CREATE TABLE "tags" (
+  "id" NUMBER ,
+  "post_id" NUMBER DEFAULT ,
+  "tag" VARCHAR(255),
+  "tag_order" NUMBER DEFAULT ,
+  "created_at" TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);

test-api/data/schema/pg.sql

@@ -53,3 +53,11 @@ CREATE TABLE sponsors (
   created_at TIMESTAMPTZ DEFAULT NOW()
 );
 
+DROP TABLE IF EXISTS tags;
+CREATE TABLE tags (
+  id SERIAL PRIMARY KEY,
+  post_id INTEGER NOT NULL,
+  tag VARCHAR(255),
+  tag_order INTEGER DEFAULT 1,
+  created_at TIMESTAMPTZ DEFAULT NOW()
+);

test-api/data/schema/sqlite3.sql

@@ -53,3 +53,11 @@ CREATE TABLE sponsors (
   created_at DEFAULT CURRENT_TIMESTAMP
 );
 
+DROP TABLE IF EXISTS tags;
+CREATE TABLE tags (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  post_id INTEGER NOT NULL,
+  tag VARCHAR(255),
+  tag_order INTEGER DEFAULT 1,
+  created_at DEFAULT CURRENT_TIMESTAMP
+);

test-api/data/setup/demo.js

@@ -11,6 +11,7 @@ const numPosts = 50
 const numComments = 300
 const numRelationships = 15
 const numLikes = 300
+const numTags = 200
 
 module.exports = async db => {
   const knex = await require('../schema/setup')(db, 'demo')
@@ -117,5 +118,17 @@ module.exports = async db => {
     }
   ])
 
+  console.log('creating tags...')
+  const tags = new Array(numTags)
+  for (let i of count(numTags)) {
+    tags[i] = {
+      tag: faker.random.word(),
+      post_id: faker.random.number({ min: 1, max: numPosts }),
+      tag_order: i,
+      created_at: faker.date.past()
+    }
+  }
+  await knex.batchInsert('tags', tags, 50)
+
   await knex.destroy()
 }

test-api/data/setup/test1.js

@@ -182,5 +182,28 @@ module.exports = async db => {
     }
   ])
 
+  await knex.batchInsert('tags', [
+    {
+      post_id: 1,
+      tag: 'foo',
+      tag_order: 1
+    },
+    {
+      post_id: 1,
+      tag: 'bar',
+      tag_order: 2
+    },
+    {
+      post_id: 1,
+      tag: 'baz',
+      tag_order: 3
+    },
+    {
+      post_id: 2,
+      tag: 'foo',
+      tag_order: 1
+    }
+  ])
+
   await knex.destroy()
 }

test-api/schema-paginated/Post.js

@@ -2,7 +2,8 @@ import {
   GraphQLObjectType,
   GraphQLString,
   GraphQLInt,
-  GraphQLBoolean
+  GraphQLBoolean,
+  GraphQLList
 } from 'graphql'
 
 import {
@@ -15,6 +16,7 @@ import {
 
 import { User } from './User'
 import { CommentConnection } from './Comment'
+import { Tag, TagConnection } from './Tag'
 import { Authored } from './Authored/Interface'
 import { nodeInterface } from './Node'
 import { q, bool } from '../shared'
@@ -161,6 +163,80 @@ export const Post = new GraphQLObjectType({
           sqlColumn: 'created_at'
         }
       }
+    },
+    tags: {
+      type: new GraphQLList(Tag),
+      resolve: source => {
+        return source.tags.map(tag => tag.tag)
+      },
+      extensions: {
+        joinMonster: {
+          orderBy: 'tag_order',
+          ...(STRATEGY === 'batch'
+            ? {
+                sqlBatch: {
+                  thisKey: 'id',
+                  parentKey: 'post_id'
+                }
+              }
+            : {
+                sqlJoin: (postTable, tagTable) =>
+                  `${postTable}.${q('id', DB)} = ${tagTable}.${q(
+                    'post_id',
+                    DB
+                  )}`
+              })
+        }
+      }
+    },
+    tagsConnection: {
+      type: TagConnection,
+      resolve: source => source.tags.map(tag => tag.tag),
+      extensions: {
+        joinMonster: {
+          orderBy: 'tag_order',
+          ...(STRATEGY === 'batch'
+            ? {
+                sqlBatch: {
+                  thisKey: 'id',
+                  parentKey: 'post_id'
+                }
+              }
+            : {
+                sqlJoin: (postTable, tagTable) =>
+                  `${postTable}.${q('id', DB)} = ${tagTable}.${q(
+                    'post_id',
+                    DB
+                  )}`
+              })
+        }
+      }
+    },
+    firstTag: {
+      type: Tag,
+      resolve: source => {
+        return source.tags[0].tag
+      },
+      extensions: {
+        joinMonster: {
+          limit: 1,
+          orderBy: 'tag_order',
+          ...(STRATEGY === 'batch'
+            ? {
+                sqlBatch: {
+                  thisKey: 'id',
+                  parentKey: 'post_id'
+                }
+              }
+            : {
+                sqlJoin: (postTable, tagTable) =>
+                  `${postTable}.${q('id', DB)} = ${tagTable}.${q(
+                    'post_id',
+                    DB
+                  )}`
+              })
+        }
+      }
     }
   })
 })

test-api/schema-paginated/QueryRoot.js

@@ -13,6 +13,7 @@ import {
 
 import knex from './database'
 import { User, UserConnection } from './User'
+import { Post } from './Post'
 import Sponsor from './Sponsor'
 import { nodeField } from './Node'
 import ContextPost from './ContextPost'
@@ -138,6 +139,31 @@ export default new GraphQLObjectType({
         )
       }
     },
+    post: {
+      type: Post,
+      args: {
+        id: {
+          description: 'The posts ID number',
+          type: GraphQLInt
+        }
+      },
+      extensions: {
+        joinMonster: {
+          where: (postsTable, args) => {
+            // eslint-disable-line no-unused-vars
+            if (args.id) return `${postsTable}.${q('id', DB)} = ${args.id}`
+          }
+        }
+      },
+      resolve: (parent, args, context, resolveInfo) => {
+        return joinMonster(
+          resolveInfo,
+          context,
+          sql => dbCall(sql, knex, context),
+          options
+        )
+      }
+    },
     sponsors: {
       type: new GraphQLList(Sponsor),
       resolve: (parent, args, context, resolveInfo) => {

test-api/schema-paginated/Tag.js

@@ -0,0 +1,36 @@
+import { GraphQLInt, GraphQLScalarType, Kind } from 'graphql'
+import { q } from '../shared'
+import { connectionDefinitions } from 'graphql-relay'
+const { PAGINATE, DB } = process.env
+
+// This is to test Scalars being extended with join-monster properties and is not a great way to actually model tags
+export const Tag = new GraphQLScalarType({
+  name: 'Tag',
+  description: 'Custom scalar representing a tag',
+  extensions: {
+    joinMonster: {
+      sqlTable: `(SELECT * FROM ${q('tags', DB)})`,
+      uniqueKey: 'id',
+      alwaysFetch: ['id', 'tag', 'tag_order']
+    }
+  },
+  parseValue: String,
+  serialize: String,
+  parseLiteral(ast) {
+    if (ast.kind === Kind.STRING) {
+      return ast.value
+    }
+    return null
+  }
+})
+
+const connectionConfig = { nodeType: Tag }
+if (PAGINATE === 'offset') {
+  connectionConfig.connectionFields = {
+    total: { type: GraphQLInt }
+  }
+}
+const { connectionType: TagConnection } = connectionDefinitions(
+  connectionConfig
+)
+export { TagConnection }