diff --git a/API_DOCS.md b/API_DOCS.md new file mode 100644 index 0000000..d3ce6d6 --- /dev/null +++ b/API_DOCS.md @@ -0,0 +1,136 @@ +# dataloader-codegen API Documentation + +This is the full documentation for the options you can pass to dataloader-codegen. + +## Command Line Interface + +```bash +$ dataloader-codegen --config string --output string +``` + +See `$ dataloader-codegen --help` for all available options. + +## Generated `getLoaders` function + +The output file that dataloader-codegen generates exports a default `getLoaders` function. + +You probably want to call `getLoaders` per network request, and attatch the loaders to the context object for the resolvers to access. + +### API + +```js +getLoaders(resources[, options]) +``` + +### `getLoaders` arguments + +- **`resources`** + + An object containing _resources_. A resource is typically a wrapper around a fetch statement if you're already using [swagger codegen](https://github.com/OpenAPITools/openapi-generator) or something similar, but these can be any function that returns data. + + You must describe the shape and behaviour of the resources you want to use the config file (documented below). + +- **`options`** + + (Optional) Object containing options to augment the runtime behaviour of the loaders. + + See the type definition here: https://github.com/Yelp/dataloader-codegen/blob/6ce10/src/codegen.ts#L85-L90 + + - **`errorHandler`** + + (Optional) Provide a function to wrap the underlying resource call. Useful if you want to handle 'expected' errors (e.g. 4xxs, 5xxs) before handing over to the resolver method. + + **Interface:** + + ```js + errorHandler(resourcePath: Array, error: Error): Promise + ``` + + - **`resourceMiddleware`** + + (Optional) Object containing functions to run before and after the resource runs. + + - **`before`** + + (Optional) Takes in the arguments about to be passed to the resource. This is a good place to log + calls to resources / transform the request. + + **Interface**: + + ```js + before(resourcePath: Array, resourceArgs: T): Promise + ``` + + - **`after`** + + (Optional) Takes in the response from the resource. Returns a modified response. + + **Interface**: + + ```js + after(resourcePath: Array, response: T): Promise + ``` + +### Example + +To see an example call to `getLoaders`, [check out the SWAPI example](./examples/swapi/swapi-server.js) or [the tests](./__tests__/implementation.test.js). + +## Config File + +The config file should be a [YAML](https://yaml.org/) file in the following format: + +```yaml +resources: + string: + [...string:] + isBatchResource: boolean + docsLink: string + batchKey: string (can only use if isBatchResource=true) + newKey: string (can only use if isBatchResource=true) + reorderResultsKey: ?string (can only use if isBatchResource=true) + nestedPath: ?string (can only use if isBatchResource=true) + commaSeparatedBatchKey: ?string (can only use if isBatchResource=true) + isResponseDictionary: ?boolean (can only use if isBatchResource=true) + +typings: + language: flow + embedResourcesType: + imports: string + ResourcesType: string +``` + +### Example + +To see an example config, [check out the SWAPI example](./examples/swapi/swapi.dataloader-config.yaml). + +### `resources` + +Describes the shape and behaviour of the resources object you will pass to `getLoaders`. Supports an arbitrary level of nesting. + +**Note:** You only need to specify the resources that you want to generate loaders for - you don't need to describe _everything_ that's in the resources object if you only need a subset. + +#### `resources` Parameters + +| Key | Value Description | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `isBatchResource` | Is this a batch resource? (Can you pass it a list of keys and get a list of results back?) | +| `docsLink` | The URL for the documentation of the resource. Useful for others to verify information is correct, and may be used in stack traces. | +| `batchKey` | The argument to the resource that represents the list of entities we want to fetch. (e.g. 'user_ids') | +| `newKey` | The argument we'll replace the batchKey with - should be a singular version of the `batchKey` (e.g. 'user_id') | +| `reorderResultsByKey` | (Optional) If the resource itself does not guarantee ordering, use this to specify which key in the response objects corresponds to an element in `batchKey`. Transforms and re-order the response to the same order as requested from the DataLoaders. | +| `nestedPath` | (Optional) If the resource returns the list of results in a nested path (e.g. `{ results: [ 1, 2, 3 ] }`), this tells the DataLoader where in the response to find the results. (e.g. 'results'). | +| `commaSeparatedBatchKey` | (Optional) Set to true if the interface of the resource takes the batch key as a comma separated list (rather than an array of IDs, as is more common). Default: false | +| `isResponseDictionary` | (Optional) Set to true if the batch resource returns the results as a dictionary with key mapped to values (instead of a list of items). If this option is supplied `reorderResultsByKey` should not be. Default: false | + +### `typings` + +Use this to generate type definitions for the generated DataLoaders. At this time, we only support Flow. (Please open an issue if you're interested in helping us support TypeScript!) + +#### `typings` Parameters + +| Key | Value Description | +| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| `language` | Must be 'flow' until we support other options. | +| `embedResourcesType.imports` | Lets you inject an arbitrary import statement into the generated file, to help you write the type statement below. | +| `embedResourcesType.ResourcesType` | Inject code to describe the shape of the resources object you're going to pass into `getLoaders`. Should start with `type ResourcesType = ...` | +| | diff --git a/README.md b/README.md index 8b70a48..3f1e096 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ and can make a more efficient set of HTTP requests to the underlying resource. ## Usage -1. Create `dataloader-config.yaml` to describe the shape and behaviour of your resources. (See the docs for more info.) +1. Create `dataloader-config.yaml` to describe the shape and behaviour of your resources. (See [the docs](./API_DOCS.md) for detailed info.) **Example** @@ -71,16 +71,19 @@ and can make a more efficient set of HTTP requests to the underlying resource. 2. Call `dataloader-codegen` and pass in your config file: ```bash - $ dataloader-codegen --config swapi.dataloader-config.yaml --output swapi-loaders.js + $ dataloader-codegen --config swapi.dataloader-config.yaml --output __codegen__/swapi-loaders.js ``` See `--help` for more options. -3. Import the generated loaders and use them in your [resolver methods](https://www.apollographql.com/docs/graphql-tools/resolvers/) as normal! +3. Import the generated loaders and use them in your [resolver methods](https://www.apollographql.com/docs/graphql-tools/resolvers/): ```js - // StarWarsAPI returns a clientlib containing fetch calls to swapi.co - const swapiLoaders = createSwapiLoaders.default(StarWarsAPI()); + import getLoaders from './__codegen__/swapi-loaders'; + + // StarWarsAPI is a clientlib containing fetch calls to swapi.co + // getLoaders is the function that dataloader-codegen generates for us + const swapiLoaders = getLoaders(StarWarsAPI); class Planet { constructor(id) { @@ -95,7 +98,7 @@ and can make a more efficient set of HTTP requests to the underlying resource. } ``` - _(See the [swapi example](./examples/swapi/swapi-server.js) to see this in context.)_ + Check out the [swapi example](./examples/swapi/swapi-server.js) to see a working example of this. ## Batch Resources @@ -140,7 +143,7 @@ resources: newKey: user_id ``` -See the docs for more information on how to configure resources. +See [the full docs](./API_DOCS.md) for more information on how to configure resources. ## Contributing diff --git a/__tests__/implementation.test.js b/__tests__/implementation.test.js index 83dfe28..03d7fe9 100644 --- a/__tests__/implementation.test.js +++ b/__tests__/implementation.test.js @@ -25,9 +25,9 @@ tmp.setGracefulCleanup(); jest.setTimeout(10000); expect.extend({ - toBeError(received, message = '') { + toBeError(received, message = '', errorType = 'Error') { const regex = _.isRegExp(message) ? message : new RegExp(_.escapeRegExp(message)); - const pass = _.isError(received) && regex.test(received.message); + const pass = _.isError(received) && regex.test(received.message) && received.name === errorType; if (pass) { return { message: () => `expected ${received} not to be an error with message matching: ${message}}`, @@ -35,7 +35,7 @@ expect.extend({ }; } else { return { - message: () => `expected ${received} to be an error with message matching: ${message}}`, + message: () => `expected ${received} to be ${errorType} with message matching: ${message}}`, pass: false, }; } @@ -292,7 +292,133 @@ test('batch endpoint (multiple requests)', async () => { }); }); -test('batch endpoint (multiple requests, error handling)', async () => { +test('batch endpoint that throws errors', async () => { + const config = { + resources: { + foo: { + isBatchResource: true, + docsLink: 'example.com/docs/bar', + batchKey: 'foo_ids', + newKey: 'foo_id', + }, + }, + }; + + const resources = { + foo: ({ foo_ids, include_extra_info }) => { + if (_.isEqual(foo_ids, [1, 3])) { + expect(include_extra_info).toBe(false); + throw new Error('yikes'); + } + + if (_.isEqual(foo_ids, [2, 4, 5])) { + expect(include_extra_info).toBe(true); + return Promise.resolve([ + { + foo_id: 2, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + { + foo_id: 4, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + { + foo_id: 5, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + ]); + } + }, + }; + + await createDataLoaders(config, async getLoaders => { + const loaders = getLoaders(resources); + + const results = await loaders.foo.loadMany([ + { foo_id: 1, include_extra_info: false }, + { foo_id: 2, include_extra_info: true }, + { foo_id: 3, include_extra_info: false }, + { foo_id: 4, include_extra_info: true }, + { foo_id: 5, include_extra_info: true }, + ]); + + expect(results).toMatchObject([ + expect.toBeError(/yikes/), + { foo_id: 2, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + expect.toBeError(/yikes/), + { foo_id: 4, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + { foo_id: 5, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + ]); + }); +}); + +test('batch endpoint that rejects', async () => { + const config = { + resources: { + foo: { + isBatchResource: true, + docsLink: 'example.com/docs/bar', + batchKey: 'foo_ids', + newKey: 'foo_id', + }, + }, + }; + + const resources = { + foo: ({ foo_ids, include_extra_info }) => { + if (_.isEqual(foo_ids, [1, 3])) { + expect(include_extra_info).toBe(false); + return Promise.reject('yikes'); + } + + if (_.isEqual(foo_ids, [2, 4, 5])) { + expect(include_extra_info).toBe(true); + return Promise.resolve([ + { + foo_id: 2, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + { + foo_id: 4, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + { + foo_id: 5, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + ]); + } + }, + }; + + await createDataLoaders(config, async getLoaders => { + const loaders = getLoaders(resources); + + const results = await loaders.foo.loadMany([ + { foo_id: 1, include_extra_info: false }, + { foo_id: 2, include_extra_info: true }, + { foo_id: 3, include_extra_info: false }, + { foo_id: 4, include_extra_info: true }, + { foo_id: 5, include_extra_info: true }, + ]); + + expect(results).toMatchObject([ + expect.toBeError(/yikes/, 'NonError'), + { foo_id: 2, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + expect.toBeError(/yikes/, 'NonError'), + { foo_id: 4, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + { foo_id: 5, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + ]); + }); +}); + +test('batch endpoint (multiple requests, default error handling)', async () => { const config = { resources: { foo: { @@ -355,6 +481,74 @@ test('batch endpoint (multiple requests, error handling)', async () => { }); }); +test('batch endpoint (multiple requests, custom error handling)', async () => { + function errorHandler(resourcePath, error) { + expect(resourcePath).toEqual(['foo']); + return new Error('hello from custom error handler'); + } + + const config = { + resources: { + foo: { + isBatchResource: true, + docsLink: 'example.com/docs/bar', + batchKey: 'foo_ids', + newKey: 'foo_id', + }, + }, + }; + + const resources = { + foo: ({ foo_ids, include_extra_info }) => { + if (_.isEqual(foo_ids, [1, 3])) { + expect(include_extra_info).toBe(false); + return new Error('yikes'); + } + + if (_.isEqual(foo_ids, [2, 4, 5])) { + expect(include_extra_info).toBe(true); + return Promise.resolve([ + { + foo_id: 2, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + { + foo_id: 4, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + { + foo_id: 5, + foo_value: 'greetings', + extra_stuff: 'lorem ipsum', + }, + ]); + } + }, + }; + + await createDataLoaders(config, async getLoaders => { + const loaders = getLoaders(resources, { errorHandler }); + + const results = await loaders.foo.loadMany([ + { foo_id: 1, include_extra_info: false }, + { foo_id: 2, include_extra_info: true }, + { foo_id: 3, include_extra_info: false }, + { foo_id: 4, include_extra_info: true }, + { foo_id: 5, include_extra_info: true }, + ]); + + expect(results).toMatchObject([ + expect.toBeError(/hello from custom error handler/), + { foo_id: 2, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + expect.toBeError(/hello from custom error handler/), + { foo_id: 4, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + { foo_id: 5, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, + ]); + }); +}); + test('batch endpoint (multiple requests, error handling, nestedPath)', async () => { const config = { eek: true, @@ -594,9 +788,18 @@ test('batch endpoint without reorderResultsByKey throws error for response with ]); expect(results).toMatchObject([ - expect.toBeError('[dataloader-codegen :: foo] Resource returned 2 items, but we requested 3 items'), - expect.toBeError('[dataloader-codegen :: foo] Resource returned 2 items, but we requested 3 items'), - expect.toBeError('[dataloader-codegen :: foo] Resource returned 2 items, but we requested 3 items'), + expect.toBeError( + '[dataloader-codegen :: foo] Resource returned 2 items, but we requested 3 items', + 'BatchItemNotFoundError', + ), + expect.toBeError( + '[dataloader-codegen :: foo] Resource returned 2 items, but we requested 3 items', + 'BatchItemNotFoundError', + ), + expect.toBeError( + '[dataloader-codegen :: foo] Resource returned 2 items, but we requested 3 items', + 'BatchItemNotFoundError', + ), { foo_id: 4, foo_value: 'greetings' }, ]); }); @@ -655,7 +858,10 @@ test('batch endpoint with reorderResultsByKey handles response with non existant expect(results).toMatchObject([ { foo_id: 1, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, - expect.toBeError('[dataloader-codegen :: foo] Response did not contain item with foo_id = 2'), + expect.toBeError( + '[dataloader-codegen :: foo] Response did not contain item with foo_id = 2', + 'BatchItemNotFoundError', + ), { foo_id: 3, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, { foo_id: 4, foo_value: 'greetings', extra_stuff: 'lorem ipsum' }, ]); @@ -727,8 +933,69 @@ test('batch endpoint with isResponseDictionary handles a response that returns a const results = await loaders.foo.loadMany([{ foo_id: 1 }, { foo_id: 2 }, { foo_id: 3 }]); expect(results).toEqual([ { foo_id: 1, foo_value: 'hello' }, - expect.toBeError('[dataloader-codegen :: foo] Could not find key = "2" in the response dict'), + expect.toBeError( + '[dataloader-codegen :: foo] Could not find key = "2" in the response dict', + 'BatchItemNotFoundError', + ), + { foo_id: 3, foo_value: '!' }, + ]); + }); +}); + +test('middleware can transform the request args and the resource response', async () => { + function before(resourcePath, resourceArgs) { + expect(resourcePath).toEqual(['foo']); + expect(resourceArgs).toEqual([{ foo_ids: [100, 200, 300] }]); + + // modify the arguments to the resource + return [{ foo_ids: [1, 2, 3] }]; + } + + function after(resourcePath, response) { + expect(resourcePath).toEqual(['foo']); + expect(response).toEqual([ + { foo_id: 1, foo_value: 'hello' }, + { foo_id: 2, foo_value: 'world' }, { foo_id: 3, foo_value: '!' }, ]); + + return [ + { foo_id: 1, foo_value: 'goodbye' }, + { foo_id: 2, foo_value: 'world' }, + { foo_id: 3, foo_value: '?' }, + ]; + } + + const config = { + resources: { + foo: { + isBatchResource: true, + docsLink: 'example.com/docs/bar', + batchKey: 'foo_ids', + newKey: 'foo_id', + }, + }, + }; + + const resources = { + foo: ({ foo_ids }) => { + expect(foo_ids).toEqual([1, 2, 3]); + return Promise.resolve([ + { foo_id: 1, foo_value: 'hello' }, + { foo_id: 2, foo_value: 'world' }, + { foo_id: 3, foo_value: '!' }, + ]); + }, + }; + + await createDataLoaders(config, async getLoaders => { + const loaders = getLoaders(resources, { resourceMiddleware: { before, after } }); + + const results = await loaders.foo.loadMany([{ foo_id: 100 }, { foo_id: 200 }, { foo_id: 300 }]); + expect(results).toEqual([ + { foo_id: 1, foo_value: 'goodbye' }, + { foo_id: 2, foo_value: 'world' }, + { foo_id: 3, foo_value: '?' }, + ]); }); }); diff --git a/package.json b/package.json index 38626b1..2b09264 100644 --- a/package.json +++ b/package.json @@ -48,6 +48,7 @@ "aggregate-error": "^3.0.1", "ajv": "^6.11.0", "dataloader": "^2.0.0", + "ensure-error": "^2.0.0", "js-yaml": "^3.13.1", "lodash": "^4.17.15", "object-hash": "^2.0.0", diff --git a/src/codegen.ts b/src/codegen.ts index 68adf2c..dddfed9 100644 --- a/src/codegen.ts +++ b/src/codegen.ts @@ -54,6 +54,7 @@ export default function codegen( import _ from 'lodash'; import invariant from 'assert'; + import ensureError from 'ensure-error'; import DataLoader from 'dataloader'; import { BatchItemNotFoundError, @@ -83,9 +84,10 @@ export default function codegen( type ExtractPromisedReturnValue = ((...A) => Promise) => R; export type DataLoaderCodegenOptions = {| + errorHandler?: (resourcePath: $ReadOnlyArray, error: Error) => Promise, resourceMiddleware?: {| - before?: (resourcePath: $ReadOnlyArray, resourceArgs: T) => T, - after?: (resourcePath: $ReadOnlyArray, response: T) => T, + before?: (resourcePath: $ReadOnlyArray, resourceArgs: T) => Promise, + after?: (resourcePath: $ReadOnlyArray, response: T) => Promise, |}; |}; diff --git a/src/implementation.ts b/src/implementation.ts index e184f85..71e2ea4 100644 --- a/src/implementation.ts +++ b/src/implementation.ts @@ -137,48 +137,71 @@ function getBatchLoader(resourceConfig: BatchResourceConfig, resourcePath: Reado }]; if (options && options.resourceMiddleware && options.resourceMiddleware.before) { - resourceArgs = await options.resourceMiddleware.before(${JSON.stringify( - resourcePath, - )}, resourceArgs); + resourceArgs = await options.resourceMiddleware.before( + ${JSON.stringify(resourcePath)}, + resourceArgs + ); + } + + let response; + try { + // Finally, call the resource! + response = await ${resourceReference}(...resourceArgs); + } catch (error) { + /** + * Apply some default error handling to catch and handle all errors/rejected promises. + * + * If we let errors here go unhandled here, it will bubble up and DataLoader will return an error for all + * keys requested. We can do slightly better by returning the error object for just the keys in this batch request. + * + * We use ensureError because 'error' might actually be a string or something in the case of a rejected promise. + */ + response = ensureError(error); } - // Finally, call the resource! - let response = await ${resourceReference}(...resourceArgs); + // If there's a custom error handler, do something with the error + if (options && options.errorHandler && response instanceof Error) { + response = await options.errorHandler( + ${JSON.stringify(resourcePath)}, + response + ); + } if (options && options.resourceMiddleware && options.resourceMiddleware.after) { - response = await options.resourceMiddleware.after(${JSON.stringify( - resourcePath, - )}, response); + response = await options.resourceMiddleware.after( + ${JSON.stringify(resourcePath)}, + response + ); } `; })()} - ${(() => { - if (typeof resourceConfig.nestedPath === 'string') { - return ` - /** - * Un-nest the actual data from the resource return value. - * - * e.g. - * \`\`\`js - * { - * foos: [ - * { id: 1, value: 'hello' }, - * { id: 2, value: 'world' }, - * ] - * } - * \`\`\` - * - * Becomes - * - * \`\`\`js - * [ - * { id: 1, value: 'hello' }, - * { id: 2, value: 'world' }, - * ] - * \`\`\` - */ - if (!(response instanceof Error)) { + if (!(response instanceof Error)) { + ${(() => { + if (typeof resourceConfig.nestedPath === 'string') { + return ` + /** + * Un-nest the actual data from the resource return value. + * + * e.g. + * \`\`\`js + * { + * foos: [ + * { id: 1, value: 'hello' }, + * { id: 2, value: 'world' }, + * ] + * } + * \`\`\` + * + * Becomes + * + * \`\`\`js + * [ + * { id: 1, value: 'hello' }, + * { id: 2, value: 'world' }, + * ] + * \`\`\` + */ response = _.get( response, '${resourceConfig.nestedPath}', @@ -191,12 +214,12 @@ function getBatchLoader(resourceConfig: BatchResourceConfig, resourcePath: Reado ].join(' ') ), ); - } - `; - } else { - return ''; - } - })()} + `; + } else { + return ''; + } + })()} + } if (!(response instanceof Error)) { ${(() => { diff --git a/src/runtimeHelpers.ts b/src/runtimeHelpers.ts index f4fcb0c..00e045d 100644 --- a/src/runtimeHelpers.ts +++ b/src/runtimeHelpers.ts @@ -30,6 +30,13 @@ export class BatchItemNotFoundError extends Error { } } +/** + * An error class used internally to wrap an error returned from a batch resource call. + * Should be caught and handled internally - never exposed to the outside world. + * When created, we store the `reorderResultsByValue` - this lets the ordering logic know + * where in the return array to place this object. (We do this so we can add extra attributes + * to the error object in a typesafe way) + */ export class CaughtResourceError extends Error { cause: Error; reorderResultsByValue: string | number | null; diff --git a/yarn.lock b/yarn.lock index 3f8b434..919684f 100644 --- a/yarn.lock +++ b/yarn.lock @@ -1942,6 +1942,11 @@ end-of-stream@^1.1.0: dependencies: once "^1.4.0" +ensure-error@^2.0.0: + version "2.0.0" + resolved "https://registry.yarnpkg.com/ensure-error/-/ensure-error-2.0.0.tgz#b8359a992601601b3541af9472f6a49d9dca1458" + integrity sha512-1ela4oR5A+TdtFpfiQrZKFUbsOi4JuIYmz2qSGFar6pEdRa54E15mKHVVYrAq1OQhd6b6nVrCaQxQlo6kYwhaw== + es-abstract@^1.5.1: version "1.15.0" resolved "https://registry.yarnpkg.com/es-abstract/-/es-abstract-1.15.0.tgz#8884928ec7e40a79e3c9bc812d37d10c8b24cc57"