Skip to content

Commit

Permalink
Pluggable documentStore replacing experimental option (#5644)
Browse files Browse the repository at this point in the history
Co-authored-by: David Glasser <glasser@davidglasser.net>
Co-authored-by: Stephen Barlow <stephen@apollographql.com>
  • Loading branch information
3 people committed Oct 5, 2021
1 parent 4efa414 commit b2c37ae
Show file tree
Hide file tree
Showing 9 changed files with 209 additions and 66 deletions.
17 changes: 15 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,21 @@ The version headers in this history reflect the versions of Apollo Server itself
- [`@apollo/gateway`](https://github.com/apollographql/federation/blob/HEAD/gateway-js/CHANGELOG.md)
- [`@apollo/federation`](https://github.com/apollographql/federation/blob/HEAD/federation-js/CHANGELOG.md)

## vNEXT

## vNEXT (minor)

- `apollo-server-core`: You can now specify your own `DocumentStore` (a `KeyValueStore<DocumentNode>`) for Apollo Server's cache of parsed and validated GraphQL operation abstract syntax trees via the new `documentStore` constructor option. This replaces the `experimental_approximateDocumentStoreMiB` option. You can replace `new ApolloServer({experimental_approximateDocumentStoreMiB: approximateDocumentStoreMiB, ...moreOptions})` with:
```typescript
import { InMemoryLRUCache } from 'apollo-server-caching';
import type { DocumentNode } from 'graphql';
new ApolloServer({
documentStore: new InMemoryLRUCache<DocumentNode>({
maxSize: Math.pow(2, 20) * approximateDocumentStoreMiB,
sizeCalculator: InMemoryLRUCache.jsonBytesSizeCalculator,
}),
...moreOptions,
})
```
[PR #5644](https://github.com/apollographql/apollo-server/pull/5644) [Issue #5634](https://github.com/apollographql/apollo-server/issues/5634)
- `apollo-server-core`: For ease of testing, you can specify the node environment via `new ApolloServer({nodeEnv})` in addition to via the `NODE_ENV` environment variable. The environment variable is now only read during server startup (and in some error cases) rather than on every request. [PR #5657](https://github.com/apollographql/apollo-server/pull/5657)
- `apollo-server-koa`: The peer dependency on `koa` (added in v3.0.0) should be a `^` range dependency rather than depending on exactly one version, and it should not be automatically increased when new versions of `koa` are released. [PR #5759](https://github.com/apollographql/apollo-server/pull/5759)
- `apollo-server-fastify`: Export `ApolloServerFastifyConfig` and `FastifyContext` TypeScript types. [PR #5743](https://github.com/apollographql/apollo-server/pull/5743)
Expand Down
70 changes: 41 additions & 29 deletions docs/source/api/apollo-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,47 @@ An array containing custom functions to use as additional [validation rules](htt
</tr>


<tr>
<td>

##### `documentStore`

`KeyValueCache<DocumentNode>` or `null`
</td>
<td>

A key-value cache that Apollo Server uses to store previously encountered GraphQL operations (as `DocumentNode`s). It does _not_ store query _results_.

Whenever Apollo Server receives an incoming operation, it checks whether that exact operation is present in its `documentStore`. If it's present, Apollo Server can safely skip parsing and validating the operation, thereby improving performance.

The default `documentStore` is an [`InMemoryLRUCache`](https://github.com/apollographql/apollo-server/blob/main/packages/apollo-server-caching/src/InMemoryLRUCache.ts) with an approximate size of 30MiB. This is usually sufficient unless the server processes a large number of unique operations. Provide this option if you want to change the cache size or store the cache information in an alternate location.

To use `InMemoryLRUCache` but change its size to an amount `approximateDocumentStoreMiB`:

<div style="max-width: 400px;">

```typescript
import { InMemoryLRUCache } from 'apollo-server-caching';
import type { DocumentNode } from 'graphql';
new ApolloServer({
documentStore: new InMemoryLRUCache<DocumentNode>({
maxSize: Math.pow(2, 20) * approximateDocumentStoreMiB,
sizeCalculator: InMemoryLRUCache.jsonBytesSizeCalculator,
}),
})
```

</div>

**Do not share a `documentStore` between multiple `ApolloServer` instances**, _unless_ you assign a unique prefix to each instance's entries (for example, using [`PrefixingKeyValueCache`](https://github.com/apollographql/apollo-server/blob/main/packages/apollo-server-caching/src/PrefixingKeyValueCache.ts)). Apollo Server skips parsing and validating any operation that's present in its `documentStore`, so if servers with _different_ schemas share the _same_ `documentStore`, a server might execute an operation that its schema doesn't support.

Pass `null` to disable this cache entirely.

Available in Apollo Server v3.4.0 and later.
</td>
</tr>


<tr>
<td colspan="2">

Expand Down Expand Up @@ -406,35 +447,6 @@ If this is set to any string value, use that value instead of the environment va
</tbody>
</table>

### Experimental options

**These options are experimental.** They might be removed or change at any time, even within a patch release.

<table class="field-table">
<thead>
<tr>
<th>Name /<br/>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>

##### `experimental_approximateDocumentStoreMiB`

`number`
</td>
<td>

Sets the approximate size (in MiB) of the server's `DocumentNode` cache. The server checks the SHA-256 hash of each incoming operation against cached `DocumentNode`s, and skips unnecessary parsing and validation if a match is found.

The cache's default size is 30MiB, which is usually sufficient unless the server processes a large number of unique operations.
</td>
</tr>
</tbody>
</table>

### Middleware-specific `context` fields

The `context` object passed between Apollo Server resolvers automatically includes certain fields, depending on which [Node.js middleware](../integrations/middleware/) you're using:
Expand Down
7 changes: 7 additions & 0 deletions packages/apollo-server-caching/src/InMemoryLRUCache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,11 @@ export class InMemoryLRUCache<V = string> implements KeyValueCache<V> {
async getTotalSize() {
return this.store.length;
}

// This is a size calculator based on the number of bytes in a JSON
// encoding of the stored object. It happens to be what ApolloServer
// uses for its default DocumentStore and may be helpful to others as well.
static jsonBytesSizeCalculator<T>(obj: T): number {
return Buffer.byteLength(JSON.stringify(obj), 'utf8');
}
}
32 changes: 16 additions & 16 deletions packages/apollo-server-core/src/ApolloServer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import type {
Config,
Context,
ContextFunction,
DocumentStore,
PluginDefinition,
} from './types';

Expand Down Expand Up @@ -71,17 +72,13 @@ const NoIntrospection = (context: ValidationContext) => ({
},
});

function approximateObjectSize<T>(obj: T): number {
return Buffer.byteLength(JSON.stringify(obj), 'utf8');
}

export type SchemaDerivedData = {
schema: GraphQLSchema;
schemaHash: SchemaHash;
// A store that, when enabled (default), will store the parsed and validated
// versions of operations in-memory, allowing subsequent parses/validates
// on the same operation to be executed immediately.
documentStore?: InMemoryLRUCache<DocumentNode>;
documentStore: DocumentStore | null;
};

type ServerState =
Expand Down Expand Up @@ -144,7 +141,6 @@ export class ApolloServerBase<
private toDispose = new Set<() => Promise<void>>();
private toDisposeLast = new Set<() => Promise<void>>();
private drainServers: (() => Promise<void>) | null = null;
private experimental_approximateDocumentStoreMiB: Config['experimental_approximateDocumentStoreMiB'];
private stopOnTerminationSignals: boolean;
private landingPage: LandingPage | null = null;

Expand All @@ -171,7 +167,7 @@ export class ApolloServerBase<
// requestOptions.
mocks,
mockEntireSchema,
experimental_approximateDocumentStoreMiB,
documentStore,
...requestOptions
} = this.config;

Expand Down Expand Up @@ -206,8 +202,6 @@ export class ApolloServerBase<

this.parseOptions = parseOptions;
this.context = context;
this.experimental_approximateDocumentStoreMiB =
experimental_approximateDocumentStoreMiB;

const isDev = this.config.nodeEnv !== 'production';

Expand Down Expand Up @@ -671,13 +665,18 @@ export class ApolloServerBase<
private generateSchemaDerivedData(schema: GraphQLSchema): SchemaDerivedData {
const schemaHash = generateSchemaHash(schema!);

// Initialize the document store. This cannot currently be disabled.
const documentStore = this.initializeDocumentStore();

return {
schema,
schemaHash,
documentStore,
// The DocumentStore is schema-derived because we put documents in it after
// checking that they pass GraphQL validation against the schema and use
// this to skip validation as well as parsing. So we can't reuse the same
// DocumentStore for different schemas because that might make us treat
// invalid operations as valid.
documentStore:
this.config.documentStore === undefined
? this.initializeDocumentStore()
: this.config.documentStore,
};
}

Expand Down Expand Up @@ -875,9 +874,10 @@ export class ApolloServerBase<
// only using JSON.stringify on the DocumentNode (and thus doesn't account
// for unicode characters, etc.), but it should do a reasonable job at
// providing a caching document store for most operations.
maxSize:
Math.pow(2, 20) * (this.experimental_approximateDocumentStoreMiB || 30),
sizeCalculator: approximateObjectSize,
//
// If you want to tweak the max size, pass in your own documentStore.
maxSize: Math.pow(2, 20) * 30,
sizeCalculator: InMemoryLRUCache.jsonBytesSizeCalculator,
});
}

Expand Down
116 changes: 116 additions & 0 deletions packages/apollo-server-core/src/__tests__/documentStore.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
import gql from 'graphql-tag';
import type { DocumentNode } from 'graphql';

import { ApolloServerBase } from '../ApolloServer';
import { InMemoryLRUCache } from 'apollo-server-caching';

const typeDefs = gql`
type Query {
hello: String
}
`;

const resolvers = {
Query: {
hello() {
return 'world';
},
},
};

// allow us to access internals of the class
class ApolloServerObservable extends ApolloServerBase {
override graphQLServerOptions() {
return super.graphQLServerOptions();
}
}

const documentNodeMatcher = {
kind: 'Document',
definitions: expect.any(Array),
loc: {
start: 0,
end: 15,
},
};

const operations = {
simple: {
op: { query: 'query { hello }' },
hash: 'ec2e01311ab3b02f3d8c8c712f9e579356d332cd007ac4c1ea5df727f482f05f',
},
};

describe('ApolloServerBase documentStore', () => {
it('documentStore - undefined', async () => {
const server = new ApolloServerObservable({
typeDefs,
resolvers,
});

await server.start();

const options = await server.graphQLServerOptions();
const embeddedStore = options.documentStore as any;
expect(embeddedStore).toBeInstanceOf(InMemoryLRUCache);

await server.executeOperation(operations.simple.op);

expect(await embeddedStore.getTotalSize()).toBe(403);
expect(await embeddedStore.get(operations.simple.hash)).toMatchObject(
documentNodeMatcher,
);
});

it('documentStore - custom', async () => {
const documentStore = {
get: async function (key: string) {
return cache[key];
},
set: async function (key: string, val: DocumentNode) {
cache[key] = val;
},
delete: async function () {},
};
const cache: Record<string, DocumentNode> = {};

const getSpy = jest.spyOn(documentStore, 'get');
const setSpy = jest.spyOn(documentStore, 'set');

const server = new ApolloServerBase({
typeDefs,
resolvers,
documentStore,
});
await server.start();

await server.executeOperation(operations.simple.op);

expect(Object.keys(cache)).toEqual([operations.simple.hash]);
expect(cache[operations.simple.hash]).toMatchObject(documentNodeMatcher);

await server.executeOperation(operations.simple.op);

expect(Object.keys(cache)).toEqual([operations.simple.hash]);

expect(getSpy.mock.calls.length).toBe(2);
expect(setSpy.mock.calls.length).toBe(1);
});

it('documentStore - null', async () => {
const server = new ApolloServerObservable({
typeDefs,
resolvers,
documentStore: null,
});

await server.start();

const options = await server.graphQLServerOptions();
expect(options.documentStore).toBe(null);

const result = await server.executeOperation(operations.simple.op);

expect(result.data).toEqual({ hello: 'world' });
});
});
12 changes: 3 additions & 9 deletions packages/apollo-server-core/src/__tests__/runQuery.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1107,12 +1107,6 @@ describe('runQuery', () => {
return '{\n' + query + '}';
}

// This should use the same logic as the calculation in InMemoryLRUCache:
// https://github.com/apollographql/apollo-server/blob/94b98ff3/packages/apollo-server-caching/src/InMemoryLRUCache.ts#L23
function approximateObjectSize<T>(obj: T): number {
return Buffer.byteLength(JSON.stringify(obj), 'utf8');
}

it('validates each time when the documentStore is not present', async () => {
expect.assertions(4);

Expand Down Expand Up @@ -1167,12 +1161,12 @@ describe('runQuery', () => {
// size of the two smaller queries. All three of these queries will never
// fit into this cache, so we'll roll through them all.
const maxSize =
approximateObjectSize(parse(querySmall1)) +
approximateObjectSize(parse(querySmall2));
InMemoryLRUCache.jsonBytesSizeCalculator(parse(querySmall1)) +
InMemoryLRUCache.jsonBytesSizeCalculator(parse(querySmall2));

const documentStore = new InMemoryLRUCache<DocumentNode>({
maxSize,
sizeCalculator: approximateObjectSize,
sizeCalculator: InMemoryLRUCache.jsonBytesSizeCalculator,
});

await runRequest({ plugins, documentStore, queryString: querySmall1 });
Expand Down
5 changes: 3 additions & 2 deletions packages/apollo-server-core/src/graphqlOptions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import type {
GraphQLFormattedError,
ParseOptions,
} from 'graphql';
import type { KeyValueCache, InMemoryLRUCache } from 'apollo-server-caching';
import type { KeyValueCache } from 'apollo-server-caching';
import type { DataSource } from 'apollo-datasource';
import type { ApolloServerPlugin } from 'apollo-server-plugin-base';
import type {
Expand All @@ -18,6 +18,7 @@ import type {
Logger,
SchemaHash,
} from 'apollo-server-types';
import type { DocumentStore } from './types';

/*
* GraphQLServerOptions
Expand Down Expand Up @@ -56,7 +57,7 @@ export interface GraphQLServerOptions<
cache?: KeyValueCache;
persistedQueries?: PersistedQueryOptions;
plugins?: ApolloServerPlugin[];
documentStore?: InMemoryLRUCache<DocumentNode>;
documentStore?: DocumentStore | null;
parseOptions?: ParseOptions;
nodeEnv?: string;
}
Expand Down

0 comments on commit b2c37ae

Please sign in to comment.