openapi: improve validation testing (#2058)

## What

This PR adds an extra layer of OpenAPI validation testing to what we already have. It also fixes any issues that make the new tests fail.

## Why

While the current OpenAPI validation takes care of _some_ things, there's also things it misses, as shown by #2055. By adding the OpenAPI Enforcer package, we should hopefully be able to catch more of these errors in the future. The enforcer _does_ flag the issue in #2055 as an error.

## How

By adding the OpenAPI Enforcer package and making whatever changes it picks up on.

By adding location headers to all our 201 endpoints. I also had to change some signatures on `create` store methods so that they actually return something (a lot of them just returned `void`).

## Discussion points

### Code changes

Some of the code changes may not be necessary or we may want to change more code to align with what changes have been done. It may be worth standardizing on a pattern for `*store.create` methods, so that they always return an identifier or the stored object, for instance.

### On relative URIs

The 201 location headers use relative URIs to point to the created resources. This seemed to be the easiest way to me, as we don't need to worry about figuring out what the absolute URL of the instance is (though we could probably just concat it to the request URI?). The algorithm for determining relative URIs is described in [RFC 3986 section 5](https://www.rfc-editor.org/rfc/rfc3986#section-5).

There's also some places where I'm not sure we _can_ provide accurate location url. I think they're supposed to point _directly at_ whatever the resource is, but for some resources (such as api tokens), you can only get it as part of a collection. From [RFC 9110 on the location field](https://httpwg.org/specs/rfc9110.html#field.location) (emphasis mine):

> the Location header field in a [201 (Created)](https://httpwg.org/specs/rfc9110.html#status.201) response is supposed to provide a URI that is **specific** to the created resource.

A link to a collection is not specific. I'm not sure what best to do about this.

### Inline comments

I've added a number of inline PR comments that I'd love to get some feedback on too. Have a look and let me know what you think!

### Unfinished business

I've added some juicy comments to some of the files here. They contain non-blocking issues that I'm tracking (via github issues). We should resolve them in the future, but right now they can stay as they are. 

## Commits

* Feat: add openapi-enforcer + tests; fix _some_ issues

* Test: allow non-standard string formats

* validation: fix _some_ 201 created location header endpoints

* #1391: fix remaining 201 location headers missing

* Refactor: use the ajv options object instead of add* methods

* #1391: flag validation errors and warnings as test failures

* #1391: modify patch schema to specify either object or array

We don't provide many patch endpoints, so we _could_ create separate
patch operation objects for each one. I think that makes sense to do
as part of the larger cleanup. For now, I think it's worth to simply
turn it into one of these. While it's not entirely accurate, it's
better than what we had before.

* Refactor: make tests easier to read

* #1391: use enum for valid token types

This was previously only a description. This may seem like a breaking
change because OpenAPI would previously accept any string. However,
Joi also performs validation on this, so invalid values wouldn't work
previously either.

* #1391: Comment out default parameter values for now

This isn't the _right_ way, but it's the pragmatic solution. It's not
a big deal and this works as a stopgap solution.

* #1391:  add todo note for api token schema fixes

* #1391: update snapshot

* Revert "#1391: modify patch schema to specify either object or array"

This reverts commit 0dd5d0f.

Turns out we need to allow much more than just objects and arrays.
I'll leave this as is for now.

* Chore(#1391): update comment explaining api token schema TODO

* #1391: modify some test code and add comment

* #1391: update tests and spec to allow 'any' type in schema

* Chore: remove comment

* #1391: add tests for context field stores

* #1391: add location header for public signup links

* #1391: fix query parameter description.

There was indeed a bug in the package, and this has been addressed
now, so we can go back to describing the params properly.

package.json

@@ -174,6 +174,7 @@
     "jest": "29.0.1",
     "lint-staged": "13.0.3",
     "nock": "13.2.9",
+    "openapi-enforcer": "1.21.1",
     "prettier": "2.7.1",
     "proxyquire": "2.1.3",
     "source-map-support": "0.5.21",

src/lib/db/context-field-store.ts

@@ -4,6 +4,7 @@ import {
     IContextField,
     IContextFieldDto,
     IContextFieldStore,
+    ILegalValue,
 } from '../types/stores/context-field-store';
 
 const COLUMNS = [
@@ -16,7 +17,16 @@ const COLUMNS = [
 ];
 const TABLE = 'context_fields';
 
-const mapRow: (object) => IContextField = (row) => ({
+type ContextFieldDB = {
+    name: string;
+    description: string;
+    stickiness: boolean;
+    sort_order: number;
+    legal_values: ILegalValue[];
+    created_at: Date;
+};
+
+const mapRow = (row: ContextFieldDB): IContextField => ({
     name: row.name,
     description: row.description,
     stickiness: row.stickiness,
@@ -88,15 +98,17 @@ class ContextFieldStore implements IContextFieldStore {
         return present;
     }
 
+    // TODO: write tests for the changes you made here?
     async create(contextField: IContextFieldDto): Promise<IContextField> {
-        const row = await this.db(TABLE)
+        const [row] = await this.db(TABLE)
             .insert(this.fieldToRow(contextField))
             .returning('*');
+
         return mapRow(row);
     }
 
     async update(data: IContextFieldDto): Promise<IContextField> {
-        const row = await this.db(TABLE)
+        const [row] = await this.db(TABLE)
             .where({ name: data.name })
             .update(this.fieldToRow(data))
             .returning('*');

src/lib/openapi/spec/api-token-schema.ts

@@ -15,7 +15,7 @@ export const apiTokenSchema = {
         },
         type: {
             type: 'string',
-            description: `${Object.values(ApiTokenType).join(', ')}.`,
+            enum: Object.values(ApiTokenType),
         },
         environment: {
             type: 'string',

src/lib/openapi/spec/create-api-token-schema.ts

@@ -1,6 +1,22 @@
 import { FromSchema } from 'json-schema-to-ts';
 import { ApiTokenType } from '../../types/models/api-token';
 
+// TODO: (openapi) this schema isn't entirely correct: `project` and `projects`
+// are mutually exclusive.
+//
+// That is, when creating a token, you can provide either `project` _or_
+// `projects`, but *not* both.
+//
+// We should be able to annotate this using `oneOf` and `allOf`, but making
+// `oneOf` only valid for _either_ `project` _or_ `projects` is tricky.
+//
+// I've opened an issue to get some help (thought it was a bug initially).
+// There's more info available at:
+//
+// https://github.com/ajv-validator/ajv/issues/2096
+//
+// This also applies to apiTokenSchema and potentially other related schemas.
+
 export const createApiTokenSchema = {
     $id: '#/components/schemas/createApiTokenSchema',
     type: 'object',
@@ -14,7 +30,7 @@ export const createApiTokenSchema = {
         },
         type: {
             type: 'string',
-            description: `${Object.values(ApiTokenType).join(', ')}.`,
+            enum: Object.values(ApiTokenType),
         },
         environment: {
             type: 'string',

src/lib/openapi/spec/set-ui-config-schema.ts

@@ -12,7 +12,6 @@ export const setUiConfigSchema = {
             properties: {
                 frontendApiOrigins: {
                     type: 'array',
-                    additionalProperties: false,
                     items: { type: 'string' },
                 },
             },

src/lib/openapi/util/create-response-schema.ts

@@ -14,3 +14,27 @@ export const createResponseSchema = (
         },
     };
 };
+
+export const resourceCreatedResponseSchema = (
+    schemaName: string,
+): OpenAPIV3.ResponseObject => {
+    return {
+        headers: {
+            location: {
+                description: 'The location of the newly created resource.',
+                schema: {
+                    type: 'string',
+                    format: 'uri',
+                },
+            },
+        },
+        description: `The resource was successfully created.`,
+        content: {
+            'application/json': {
+                schema: {
+                    $ref: `#/components/schemas/${schemaName}`,
+                },
+            },
+        },
+    };
+};

src/lib/openapi/validate.ts

@@ -1,5 +1,4 @@
 import Ajv, { ErrorObject } from 'ajv';
-import addFormats from 'ajv-formats';
 import { SchemaId, schemas } from './index';
 import { omitKeys } from '../util/omit-keys';
 
@@ -12,13 +11,15 @@ const ajv = new Ajv({
     schemas: Object.values(schemas).map((schema) =>
         omitKeys(schema, 'components'),
     ),
-});
-
-addFormats(ajv, ['date-time']);
 
-// example was superseded by examples in openapi 3.1, but we're still on 3.0, so
-// let's add it back in!
-ajv.addKeyword('example');
+    // example was superseded by examples in openapi 3.1, but we're still on 3.0, so
+    // let's add it back in!
+    keywords: ['example'],
+    formats: {
+        'date-time': true,
+        uri: true,
+    },
+});
 
 export const validateSchema = (
     schema: SchemaId,

src/lib/routes/admin-api/api-token.ts

@@ -19,7 +19,10 @@ import { createApiToken } from '../../schema/api-token-schema';
 import { OpenApiService } from '../../services/openapi-service';
 import { IUnleashServices } from '../../types';
 import { createRequestSchema } from '../../openapi/util/create-request-schema';
-import { createResponseSchema } from '../../openapi/util/create-response-schema';
+import {
+    createResponseSchema,
+    resourceCreatedResponseSchema,
+} from '../../openapi/util/create-response-schema';
 import {
     apiTokensSchema,
     ApiTokensSchema,
@@ -96,7 +99,7 @@ export class ApiTokenController extends Controller {
                     operationId: 'createApiToken',
                     requestBody: createRequestSchema('createApiTokenSchema'),
                     responses: {
-                        201: createResponseSchema('apiTokenSchema'),
+                        201: resourceCreatedResponseSchema('apiTokenSchema'),
                     },
                 }),
             ],
@@ -162,6 +165,7 @@ export class ApiTokenController extends Controller {
             res,
             apiTokenSchema.$id,
             serializeDates(token),
+            { location: `api-tokens` },
         );
     }
 

src/lib/routes/admin-api/context.ts

@@ -24,7 +24,10 @@ import {
 import { ContextFieldsSchema } from '../../openapi/spec/context-fields-schema';
 import { UpsertContextFieldSchema } from '../../openapi/spec/upsert-context-field-schema';
 import { createRequestSchema } from '../../openapi/util/create-request-schema';
-import { createResponseSchema } from '../../openapi/util/create-response-schema';
+import {
+    createResponseSchema,
+    resourceCreatedResponseSchema,
+} from '../../openapi/util/create-response-schema';
 import { serializeDates } from '../../types/serialize-dates';
 import NotFoundError from '../../error/notfound-error';
 import { NameSchema } from '../../openapi/spec/name-schema';
@@ -98,7 +101,9 @@ export class ContextController extends Controller {
                         'upsertContextFieldSchema',
                     ),
                     responses: {
-                        201: emptyResponse,
+                        201: resourceCreatedResponseSchema(
+                            'contextFieldSchema',
+                        ),
                     },
                 }),
             ],
@@ -189,13 +194,19 @@ export class ContextController extends Controller {
 
     async createContextField(
         req: IAuthRequest<void, void, UpsertContextFieldSchema>,
-        res: Response,
+        res: Response<ContextFieldSchema>,
     ): Promise<void> {
         const value = req.body;
         const userName = extractUsername(req);
 
-        await this.contextService.createContextField(value, userName);
-        res.status(201).end();
+        const result = await this.contextService.createContextField(
+            value,
+            userName,
+        );
+        res.status(201)
+            .header('location', `context/${result.name}`)
+            .json(serializeDates(result))
+            .end();
     }
 
     async updateContextField(

src/lib/routes/admin-api/feature.ts

@@ -25,7 +25,10 @@ import { TagsSchema } from '../../openapi/spec/tags-schema';
 import { serializeDates } from '../../types/serialize-dates';
 import { OpenApiService } from '../../services/openapi-service';
 import { createRequestSchema } from '../../openapi/util/create-request-schema';
-import { createResponseSchema } from '../../openapi/util/create-response-schema';
+import {
+    createResponseSchema,
+    resourceCreatedResponseSchema,
+} from '../../openapi/util/create-response-schema';
 import { emptyResponse } from '../../openapi/util/standard-responses';
 
 const version = 1;
@@ -123,7 +126,9 @@ class FeatureController extends Controller {
                     tags: ['Features'],
                     operationId: 'addTag',
                     requestBody: createRequestSchema('tagSchema'),
-                    responses: { 201: createResponseSchema('tagSchema') },
+                    responses: {
+                        201: resourceCreatedResponseSchema('tagSchema'),
+                    },
                 }),
             ],
         });
@@ -221,7 +226,7 @@ class FeatureController extends Controller {
             req.body,
             userName,
         );
-        res.status(201).json(tag);
+        res.status(201).header('location', `${featureName}/tags`).json(tag);
     }
 
     // TODO

src/lib/routes/admin-api/public-signup.ts

@@ -8,7 +8,10 @@ import { IAuthRequest } from '../unleash-types';
 import { IUnleashConfig, IUnleashServices } from '../../types';
 import { OpenApiService } from '../../services/openapi-service';
 import { createRequestSchema } from '../../openapi/util/create-request-schema';
-import { createResponseSchema } from '../../openapi/util/create-response-schema';
+import {
+    createResponseSchema,
+    resourceCreatedResponseSchema,
+} from '../../openapi/util/create-response-schema';
 import { serializeDates } from '../../types/serialize-dates';
 import { emptyResponse } from '../../openapi/util/standard-responses';
 import { PublicSignupTokenService } from '../../services/public-signup-token-service';
@@ -92,7 +95,9 @@ export class PublicSignupController extends Controller {
                         'publicSignupTokenCreateSchema',
                     ),
                     responses: {
-                        201: createResponseSchema('publicSignupTokenSchema'),
+                        201: resourceCreatedResponseSchema(
+                            'publicSignupTokenSchema',
+                        ),
                     },
                 }),
             ],
@@ -255,6 +260,7 @@ export class PublicSignupController extends Controller {
             res,
             publicSignupTokensSchema.$id,
             serializeDates(token),
+            { location: `tokens/${token.secret}` },
         );
     }
 

src/lib/routes/admin-api/strategy.ts

@@ -15,7 +15,10 @@ import { IAuthRequest } from '../unleash-types';
 import { OpenApiService } from '../../services/openapi-service';
 import { emptyResponse } from '../../openapi/util/standard-responses';
 import { createRequestSchema } from '../../openapi/util/create-request-schema';
-import { createResponseSchema } from '../../openapi/util/create-response-schema';
+import {
+    createResponseSchema,
+    resourceCreatedResponseSchema,
+} from '../../openapi/util/create-response-schema';
 import {
     strategySchema,
     StrategySchema,
@@ -102,7 +105,9 @@ class StrategyController extends Controller {
                     tags: ['Strategies'],
                     operationId: 'createStrategy',
                     requestBody: createRequestSchema('upsertStrategySchema'),
-                    responses: { 201: emptyResponse },
+                    responses: {
+                        201: resourceCreatedResponseSchema('strategySchema'),
+                    },
                 }),
             ],
         });
@@ -193,12 +198,18 @@ class StrategyController extends Controller {
 
     async createStrategy(
         req: IAuthRequest<unknown, UpsertStrategySchema>,
-        res: Response<void>,
+        res: Response<StrategySchema>,
     ): Promise<void> {
         const userName = extractUsername(req);
 
-        await this.strategyService.createStrategy(req.body, userName);
-        res.status(201).end();
+        const strategy = await this.strategyService.createStrategy(
+            req.body,
+            userName,
+        );
+        res.header('location', `strategies/${strategy.name}`)
+            .status(201)
+            .json(strategy)
+            .end();
     }
 
     async updateStrategy(

src/lib/routes/admin-api/tag-type.ts

@@ -13,7 +13,10 @@ import TagTypeService from '../../services/tag-type-service';
 import { Logger } from '../../logger';
 import { IAuthRequest } from '../unleash-types';
 import { createRequestSchema } from '../../openapi/util/create-request-schema';
-import { createResponseSchema } from '../../openapi/util/create-response-schema';
+import {
+    createResponseSchema,
+    resourceCreatedResponseSchema,
+} from '../../openapi/util/create-response-schema';
 import { TagTypesSchema } from '../../openapi/spec/tag-types-schema';
 import { ValidateTagTypeSchema } from '../../openapi/spec/validate-tag-type-schema';
 import {
@@ -66,7 +69,9 @@ class TagTypeController extends Controller {
                 openApiService.validPath({
                     tags: ['Tags'],
                     operationId: 'createTagType',
-                    responses: { 201: createResponseSchema('tagTypeSchema') },
+                    responses: {
+                        201: resourceCreatedResponseSchema('tagTypeSchema'),
+                    },
                     requestBody: createRequestSchema('tagTypeSchema'),
                 }),
             ],
@@ -164,7 +169,9 @@ class TagTypeController extends Controller {
             req.body,
             userName,
         );
-        res.status(201).json(tagType);
+        res.status(201)
+            .header('location', `tag-types/${tagType.name}`)
+            .json(tagType);
     }
 
     async updateTagType(