OAS validation passes given an invalid JSON Schema document

[Drew-Kimberly]

Context

We use Prism for API contract testing. I've uncovered some false-positives where our contract tests pass despite API behavior that's divergent from the OpenAPI spec. After some debugging, I discovered that the normalized JSON schema document associated to a relevant IHttpOperation was invalid JSON Schema. AJV fails to compile given the invalid schema and the error is eaten here, resulting in a facade of confidence as the validation routine passes.

The troubling thing is that, since this library transforms/normalizes the OAS, a valid spec can result in an invalid JSON Schema document. In our case I've identified one such issue and submitted a fix: stoplightio/http-spec#271. The validation should immediately fail when the input JSON schema is invalid.

Current Behavior

validateAgainstSchema does not throw nor return validation errors when the input JSON Schema is invalid.

Expected Behavior

validateAgainstSchema should either throw or return a validation error when the input JSON Schema is invalid.

Steps to Reproduce

Here's a test (body.spec.ts) that demonstrates the issue:

describe('invalid schema', () => {
  it('throws when the schema is invalid', () => {
    // @ts-expect-error invalid schema for test case
    const mockSchema: JSONSchema = { type: 'foo' };

      assertLeft(
        validate(
          'test',
          [{ id: faker.random.word(), mediaType: 'application/json', schema: mockSchema, examples: [], encodings: [] }],
          ValidationContext.Input,
          'application/json'
        ),
        error =>
          expect(error).toBeDefined()
      );
  })
})

By introspecting the error that's eaten we see:

Error: schema is invalid: data/type must be equal to one of the allowed values, data/type must be array, data/type must match a schema in anyOf