Support for oneOf keyword

[Nickcw6]

Following up from this thread: https://pact-foundation.slack.com/archives/CLS16AVEE/p1681405717867409

Would it be possible to support oneOf/anyOf keywords in OAS schemas please? Or failing that, any guidance on workarounds? The current documentation implies this should be possible in a similar manner to allOf, but unfortunately this still results in a bunch of should NOT have additional properties errors.

There does seem to be a workaround by adding additionalProperties: true wherever there a oneOf keyword is used, but this in doing so you're losing some of the benefits contract testing provides.

Have attached an example OAS file, a merged & derefernced version (using this tool) & Pact file: oas-pact-oneof-example.zip

Command ran for comparison:

swagger-mock-validator oas/vets_example_oas-transformed.json pacts/vetbookings-vetclinic.json

Errors received:

Mock file "pacts/vetbookings-vetclinic.json" is not compatible with spec file "oas/vets_example_oas-transformed.json"
7 error(s)
        response.body.incompatible: 7
0 warning(s)
{
  warnings: [],
  errors: [
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - id',
      mockDetails: {
        interactionDescription: 'A request to view owner 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        interactionState: 'There is a pet owner with ID 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        location: '[root].interactions[0].response.body.pets[0]',
        mockFile: 'pacts/vetbookings-vetclinic.json',
        value: {
          id: '6366c28a-1a6e-4233-a56a-bb902060d642',
          name: 'Dave',
          age: 4,
          species: 'Cat',
          discharged: false,
          colour: 'Black',
          mischievous_scale: 10
        }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./clinic/owner/{id}.get.responses.200.content.application/json.schema.properties.pets.items.additionalProperties',
        pathMethod: 'get',
        pathName: '/clinic/owner/{id}',
        specFile: 'oas/vets_example_oas-transformed.json',
        value: undefined
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - name',
      mockDetails: {
        interactionDescription: 'A request to view owner 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        interactionState: 'There is a pet owner with ID 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        location: '[root].interactions[0].response.body.pets[0]',
        mockFile: 'pacts/vetbookings-vetclinic.json',
        value: {
          id: '6366c28a-1a6e-4233-a56a-bb902060d642',
          name: 'Dave',
          age: 4,
          species: 'Cat',
          discharged: false,
          colour: 'Black',
          mischievous_scale: 10
        }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./clinic/owner/{id}.get.responses.200.content.application/json.schema.properties.pets.items.additionalProperties',
        pathMethod: 'get',
        pathName: '/clinic/owner/{id}',
        specFile: 'oas/vets_example_oas-transformed.json',
        value: undefined
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - age',
      mockDetails: {
        interactionDescription: 'A request to view owner 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        interactionState: 'There is a pet owner with ID 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        location: '[root].interactions[0].response.body.pets[0]',
        mockFile: 'pacts/vetbookings-vetclinic.json',
        value: {
          id: '6366c28a-1a6e-4233-a56a-bb902060d642',
          name: 'Dave',
          age: 4,
          species: 'Cat',
          discharged: false,
          colour: 'Black',
          mischievous_scale: 10
        }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./clinic/owner/{id}.get.responses.200.content.application/json.schema.properties.pets.items.additionalProperties',
        pathMethod: 'get',
        pathName: '/clinic/owner/{id}',
        specFile: 'oas/vets_example_oas-transformed.json',
        value: undefined
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - species',
      mockDetails: {
        interactionDescription: 'A request to view owner 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        interactionState: 'There is a pet owner with ID 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        location: '[root].interactions[0].response.body.pets[0]',
        mockFile: 'pacts/vetbookings-vetclinic.json',
        value: {
          id: '6366c28a-1a6e-4233-a56a-bb902060d642',
          name: 'Dave',
          age: 4,
          species: 'Cat',
          discharged: false,
          colour: 'Black',
          mischievous_scale: 10
        }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./clinic/owner/{id}.get.responses.200.content.application/json.schema.properties.pets.items.additionalProperties',
        pathMethod: 'get',
        pathName: '/clinic/owner/{id}',
        specFile: 'oas/vets_example_oas-transformed.json',
        value: undefined
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - discharged',
      mockDetails: {
        interactionDescription: 'A request to view owner 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        interactionState: 'There is a pet owner with ID 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        location: '[root].interactions[0].response.body.pets[0]',
        mockFile: 'pacts/vetbookings-vetclinic.json',
        value: {
          id: '6366c28a-1a6e-4233-a56a-bb902060d642',
          name: 'Dave',
          age: 4,
          species: 'Cat',
          discharged: false,
          colour: 'Black',
          mischievous_scale: 10
        }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./clinic/owner/{id}.get.responses.200.content.application/json.schema.properties.pets.items.additionalProperties',
        pathMethod: 'get',
        pathName: '/clinic/owner/{id}',
        specFile: 'oas/vets_example_oas-transformed.json',
        value: undefined
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - colour',
      mockDetails: {
        interactionDescription: 'A request to view owner 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        interactionState: 'There is a pet owner with ID 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        location: '[root].interactions[0].response.body.pets[0]',
        mockFile: 'pacts/vetbookings-vetclinic.json',
        value: {
          id: '6366c28a-1a6e-4233-a56a-bb902060d642',
          name: 'Dave',
          age: 4,
          species: 'Cat',
          discharged: false,
          colour: 'Black',
          mischievous_scale: 10
        }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./clinic/owner/{id}.get.responses.200.content.application/json.schema.properties.pets.items.additionalProperties',
        pathMethod: 'get',
        pathName: '/clinic/owner/{id}',
        specFile: 'oas/vets_example_oas-transformed.json',
        value: undefined
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - mischievous_scale',
      mockDetails: {
        interactionDescription: 'A request to view owner 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        interactionState: 'There is a pet owner with ID 9b4f0df1-e41b-497d-b740-e1c65b89734c',
        location: '[root].interactions[0].response.body.pets[0]',
        mockFile: 'pacts/vetbookings-vetclinic.json',
        value: {
          id: '6366c28a-1a6e-4233-a56a-bb902060d642',
          name: 'Dave',
          age: 4,
          species: 'Cat',
          discharged: false,
          colour: 'Black',
          mischievous_scale: 10
        }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./clinic/owner/{id}.get.responses.200.content.application/json.schema.properties.pets.items.additionalProperties',
        pathMethod: 'get',
        pathName: '/clinic/owner/{id}',
        specFile: 'oas/vets_example_oas-transformed.json',
        value: undefined
      },
      type: 'error'
    }
  ]
}

Error: Mock file "pacts/vetbookings-vetclinic.json" is not compatible with spec file "oas/vets_example_oas-transformed.json"

Thanks!

[github-actions]

👋 Hi! The 'smartbear-supported' label has just been added to this issue, which will create an internal tracking ticket in PactFlow's Jira (PACT-979). We will use this to prioritise and assign a team member to this task. All activity will be public on this ticket. For now, sit tight and we'll update this ticket once we have more information on the next steps.

See our documentation for more information.

[mefellows]

Hi Nick, we need to review the details of this to understand why it's not working as expected. We should be able to support this, however the way JSON schemas are applied in certain scenarios makes the application of the input to the schemas mutually exclusive, resulting in the errors you see.

I don't have a timeline to share with you just yet, but we will be reviewing it in this quarter.

[mefellows]

@Nickcw6 in investigating a similar issue with another contract, I stumbled upon the same conclusion you came to (I clearly missed the insight you added, but probably best I did!)

There does seem to be a workaround by adding additionalProperties: true wherever there a oneOf keyword is used, but this in doing so you're losing some of the benefits contract testing provides.

I'm not sure what benefits are actually lost in this case. I suspect there is some bug in the validator because when you add it, it seems to do what we want:

OAS:

openapi: 3.0.1
info:
  title: anyOf / oneOf example
  description: Demonstrate the anyOf and oneOf issue
  version: 1.0.0
paths:
  /products/{id}:
    get:
      summary: Find product by ID
      description: Returns a single product
      operationId: getProductByID
      parameters:
      - name: id
        in: path
        description: ID of product to get
        schema:
          type: string
        required: true
        example: "10"
      responses:
        "200":
          description: successful operation
          content:
            "application/json":
              schema:
                type: "object"
                additionalProperties: true
                oneOf:
                  - $ref: '#/components/schemas/Cat'
                  - $ref: '#/components/schemas/Product'
              examples:
                application/json:
                  value:
                    id: "1234"
                    type: "food"
                    price: 42
                    name: "thing"
        "400":
          description: Invalid ID supplied
          content: {}
        "404":
          description: Product not found
          content: {}
components:
  schemas:
    Product:
      type: object
      required:
        - name
        - price
        - id
        - type
        - version
      additionalProperties: false
      properties:
        id:
          type: string
        type:
          type: string
        name:
          type: string
        version:
          type: string
        price:
          type: number
    Cat:
      type: object
      required:
        - name
        - owner
      additionalProperties: false
      properties:
        name:
          type: string
        owner:
          type: string

Pact file:

{
  "consumer": {
    "name": "pactflow-example-consumer"
  },
  "provider": {
    "name": "collaborative-contracts-provider"
  },
  "interactions": [
    {
      "description": "a request to get a product",
      "request": {
        "method": "GET",
        "path": "/products/10",
        "headers": {
          "Authorization": "Bearer 2019-01-14T11:34:18.045Z"
        }
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "name": "10",
          "type": "CREDIT_CARD",
          "price": 28
        }
      }
    },
    {
      "description": "a request to get a cat shaped product",
      "request": {
        "method": "GET",
        "path": "/products/cat",
        "headers": {
          "Authorization": "Bearer 2019-01-14T11:34:18.045Z"
        }
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "name": "kitty",
          "owner": "crazy cat lady"
        }
      }
    }
  ]
}

If you were to modify one of the interactions to have a key that doesn't exist, the test fails as one would expect:

➜  npx @pactflow/swagger-mock-validator ~/Downloads/test-oneof-oas.yml ~/Downloads/test-oneof-pact.json                                                                                                                               <aws:pactflow-prod-admin>
Mock file "/Users/matthew.fellows/Downloads/test-oneof-pact.json" is not compatible with spec file "/Users/matthew.fellows/Downloads/test-oneof-oas.yml"
3 error(s)
	response.body.incompatible: 3
0 warning(s)
{
  warnings: [],
  errors: [
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - notvalidattribute',
      mockDetails: {
        interactionDescription: 'a request to get a cat shaped product',
        interactionState: '[none]',
        location: '[root].interactions[0].response.body',
        mockFile: '/Users/matthew.fellows/Downloads/test-oneof-pact.json',
        value: { notvalidattribute: 'kitty' }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./products/{id}.get.responses.200.content.application/json.schema.oneOf.0.additionalProperties',
        pathMethod: 'get',
        pathName: '/products/{id}',
        specFile: '/Users/matthew.fellows/Downloads/test-oneof-oas.yml',
        value: false
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - notvalidattribute',
      mockDetails: {
        interactionDescription: 'a request to get a cat shaped product',
        interactionState: '[none]',
        location: '[root].interactions[0].response.body',
        mockFile: '/Users/matthew.fellows/Downloads/test-oneof-pact.json',
        value: { notvalidattribute: 'kitty' }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./products/{id}.get.responses.200.content.application/json.schema.oneOf.1.additionalProperties',
        pathMethod: 'get',
        pathName: '/products/{id}',
        specFile: '/Users/matthew.fellows/Downloads/test-oneof-oas.yml',
        value: false
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should match exactly one schema in oneOf',
      mockDetails: {
        interactionDescription: 'a request to get a cat shaped product',
        interactionState: '[none]',
        location: '[root].interactions[0].response.body',
        mockFile: '/Users/matthew.fellows/Downloads/test-oneof-pact.json',
        value: { notvalidattribute: 'kitty' }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./products/{id}.get.responses.200.content.application/json.schema.oneOf',
        pathMethod: 'get',
        pathName: '/products/{id}',
        specFile: '/Users/matthew.fellows/Downloads/test-oneof-oas.yml',
        value: [ [Object], [Object] ]
      },
      type: 'error'
    }
  ]
}

Error: Mock file "/Users/matthew.fellows/Downloads/test-oneof-pact.json" is not compatible with spec file "/Users/matthew.fellows/Downloads/test-oneof-oas.yml"
    at /Users/matthew.fellows/.nvm/versions/node/v18.16.0/lib/node_modules/@pactflow/swagger-mock-validator/dist/cli.js:89:36
    at Generator.next (<anonymous>)
    at fulfilled (/Users/matthew.fellows/.nvm/versions/node/v18.16.0/lib/node_modules/@pactflow/swagger-mock-validator/dist/cli.js:6:58)

The oneOf and anyOf semantics are also preserved, if you update the pact test to only use name in the response bodies with oneOf set, you get:

➜  npx @pactflow/swagger-mock-validator ~/Downloads/test-oneof-oas.yml ~/Downloads/test-oneof-pact.json                                                                                                                               <aws:pactflow-prod-admin>
Mock file "/Users/matthew.fellows/Downloads/test-oneof-pact.json" is not compatible with spec file "/Users/matthew.fellows/Downloads/test-oneof-oas.yml"
2 error(s)
	response.body.incompatible: 2
0 warning(s)
{
  warnings: [],
  errors: [
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should match exactly one schema in oneOf',
      mockDetails: {
        interactionDescription: 'a request to get a product',
        interactionState: '[none]',
        location: '[root].interactions[0].response.body',
        mockFile: '/Users/matthew.fellows/Downloads/test-oneof-pact.json',
        value: { name: '10' }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./products/{id}.get.responses.200.content.application/json.schema.oneOf',
        pathMethod: 'get',
        pathName: '/products/{id}',
        specFile: '/Users/matthew.fellows/Downloads/test-oneof-oas.yml',
        value: [ [Object], [Object] ]
      },
      type: 'error'
    },
    {
      code: 'response.body.incompatible',
      message: 'Response body is incompatible with the response body schema in the spec file: should match exactly one schema in oneOf',
      mockDetails: {
        interactionDescription: 'a request to get a cat shaped product',
        interactionState: '[none]',
        location: '[root].interactions[1].response.body',
        mockFile: '/Users/matthew.fellows/Downloads/test-oneof-pact.json',
        value: { name: 'kitty' }
      },
      source: 'spec-mock-validation',
      specDetails: {
        location: '[root].paths./products/{id}.get.responses.200.content.application/json.schema.oneOf',
        pathMethod: 'get',
        pathName: '/products/{id}',
        specFile: '/Users/matthew.fellows/Downloads/test-oneof-oas.yml',
        value: [ [Object], [Object] ]
      },
      type: 'error'
    }
  ]
}

Error: Mock file "/Users/matthew.fellows/Downloads/test-oneof-pact.json" is not compatible with spec file "/Users/matthew.fellows/Downloads/test-oneof-oas.yml"

Changing to anyOf will once again get it to pass.

[Nickcw6]

Thanks @mefellows - have revisited this and have been able to get this working as expected. Stumbled across that OAS examples repo you put together which was super helpful - particularly the bit about inheritance. Our initial spec was missing the discriminator which is why we were having issues with it, and also seemed to be why it was resolving weird when using the merging tool.

[mefellows]

Thanks Nick - great to hear that helped. We've continuing to make changes to this over the next little while so I'm compiling a list of examples to elucidate the behaviour.

I'll link that from the README here, and also over in the PactFlow docs in the coming weeks.

Any additional feedback on the above please do feel free to update us here.

[mefellows]

I believe this was fixed (and improved) in #32.

The examples in that linked repo have been updated accordingly also.

Closing this one now.