phases=[Phase.explicit] does not consistently use examples request body

[charliehohenstein30]

I'm currently trying to use only explicit examples in my openApi spec file. However, phases=[Phase.explicit] does not always use all of the example values that are declared in the RequestBody. This should be related to #381

Here is how I am running my test:

# start pytest
@schema.parametrize()
@settings(max_examples=1, phases=[Phase.explicit], deadline=20000)
def test_all_endpoints(case):

    # Checking if Schema matches response
    case.validate_response(response)

Here is an example of the request object Im using:

type: object
      properties:
        _settings:
            type: array
            items:
              type: object
              properties:
                type:
                  type: string
                name:
                  type: string
                category:
                  type: string
                value:
                  type: boolean
                org_unit_id:
                  type: string
        _org_unit_id:
          type: integer
      example:
        _settings: [{"type": "org_unit_settings","name": "therapeutic_alternatives_org_unit_id","category": "data_filtering","value": "1","org_unit_id": "1"}]
        _org_unit_id: 477

An example of what happens is that it will run one time correctly and has a request body:

REQUEST BODY: {'_settings': [{'type': 'org_unit_settings', 'name': 'therapeutic_alternatives_org_unit_id', 'category': 'data_filtering', 'value': 1, 'org_unit_id': 1}], '_org_unit_id': 477}

And then will run another time and the request body will be incorrect as this

REQUEST BODY: {'_org_unit_id': 0}

[charliehohenstein30]

In other words, It'd be nice to have a way to always use explicit examples in the openApi spec file for every single request that is made

[Stranger6667]

Hi @charliehohenstein30 !
It is strange, in test_specified_example_body_media_type_override we have similar case, but it is executed only once.
As far as I see, both fields from "example" are used in the request body - "_settings" and "_org_unit_id". It seems like I am missing something, could you, please, elaborate?
Also, could you, please, share some more information from your schema so I can reproduce it?

[charliehohenstein30]

Hi @Stranger6667
I figured out how to fix the problem specifically for my use case with the following code ..

# start pytest
@schema.parametrize()
@settings(max_examples=1, phases=[Phase.explicit], deadline=20000)
def test_all_endpoints(case):
        
    #Load Request Body
    if 'requestBody' in case.endpoint.definition:
        case.body = case.endpoint.definition['requestBody']['content']['application/json']['schema']['example']

Thank you!

[Stranger6667]

I think it might be caused by https://github.com/kiwicom/schemathesis/blob/b4c6659542f8a6fce76597609290c5d4dea8a05b/src/schemathesis/_hypothesis.py#L77

If there are multiple places with examples (e.g body and query) then when using example for query, data for body will be generated as usual, probably it would be better to merge examples from body, query, etc into a single example

[Stranger6667]

I'll make a new release with that change today.

[Stranger6667]

Hi @charliehohenstein30 !
Please, try the new version - https://pypi.org/project/schemathesis/0.25.0/
It should solve the issue (if my assumption about your case is correct). Let me know if it solves your case :)

Cheers

[charliehohenstein30]

Yes, this fixes the issue! Thank you so much I appreciate it! 👍 @Stranger6667