Validate doesnt seem to match schema correctly?

[dicolanl]

I am using validate against API responses.

 validate(openapi_schema_path, response.json())

seems to work on many of the apis. But when the api returns something like:

{"items":[4,5,6,1,2]}

which is an object with items property and an array of integers or objects in other cases. The validate throws

validation failed:
[6, 1, 2, 4, 5] is not of type 'object', 'boolean'

Failed validating 'type' in metaschema['allOf'][1]['properties']['items']['$dynamicRef']['allOf'][0]:
    {'$defs': {'anchorString': {'pattern': '^[A-Za-z_][-A-Za-z0-9._]*$',
                                'type': 'string'},
               'uriReferenceString': {'format': 'uri-reference',
                                      'type': 'string'},
               'uriString': {'format': 'uri', 'type': 'string'}},
     '$dynamicAnchor': 'meta',
     '$id': 'https://json-schema.org/draft/2020-12/meta/core',
     '$schema': 'https://json-schema.org/draft/2020-12/schema',
     '$vocabulary': {'https://json-schema.org/draft/2020-12/vocab/core': True},
     'properties': {'$anchor': {'$ref': '#/$defs/anchorString'},
                    '$comment': {'type': 'string'},
                    '$defs': {'additionalProperties': {'$dynamicRef': '#meta'},
                              'type': 'object'},
                    '$dynamicAnchor': {'$ref': '#/$defs/anchorString'},
                    '$dynamicRef': {'$ref': '#/$defs/uriReferenceString'},
                    '$id': {'$comment': 'Non-empty fragments not allowed.',
                            '$ref': '#/$defs/uriReferenceString',
                            'pattern': '^[^#]*#?$'},
                    '$ref': {'$ref': '#/$defs/uriReferenceString'},
                    '$schema': {'$ref': '#/$defs/uriString'},
                    '$vocabulary': {'additionalProperties': {'type': 'boolean'},
                                    'propertyNames': {'$ref': '#/$defs/uriString'},
                                    'type': 'object'}},
     'title': 'Core vocabulary meta-schema',
     'type': ['object', 'boolean']}

On schema['items']:
    [6, 1, 2, 4, 5]

the schema is defined as

methods:
      type: object
      properties:
        items:
          type: array
          items:
            type: integer

which matches the actual response.

Am i doing something wrong with validate?

[p1c2u]

Hi @dicolanl

thanks for the report. it's probably because you use schema keyword "items" as a property name. If you change the name it should work. I don't know the solution for the issue at this point.

EDIT:

 validate(openapi_schema_path, response.json())

Are you sure you pass parameters correctly? validate expects instance as a first argument and schema as a second one. In your example it looks like opposite

[dicolanl]

it seems to validate fine, if i flip them i get

API endpoint '/api?_limit=30' validation failed:
'/Volumes/Repos/src/openapi.yaml' is not of type 'object', 'boolean'

[dicolanl]

I changed my code but now nothing validates? ideas?

def validate_api_endpoint(api_endpoint, request_payload=None, response_schema=None):
    url = f"{base_url}{api_endpoint}"
    headers = {'Authorization': 'token'}
    try:
        # Make an API request
        response = requests.get(url, json=request_payload, verify=False, headers=headers)
        if response.status_code == 200:
            # Validate the response against the OpenAPI schema
            validate(response.json(), response_schema)
        else:
            print(f"API endpoint '{api_endpoint}' call failed:")
            return
        
        print(f"API endpoint '{api_endpoint}' is valid according to the OpenAPI schema.")
    except Exception as e:
        print(f"API endpoint '{api_endpoint}' validation failed:")
        print(str(e))

def validate_all_endpoints():
    spec_dict, base_uri = read_from_filename(openapi_schema_path)
    validate(spec_dict)
    paths = spec_dict.get('paths', {})

    for api_endpoint, methods in paths.items():
        for http_method, _ in methods.items():
            if http_method == 'get':
               response_schema = spec_dict['paths'][api_endpoint]['get']['responses']['200']['content']['application/json']['schema']
                    validate_api_endpoint(api_endpoint, request_payload=None, response_schema=response_schema)