Data against Schema validator - $ref, allOf/anyOf/oneOf, nullable $ref objects and nullable enums.

[swiesend]

Hey @llfbandit,

your library is great and helps a lot to validate data against a rather complex schema, but I encountered some problems, which I like to share with an example.

My example OpenAPI spec defines a Zoo which contains:

1. root references like: $ref: "#/components/schemas/???". Which leads to a schemaParentNode/parentSchema problem.
2. allOf/anyOf quantor for grouped objects, see: BigFive. Which one is to pick, if I want to allow any Animal within that group?
3. anyOf/oneOf quantor for array items, see: SafariPark.animals. Which one is to pick, if I want to allow any of the given Animals by its group?
4. nullable referenced objects, as follows (see also Reference objects don't combine well with “nullable” #1368 , Clarify Semantics of nullable in OpenAPI 3.0):

"Sibling values alongside $refs are ignored. To add properties to a $ref, wrap the $ref into allOf, or move the extra properties into the referenced definition (if applicable)." (swagger-editor)

          properties:
            maybeSafari:
              $ref: "#/components/schemas/SafariPark"

-->

    SafariPark:
      type: object
      nullable: true

5. nullable enums do not work as expected:

        subspecies:
          type: string
          nullable: true
          enum:
            - PantheraLeoLeo
            - PantheraLeoMelanochaita
            - null

I have four problems:

Problem 1:
I guess with the upcoming changes in 0.9 I won't be able to instantiate the SchemaValidator with the schemaParentNode and parentSchema, as this constructor is not public anymore.

I wrote a validation method in Scala like so:

  def validate(schemaName: String, data: JsonNode, isValid: Boolean, schema: JsonNode): ValidationResults = {
    val apiContext: OAI3Context = new OAI3Context(new URI("/"), schema)
    val validationContext: ValidationContext[OAI3] = new ValidationContext(apiContext)
    validationContext.setFastFail(true)

    val parentValidator = new SchemaValidator("", schema)

    // 0.8
    val objectValidator: SchemaValidator = new SchemaValidator(
      validationContext, // context
      schemaName, // propertyName
      Try(schema.get("components").get("schemas").get(schemaName)).getOrElse(schema), // schemaNode
      schema, // schemaParentNode
      parentValidator // parentSchema
    )
    val validation = new ValidationResults();

    // 0.9-SNAPSHOT
    //    val objectValidator: SchemaValidator = new SchemaValidator(
    //      validationContext,
    //      schemaName,
    //      schema.get("components").get("schemas").get(schemaName)
    //    ???
    //    )
    //    val validation: ValidationData[Unit] = new ValidationData();

    objectValidator.validate(data, validation)
    if (validation.isValid === false) println(s"'$schemaName' $validation")
    // assert(validation.isValid === isValid)
    validation
  }

This validates:

"A `Zoo` with a `SafariPark`" should "only contain `BigFive` `Animal`s" in {
    val schema: JsonNode = TreeUtil.json.readTree(new File("zoo.json"))

    val data: JsonNode = TreeUtil.json.readTree(
      """
        |{
        |  "name": "Zoologischer Garten Berlin AG",
        |  "maybeNickname": "Zoo Berlin",
        |  "parks": {
        |    "maybeSafari": {
        |      "name": "Great Safari Park",
        |      "animals": [
        |        {
        |          "species": "LoxodontaAfricana",
        |          "subspecies": null
        |        },
        |        {
        |          "species": "PantheraLeo",
        |          "subspecies": "PantheraLeoLeo"
        |        }
        |      ]
        |    }
        |  }
        |}
        |""".stripMargin)
    val result = validate("Zoo", data, true, schema)
    assert(result.isValid === true)
  }

Problem 2:
If I try at purpose to break the schema, by changing species to a forbidden value, then I get a rather uninformative validation report like this:

'Zoo' Validation error(s) :
Zoo.parks.maybeSafari./#/components/schemas/SafariPark.animals.items.anyOf : No valid schema. (code: 1001)

Problem 3:
nullable referenced objects are not validated as nullables:

"A `Zoo` without a `SafariPark`" should "be valid, too" in {
    val schema: JsonNode = TreeUtil.json.readTree(new File("zoo.json"))

    val data: JsonNode = TreeUtil.json.readTree(
      """
        |{
        |  "name": "Zoologischer Garten Berlin AG",
        |  "maybeNickname": "Zoo Berlin",
        |  "parks": {
        |    "maybeSafari": null
        |  }
        |}
        |""".stripMargin)
    val result = validate("Zoo", data, true, schema)
    assert(result.isValid === true)
  }

Which give me the following validation error:

'Zoo' Validation error(s) :
Zoo.parks.maybeSafari.nullable : Null value is not allowed. (code: 1021)

Problem 4:
This validates fine:

"A valid `Animal` with a nullable enum" should "be valid" in {
    val schema: JsonNode = TreeUtil.json.readTree(new File("zoo.json"))

    val data: JsonNode = TreeUtil.json.readTree(
      """
        |{
        |  "species": "PantheraPardus",
        |  "subspecies": null
        |}
        |""".stripMargin)
    val result = validate("Animal", data, true, schema)
    assert(result.isValid === true)
  }

But this does not, as the validator seems to accept any value, but I expect the enum to forbid such values:

  "An invalid `Animal` with a nullable enum" should "not be valid" in {
    val schema: JsonNode = TreeUtil.json.readTree(new File("zoo.json"))

    val data: JsonNode = TreeUtil.json.readTree(
      """
        |{
        |  "species": "PantheraPardus",
        |  "subspecies": "not allowed"
        |}
        |""".stripMargin)
    val result = validate("Animal", data, false, schema)
    assert(result.isValid === false)
  }

---

This it the full zoo.yaml:

openapi: 3.0.0
info:
  title: Zootopia
  description: "The zoo is a place where animals are held. Hopefully under good circumstances."
  version: 1.0.0

paths:
  /api/v1/zoo/{name}:
    get:
      parameters:
        - in: path
          name: name
          required: true
          schema:
            type: string
      responses:
        '200':
          description: Ok
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Zoo"

components:
  schemas:
    Zoo:
      type: object
      required:
        - name
      properties:
        name:
          type: string
        maybeNickname:
          type: string
          nullable: true
        parks:
          properties:
            maybeSafari:
              $ref: "#/components/schemas/SafariPark"
              # nullable: true # "Sibling values alongside $refs are ignored. To add properties to a $ref, wrap the $ref into allOf, or move the extra properties into the referenced definition (if applicable)." (swagger-editor)
            # apes:
            # ...
            # birds:
            # ...
            # aquarium:
            # ...

    SafariPark:
      type: object
      nullable: true
      required:
        - name
        - animals
      properties:
        name:
          type: string
        animals:
          type: array
          items:
            anyOf:
              - $ref: "#/components/schemas/BigFive"

    Animal:
      type: object
      required:
        - species
      properties:
        species:
          type: string
        subspecies:
          type: string
          nullable: true
      discriminator:
        propertyName: species
        mapping:
          LoxodontaAfricana: '#/components/schemas/Elephant'
          LoxodontaCyclotis: '#/components/schemas/Elephant'
          DicerosBicornis: '#/components/schemas/Rhinoceros' 
          SyncerusCaffer: '#/components/schemas/CapeBuffalo'
          PantheraLeo: "#/components/schemas/Lion"
          PantheraPardus: "#/components/schemas/Leopard"

    BigFive:
      anyOf:
        - $ref: "#/components/schemas/Elephant"
        - $ref: "#/components/schemas/Rhinoceros"
        - $ref: "#/components/schemas/CapeBuffalo"
        - $ref: "#/components/schemas/Lion"
        - $ref: "#/components/schemas/Leopard"

    Elephant:
      allOf:
        - $ref: '#/components/schemas/Animal'
        - type: object
      properties:
        species:
          enum:
            - LoxodontaAfricana
            - LoxodontaCyclotis

    Rhinoceros:
      allOf:
        - $ref: '#/components/schemas/Animal'
        - type: object
      properties:
        species:
          enum:
            - DicerosBicornis

    CapeBuffalo:
      allOf:
        - $ref: '#/components/schemas/Animal'
        - type: object
      properties:
        species:
          enum:
            - SyncerusCaffer
        subspecies:
          enum:
            - SyncerusCafferCaffer
            - SyncerusCafferNanus
            - SyncerusCafferBrachyceros
            - SyncerusCafferAequinoctialis
            - SyncerusCafferMathewsi
            - null

    Lion:
      allOf:
        - $ref: '#/components/schemas/Animal'
        - type: object
      properties:
        species:
          enum:
            - PantheraLeo
        subspecies:
          enum:
            - PantheraLeoLeo
            - PantheraLeoMelanochaita
            - null
    
    Leopard:
      allOf:
      - $ref: '#/components/schemas/Animal'
      - type: object
      properties:
        species:
          enum:
            - PantheraPardus
        subspecies:
          enum:
            - PantheraPardusPardus
            - null

The zoo.yaml as json (converted with the swagger-editor):

{
  "openapi": "3.0.0",
  "info": {
    "title": "Zootopia",
    "description": "The zoo is a place where animals are held. Hopefully under good circumstances.",
    "version": "1.0.0"
  },
  "paths": {
    "/api/v1/zoo/{name}": {
      "get": {
        "parameters": [
          {
            "in": "path",
            "name": "name",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "Ok",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Zoo"
                }
              }
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "Zoo": {
        "type": "object",
        "required": [
          "name"
        ],
        "properties": {
          "name": {
            "type": "string"
          },
          "maybeNickname": {
            "type": "string",
            "nullable": true
          },
          "parks": {
            "properties": {
              "maybeSafari": {
                "$ref": "#/components/schemas/SafariPark"
              }
            }
          }
        }
      },
      "SafariPark": {
        "type": "object",
        "nullable": true,
        "required": [
          "name",
          "animals"
        ],
        "properties": {
          "name": {
            "type": "string"
          },
          "animals": {
            "type": "array",
            "items": {
              "anyOf": [
                {
                  "$ref": "#/components/schemas/BigFive"
                }
              ]
            }
          }
        }
      },
      "Animal": {
        "type": "object",
        "required": [
          "species"
        ],
        "properties": {
          "species": {
            "type": "string"
          },
          "subspecies": {
            "type": "string",
            "nullable": true
          }
        },
        "discriminator": {
          "propertyName": "species",
          "mapping": {
            "LoxodontaAfricana": "#/components/schemas/Elephant",
            "LoxodontaCyclotis": "#/components/schemas/Elephant",
            "DicerosBicornis": "#/components/schemas/Rhinoceros",
            "SyncerusCaffer": "#/components/schemas/CapeBuffalo",
            "PantheraLeo": "#/components/schemas/Lion",
            "PantheraPardus": "#/components/schemas/Leopard"
          }
        }
      },
      "BigFive": {
        "anyOf": [
          {
            "$ref": "#/components/schemas/Elephant"
          },
          {
            "$ref": "#/components/schemas/Rhinoceros"
          },
          {
            "$ref": "#/components/schemas/CapeBuffalo"
          },
          {
            "$ref": "#/components/schemas/Lion"
          },
          {
            "$ref": "#/components/schemas/Leopard"
          }
        ]
      },
      "Elephant": {
        "allOf": [
          {
            "$ref": "#/components/schemas/Animal"
          },
          {
            "type": "object"
          }
        ],
        "properties": {
          "species": {
            "enum": [
              "LoxodontaAfricana",
              "LoxodontaCyclotis"
            ]
          }
        }
      },
      "Rhinoceros": {
        "allOf": [
          {
            "$ref": "#/components/schemas/Animal"
          },
          {
            "type": "object"
          }
        ],
        "properties": {
          "species": {
            "enum": [
              "DicerosBicornis"
            ]
          }
        }
      },
      "CapeBuffalo": {
        "allOf": [
          {
            "$ref": "#/components/schemas/Animal"
          },
          {
            "type": "object"
          }
        ],
        "properties": {
          "species": {
            "enum": [
              "SyncerusCaffer"
            ]
          },
          "subspecies": {
            "enum": [
              "SyncerusCafferCaffer",
              "SyncerusCafferNanus",
              "SyncerusCafferBrachyceros",
              "SyncerusCafferAequinoctialis",
              "SyncerusCafferMathewsi",
              null
            ]
          }
        }
      },
      "Lion": {
        "allOf": [
          {
            "$ref": "#/components/schemas/Animal"
          },
          {
            "type": "object"
          }
        ],
        "properties": {
          "species": {
            "enum": [
              "PantheraLeo"
            ]
          },
          "subspecies": {
            "enum": [
              "PantheraLeoLeo",
              "PantheraLeoMelanochaita",
              null
            ]
          }
        }
      },
      "Leopard": {
        "allOf": [
          {
            "$ref": "#/components/schemas/Animal"
          },
          {
            "type": "object"
          }
        ],
        "properties": {
          "species": {
            "enum": [
              "PantheraPardus"
            ]
          },
          "subspecies": {
            "enum": [
              "PantheraPardusPardus",
              null
            ]
          }
        }
      }
    }
  }
}

[llfbandit]

Hi and welcome,

I'll try to answer all of this as best as I can:
1- You are setting the schema as the schema to validate and as its own parent. This is useless. You don't need the protected method, since data can be validated from a single root schema. Try it, it won't change the results.
2- Version 0.9 will report subvalidation items instead of this for all collection schemas (oneOf/anyOf/allOf). You can try it, this is already available.
3- If you set $ref, all siblings are ignored. You need to refactor your description to include it. By default, nullable is false.
4- I don't get this one. In your Animal schema, you defined subspecies as nullable. Your discriminator does not work. This only allowed when using oneOf, anyOf and allOf.

I hope those answers are helpful to you. If so, please close the issue by yourself.
Thanks.

[swiesend]

Thank you @llfbandit for your swift response!

Unfortunately I don't get it quite how to use the library correctly.

1. How do I have to initialize the SchemaValidator in order to resolve $ref: "#/components/schemas/*" correctly? To clarify my point I added some more examples .

2. Yes, this is why I wanted to use the 0.9-SNAPSHOT, but failed because I do not initialize the SchemaValidator correctly.

3. I think my spec description already includes nullable: true in the referenced SafariPark object, which is from type: object and sets nullable: true.
   Or would it be better to go for the allOf quantor and set the nullable: true in the parent of the $ref instead of inside the ref itself, like pointed out here:

           properties:
            maybeSafari:
              allOf:
                - $ref: "#/components/schemas/SafariPark"
              nullable: true

4. I don't undestand why the discriminator does not work here as it is inherited by all Animal instances, which are just grouped by BigFive. If there are some errors in the logic of my schema I would appreciate if you can point them out a little more in detail, so that I can investigate such inconsistencies.

To 1.:

I started to use the other constructor as I had issues with the $ref: "#/components/schemas/*" references, which were not resolved properly when I just pass the schemas form path components/schemas as root.

val objectValidator: SchemaValidator = new SchemaValidator("Animal", schema.get("components").get("schemas"))

Which leads to an

org.openapi4j.core.exception.ResolutionException: Reference '#/components/schemas/SafariPark' is unreachable in '/.

But if just put my zoo.json as root schema then the SchemaValidator validates against anything.

The following three examples accept this object with foo properties without complaining.

1.1 Without ValidationContext only SchemaValidator

"Some object with foo props" should "not be valid against the schema" in {
    val schema: JsonNode = TreeUtil.json.readTree(new File("zoo.json"))
    val data: JsonNode = TreeUtil.json.readTree(
      """
        |{
        |  "foo": "bar"
        |}
        |""".stripMargin)

    val objectValidator: SchemaValidator = new SchemaValidator("Animal", schema)
    val validation = new ValidationResults();
    objectValidator.validate(data, validation)
    assert(validation.isValid === false)
  }

1.2 Do I necessarily have to create a ValidationContext where the schema is the root?

"Some object with foo props" should "not be valid against the schema with a ValidationContext from root" in {
    val schema: JsonNode = TreeUtil.json.readTree(new File("zoo.json"))
    val data: JsonNode = TreeUtil.json.readTree(
      """
        |{
        |  "foo": "bar"
        |}
        |""".stripMargin)

    val apiContext: OAI3Context = new OAI3Context(new URI("/"), schema)
    val validationContext: ValidationContext[OAI3] = new ValidationContext(apiContext)
    validationContext.setFastFail(false)
    val objectValidator: SchemaValidator = new SchemaValidator(validationContext, "Animal", schema)
    val validation = new ValidationResults();
    objectValidator.validate(data, validation)
    assert(validation.isValid === false)
  }

1.3 And if I do so from which path do I have to put the schema into the SchemaValidator?

"Some object with foo props" should 
"not be valid against the schema with a ValidationContext from root and a SchemaValidator from path #/components/schemas" in {
    val schema: JsonNode = TreeUtil.json.readTree(new File("zoo.json"))
    val data: JsonNode = TreeUtil.json.readTree(
      """
        |{
        |  "foo": "bar"
        |}
        |""".stripMargin)

    val apiContext: OAI3Context = new OAI3Context(new URI("/"), schema)
    val validationContext: ValidationContext[OAI3] = new ValidationContext(apiContext)
    validationContext.setFastFail(false)
    val objectValidator: SchemaValidator = new SchemaValidator(validationContext, "Animal", schema.get("components").get("schemas"))
    val validation = new ValidationResults();
    objectValidator.validate(data, validation)
    assert(validation.isValid === false)
  }

[llfbandit]

Assuming you're now using 0.9-SNAPSHOT.

The following pseudo-code should be enough to answer 1 & 2 (you need to now what schema you will validate data against):

val apiContext: OAI3Context = new OAI3Context(your_doc_location_as_URL))
val validationContext: ValidationContext[OAI3] = new ValidationContext(apiContext)
val objectValidator: SchemaValidator = new SchemaValidator(validationContext, null, animalSchema)

3- Ok, did not see it. Need to check.

4- Sorry my answer was unclear. As far as I can tell, you are validating your data directly from Animal "abstract" schema not from BigFive or a dedicated animal instance. This is why you don't get what you expect.

[llfbandit]

3 is fixed now in latest 0.9-SNAPSHOT.

[llfbandit]

Are you still experiencing difficulties to validate your data?

For more clarification, when OAI3Context is created JSON references are registered and so known when you validate your data. Obviously, you should not re-create it each time you need validation.

[swiesend]

Thank you very much @llfbandit! Yes, this helped. I am sorry I was unavailable for some time.

But I ran into an uninformative error message, for which I opened a new PR #105.