Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple array items type validation with oneOf, anyOf #134

Closed
mgendre opened this issue Mar 1, 2016 · 16 comments
Closed

Multiple array items type validation with oneOf, anyOf #134

mgendre opened this issue Mar 1, 2016 · 16 comments

Comments

@mgendre
Copy link

mgendre commented Mar 1, 2016

Hi everyone,

I've encountered a problem to validate multiple items type in an array.

We have an array with 2 items types. We try to validate each item to 2 different schemas types.

Here is the schema:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "transfer": {
      "type": "object",
      "properties": {
        "modes": {
          "type": "array",
          "minItems": 1,
          "additionalItems": false,
          "items": {
            "oneOf": [
              {
                "type": "object",
                "properties": {
                  "mode": {
                    "enum": [
                      "ftp"
                    ]
                  },
                  "account": {
                    "type": "string",
                    "pattern": "^[a-f|0-9]{8}$"
                  }
                },
                "required": [
                  "mode",
                  "account"
                ],
                "additionalProperties": false
              },
              {
                "type": "object",
                "properties": {
                  "mode": {
                    "enum": [
                      "email"
                    ]
                  },
                  "mailingList": {
                    "type": "string",
                    "pattern": "^[a-f|0-9]{8}$"
                  }
                },
                "required": [
                  "mode",
                  "mailingList"
                ],
                "additionalProperties": false
              }
            ]
          }
        }
      },
      "required": [
        "modes"
      ],
      "additionalProperties": false
    }
  },
  "additionalProperties": false,
  "required": [
    "transfer"
  ]
}

And the data:

{
  "transfer": {
    "modes": [
      {
        "mode": "ftp",
        "account": "e69b3f54"
      },
      {
        "mode": "email",
        "mailingList": "c3d12752"
      }
    ]
  }
}

I tried to validate the data against the schema to different online tools:
http://json-schema-validator.herokuapp.com/index.jsp
http://jsonschemalint.com/draft4/#

All these tools validate successfully my data

It's like the validator try to match the 1st item to the 1st schema and the second item to the second schema. Order should be not important.

I use version 3.8.1.

Could you help me ?

Thank you in advance

@mgendre
Copy link
Author

mgendre commented Mar 1, 2016

When I remove the configuration "additionalProperties": false from the type definition on "oneOf" the problem disappear.

@epoberezkin
Copy link
Member

What are your ajv options? I suspect you may be using removeAdditional option. Without it it validates ok: https://tonicdev.com/esp/56d57c034b05fd0c00e8978b

If that is the case this issue is essentially a duplicate of #129 and it is an expected behaviour according to the JSON-schema standard.

@epoberezkin
Copy link
Member

I would suggest refactoring the schema to take the duplicated keywords and all property definitions apart from required out of oneOf:

"items": {
  "type": "object",
  "properties": {
    "mode": {
      "enum": [
        "ftp"
      ]
    },
    "account": {
      "type": "string",
      "pattern": "^[a-f|0-9]{8}$"
    },
    "mailingList": {
      "type": "string",
      "pattern": "^[a-f|0-9]{8}$"
    }
  },
  "additionalProperties": false,
  "oneOf": [
    { "required": [ "mode", "account" ] },
    { "required": [ "mode", "mailingList" ] }
  ]
}

This way it will work with removeAdditional option the way you expect it to.

The previous schema works correctly, but not how you expect probably

@mgendre
Copy link
Author

mgendre commented Mar 1, 2016

You are right @epoberezkin !

AJV is configured with removeAdditional -> true. The schema has been refactored to avoid multiple definitions inside oneOf, like you suggested and it's working.

Thank you @epoberezkin for your support !

@epoberezkin
Copy link
Member

You're welcome. I'll add a note in readme about it :)

@ribaptista
Copy link

I noticed the original schema would not validate an object containing both account and mailingList properties (that's expected), while the later refactored schema does.
Is there any workaround for this?

@epoberezkin
Copy link
Member

You can use { "not": { "required": ["account"] } } to explicitly prohibit property account.

@ngryman
Copy link

ngryman commented Aug 20, 2016

@epoberezkin Sorry to revive this but what if you do need to reference external schemas in oneOf and can't really split them like your proposed solution?

To take the above example with ref:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "transfer": {
      "type": "object",
      "properties": {
        "modes": {
          "type": "array",
          "minItems": 1,
          "additionalItems": false,
          "items": {
            "oneOf": [
              { "$ref": "ftp" },
              { "$ref": "email" }
            ]
          }
        }
      },
      "required": [
        "modes"
      ],
      "additionalProperties": false
    }
  },
  "additionalProperties": false,
  "required": [
    "transfer"
  ]
}

Thanks!

@epoberezkin
Copy link
Member

epoberezkin commented Aug 28, 2016

Btw, "additionalItems": false does nothing unless the schema in items keyword is an array.

The solutions to additionalProperties limitations are:

  • don't worry about additionalProperties and simply allow them. I think the obsession with removing them is not always justified, in many cases you make your life MUCH easier by allowing them (especially when you have multiple applications using the same data/schema). The argument against it though is that by allowing additional properties you may miss some typos in property names.
  • refactor schemas as suggested above
  • use custom switch keyword (or if/then/else) instead of anyOf - it allows to control what is validated based on some conditions (so only correct properties will be treated as additional)
  • use $merge/$patch keywords to extend schemas rather than joining them with compound keywords such as anyOf etc.

Please see FAQ that points to other related issues that show these approaches to using additionalProperties.

@dnvyrn
Copy link

dnvyrn commented Mar 9, 2018

schema:

"items": {
  "type": "object",
  "properties": {
    "mode": { "enum": [  "ftp" ]
    },
    "account": { "type": "string" },
    "mailingList": { "type": "string" },
    "otherProp": { "type": "string" }
  },
  "additionalProperties": false,
  "oneOf": [
    { "required": [ "mode", "account" ] },
    { "required": [ "mode", "mailingList", "otherProp" ] }
  ]
}

data:

{
  "transfer": {
    "modes": [
      {
        "mode": "ftp",
        "account": "e69b3f54",
        "mailingList": "c3d12752",
      },
      {
        "mode": "email",
        "mailingList": "c3d12752",
        "otherProp": "fasfasfaf"
      }
    ]
  }
}

It will validate pass. How to use { "not": { "required": ["account"] } } ?

@epoberezkin
Copy link
Member

I don’t understand the question - what are you trying to achieve? “not” can be a sibling to required and any other keyword, if that’s what you are asking.

Also it as a general JSON schema question - it should be posted to Stack Overflow.

@wigy-opensource-developer

#134 (comment)

I cannot express enough that this is an actual bug in the removeAdditional implementation that interferes with backtracking, it is not just a valid speed optimization. The removeAdditional must only remove properties after the anyOf decided on which branch to take in the matching. If any backtracking is still ongoing, no properties should be removed. As I read through your comments in related issues, I will not be able to convince you about this though 😄

In the workaround you provided above, if I fix the mode property (discriminator) to what it was in the schema OP posted:

...
"mode": {
  "enum": [
    "ftp",
    "email"
  ]
},
...

I also had to modify the oneOf

"oneOf": [
  { "mode": { "enum": [ "ftp" ] }, "required": [ "mode", "account" ] },
  { "mode": { "enum": [ "email" ] }, "required": [ "mode", "mailingList" ] }
]

otherwise it validates something with mode: email and having an account property. It works now, but cannot be modularized by using refs inside the oneOf as OP already mentioned.

See the working fix: https://runkit.com/wigy-opensource-developer/ajv-remove-additional-gotcha

@epoberezkin
Copy link
Member

epoberezkin commented Dec 11, 2019

@wigy-opensource-developer I understand that the way removeAdditional option is defined may be not fit for your purposes, it’s a much simpler thing than you want it to be.

It is consistent though with how additionalProperties keyword is defined in JSON Schema spec draft-07 and it’s a conscious design choice for Ajv.

What is important the behaviour that is created by “removeAdditional” option is not part of JSON Schema spec.

I am not 100% sure what do you mean by “backtracking“ - Ajv does no such thing. Also anyOf does not at any point decide which branch to use “to match against the data” - it simply follows the algorithm prescribed by the specification, which is to validate the data against all branches and if the data is valid against at least one branch, then the data is valid against anyOf keyword - it’s simply Boolean “or” operation on the validation results against all subschemas. At no point any decision is made about one branch “matching” the data better than the other. Ajv obviously short-circuits it and stops as soon as it finds the first branch against which the data is valid, but it is not the source of the problem you want to solve - you want the decision about which properties to remove postpone until Ajv decides what is/are the right branch(es).

As a side but related comment, it’s a common practice of JSON Schema users to use “schema matching the data” and “data valid against the schema” as equivalent interchangeable statements, but they are not equivalent; the spec only defines what the second statement means (“data valid against the schema” - just the Boolean outcome of a formal process of validating the data against each set of dependent keywords following a certain algorithm for each set of keywords) and the users also have reasonable expectations about what the first statement means (“schema matching the data” - meaning schema looking “similar” to the data, but there is no formal definition of what it actually means); the users equate these two statements that leads to massive confusion, frustration and complicates JSON schema adoption) cc @Relequestual.

Also JSON Schema has no concept of “discriminator” - it only exists as part of OpenAPI spec in connection with oneOf keyword (not anyOf), and even there, reading their v3 spec, from the validation result point of view “discriminator” is no-op - it is only there to optimise, to support tools, to choose the branch that is possibly correct based on one property and have more helpful error reporting - but its presence should have no effect on validation result (as in valid/invalid). We can discuss it further, but it’s my reading of OpenAPI spec.

There is a ticket #1119 that is about “discriminator” and how it could be supported by Ajv to improve error reporting and to optimise validation process. As a side effect, it would also lead to changes in removeAdditional behaviour that would likely make its behaviour closer to what you want (unless it’s purposefully prevented). I am not 100% sure yet whether it is a good or bad thing, it would definitely make it only more confusing though. So removeAdditional option behaviours have to be considered in the context of that ticket by whoever implements it.

@wigy-opensource-developer
Copy link

wigy-opensource-developer commented Dec 12, 2019

Thanks for your time to explain all these. Many people who just try to build on their computer science backround of algebraic types or who just used to work with unions in C or enums in Rust will have to relearn this working with Ajv.

How I see now is that if a developer has a choice to define their own schema (which is of course not always the case), they should rather use a key as a discriminator rather than a value.

{
  "modes": [
    { "ftp": { "account": "blah" }},
    { "email": { "mailingList": "foobar" }}
  ]
}

@epoberezkin
Copy link
Member

epoberezkin commented Dec 12, 2019 via email

@hansiemithun
Copy link

hansiemithun commented Feb 12, 2020

I have just handled like this for now with sass:

Maybe it will benefit some

.error-detail {
  .text-danger{
    display: none;
    &:first-child{
      display: block;
    }
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

8 participants
@ngryman @wigy-opensource-developer @epoberezkin @hansiemithun @mgendre @ribaptista @dnvyrn and others