anyOf validation errors only present when other validations fail

[mseeley]

What version of Ajv are you using? Does the issue happen if you use the latest version?

6.10.0

Ajv options object

undefined

JSON Schema

The schema intends to define a recursive structure. Valid data examples below.

const schema = {
  "$schema": "http://json-schema.org/draft-07/schema",
  "$ref": "#/definitions/stages",
  "definitions": {
    "stages": {
      "type": "array",
      "minItems": 1,
      "items": {
        "anyOf": [
          { "$ref": "#/definitions/stages" },
          { "$ref": "#/definitions/stage" }
        ]
      }
    },
    "stage": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string",
          "minLength": 1
        }
      }
    }
  }
};

Sample data

Invalid

[{ "name": "" }]

Valid (flat)

[{ "name": "A" }]

Valid (nested)

The JSON is a composite structure. It's an array of object or other arrays containing objects.

[
  { "name": "A" },
  [
    { "name": "B" },
    { "name": "C" }
  ]
]

Your code

Demo: https://runkit.com/mseeley/ajv

// See demo

Validation result, data AFTER validation, error messages

Validation fails due to the empty name attribute. Data is unchanged.

[
  {
    "keyword": "type",
    "dataPath": "[0]",
    "schemaPath": "#/type",
    "params": {
      "type": "array"
    },
    "message": "should be array"
  },
  {
    "keyword": "minLength",
    "dataPath": "[0].name",
    "schemaPath": "#/definitions/stage/properties/name/minLength",
    "params": {
      "limit": 1
    },
    "message": "should NOT be shorter than 1 characters"
  },
  {
    "keyword": "anyOf",
    "dataPath": "[0]",
    "schemaPath": "#/items/anyOf",
    "params": {},
    "message": "should match some schema in anyOf"
  }
]

What results did you expect?

I'm expecting an errors object containing only the failed minLength failure as changing name to a string with length > 0 causes the test to pass. This to me suggests the other two errors are not truly errors. I can't filter out anyOf errors as an array containing anything besides other array or object should fail anyOf.

I believe the non minLength errors are related to the anyOf check where the first member of the array could have been another array instead of an object.

Are you going to resolve the issue?

I've reviewing the FAQ and grepped for other issues (found #427, #558, #828). I'm unable to determine if this is a usage error, expected output, or an issue in validation.

If this is too close to an SO question please let me know and save yourself some time.

I appreciate any help. Thanks!

[epoberezkin]

Ajv doesn’t really know which schema is closer to the data. It simply validates it against both schemas and reports both errors that lead to anyof error. This is similar to discriminator pattern, only in this case it is driven by data type.

I am not sure if there is any satisfactory solution on implementation level. The workaround is to use if/then/else, putting type validation into if. In this case you would see errors only from one branch.

[mseeley]

Thanks @epoberezkin 👋

I'll experiment with if/then/else. Although, I'm still having a hard time understanding how the anyOf validations don't raise errors until name values to validate. Conversely, if name validates there are no anyOf errors.

It's the sometimes error behavior of anyOf that's throwing me off. This behavior expected?

[mseeley]

Ah, I see something. I think I misunderstood how the error propagates up through the schema.

Perhaps this failure is due to the item not being an array and not passing the object test due to the errors below.

[
  {
    "keyword": "type",
    "dataPath": "[0]",
    "schemaPath": "#/type",
    "params": {
      "type": "array"
    },
    "message": "should be array"
  },

This failure is the easiest to identify. Bad input string.

  {
    "keyword": "minLength",
    "dataPath": "[0].name",
    "schemaPath": "#/definitions/stage/properties/name/minLength",
    "params": {
      "limit": 1
    },
    "message": "should NOT be shorter than 1 characters"
  },

This failure is caused because one of the anyOf clauses failed due to the name value.

  {
    "keyword": "anyOf",
    "dataPath": "[0]",
    "schemaPath": "#/items/anyOf",
    "params": {},
    "message": "should match some schema in anyOf"
  }
]

Thanks for your help. I think I was looking for only the failure of the deepest leaf to be propagated.

[epoberezkin]

That is correct. Because both subschemas fail, “anyOf” also fails, so type error and minLength error precede anyOf error. Ajv is kind of dumb here - it has no knowledge about keyword semantics - it is just wired to report failed keywords.

It is a very common source of pain, to be honest. Maybe it is possible to make error reporting smarter when only errors from less correct schemas would be reported. Not sure how to make it predictable in general case though. Maybe the solution would be an extension of explicit discriminator keyword from OpenAPI when Ajv would ignore errors from branches where specific keywords fail (e.g. type in this case). Or maybe it’s possible to detect mutually exclusive parts of anyOf branches... But if/then/else is definitely a viable workaround that would lead to more sane error output.

[mseeley]

Or maybe it’s possible to detect mutually exclusive parts of anyOf branches...

I've been thinking down this path to try and humanize the output. I definitely see how it could get painful, especially once the schemas recurse.

Below is an edited list of failures for a trivial nested example: [[{ "name": "" }]];.

[
  {
    "keyword": "type",
    "dataPath": "[0][0]",
    "schemaPath": "#/type",
  },
  {
    "keyword": "minLength",
    "dataPath": "[0][0].name",
    "schemaPath": "#/definitions/stage/properties/name/minLength",
  },
  {
    "keyword": "anyOf",
    "dataPath": "[0][0]",
    "schemaPath": "#/items/anyOf",
  },
  {
    "keyword": "type",
    "dataPath": "[0]",
    "schemaPath": "#/definitions/stage/type",
  },
  {
    "keyword": "anyOf",
    "dataPath": "[0]",
    "schemaPath": "#/items/anyOf",
  }
]

For my usecase I believe it'd be possible to find the deepest dataPath. Then (waving hands) be smart about handling cases where the deepest dataPath can be found on multiple errors. Reducing the errors or sorting by dataPath depth seems like a reasonable job for a consumer. I agree with you, I don't see a general implementation pattern.

I'm struggling a bit to find a way to apply if/then/else (hence looking for ways to reduce across the errors). I'm open to any suggestions but it'd be heading off topic.

Thanks a lot for work on the library and patient explanations. Please feel free to close, no problem here, just users being users. :)

[epoberezkin]

Try {if: {type: “object”}, then: {$ref: “#stage”}, else: {$ref: “#stages”}}

[mseeley]

Oof that was pretty trivial 😳, thanks! Resulting schema object:

{
  $schema: "http://json-schema.org/draft-07/schema",
  $ref: "#/definitions/stages",
  definitions: {
    stages: {
      type: "array",
      minItems: 1,
      items: {
        if: { type: "object" },
        then: { $ref: "#/definitions/stage" },
        else: { $ref: "#/definitions/stages" }
      }
    },
    stage: {
      type: "object",
      properties: {
        name: {
          type: "string",
          minLength: 1
        }
      }
    }
  }
}

I still receive multiple errors but understand why and how to work around. Example errors: I was running with allErrors: true.

[
  {
    "keyword": "if",
    "dataPath": "/0",
    "schemaPath": "#/items/if",
    "params": {
      "failingKeyword": "then"
    },
    "message": "should match \"then\" schema"
  },
  {
    "keyword": "minLength",
    "dataPath": "/0/name",
    "schemaPath": "#/definitions/stage/properties/name/minLength",
    "params": {
      "limit": 1
    },
    "message": "should NOT be shorter than 1 characters"
  }
]

I've got some ideas to humanize the errors. I'll come back and share once it's shiny.

Thank you very much! 🏆

[mseeley]

I'm going to close this issue out, you've been more than accommodating. 👋