Error reporting in case of multi option type

[P-Product]

Hi Julian,

this time I would like to ask something. Consider the following setup

my_schema = {
    "type": [
        "string",
        {
            "type": "array",
            "items": { "type": "number" },
            "additionalItems": False
        }
    ]
}

my_json = [1, "test"]


Draft3Validator.check_schema(my_schema)
print 'schema is valid'

v = Draft3Validator(my_schema)

valid = True
for error in sorted(v.iter_errors(my_json), key=str):
    valid = False
    print error

if valid: 
    print 'json is valid'

The output I get for this is

schema is valid
[1, 'test'] is not of type 'string', {'items': {'type': 'number'}, 'additionalItems': False, 'type': 'array'}

However it seems to me that the error should read

'test' is not of type 'number'

Do you think it is possible that in such case of multiple options an error of the deepest match should be presented?
Or, as an alternative, there may be a set of errors like

[1, 'test'] is not of type 'string'
'test' is not of type 'number'

[Julian]

Well. Those error messages are meant for developers, and they are meant to be able to show what validated against what and failed, so I'm a bit reluctant to have any possibly-relevant information be hidden there.

If that's something that's desirable for user-facing stuff though I think it'd be reasonable to add on top of jsonschema of course, either in a custom validator or as a function that operated on the errors.

[Julian]

Or I guess it'd be reasonable to show them in decreasing order as long as they all still appeared, but I couldn't guess at how much work it'd be to implement. Or your second idea is reasonable sounding, but it'd break the assumption that each error needs to be fixed on its own.

[P-Product]

OK. But the error reporting is there to be able to locate the error, right? The problem is that even from the developer perspective it is hard to understand in the shown case what actually is wrong. But you know what, this is a simple example. In my work i came over this because I had a much more obscure situation, with nested multiple options, when I had to manually validate the json to know what went wrong.

[P-Product]

If it was not important at all we would simply say that a json is invalid. But we are traversing down to the offensive members, so my point is that this should be as intelligent as possible, WDYT?

[P-Product]

I have come up with a more descriptive example

Schema:

my_schema = {
    "type": [
        "number",
        {
            "type": [
                {
                    "type": "array",
                    "items": {
                        "type": [
                            "number",
                            {
                                "type": "object",
                                "properties": {
                                    "value": { "type": "string", "required": True }
                                }                                
                            }
                        ]
                    },
                    "additionalItems": False
                },
                "number"
            ]
        }
    ]
}

now all three following instances

my_json1 = " 1, { \"value1\": 1 } "
my_json2 = [ 1, { "value": 1 } ]
my_json3 = [ 1, { "val": "1" } ]

give the same error:

'...' is not of type 'number', {'type': [{'items': {'type': ['number', {'type': 'object', 'properties': {'value': {'required': True, 'type': 'string'}}}]}, 'additionalItems': False, 'type': 'array'}, 'number']}

[Julian]

Well yeah if you have a really complicated thing it's going to be hard to find I guess.

So, I'm not inherently making this easier of course, but it'd depend on how much complication would be introduced by adding something like this.

Also, besides that, I'm not sure this is the best way to help. I have a sort of debug-mode kind of thing in mind to eventually implement which will show each validator method, what it was called with, and where it failed. I'm more inclined to think that that would be a general better solution.

But if you want to take a crack at this and it's not too complicated I'd merge it.

[Julian]

Continuing discussion in #51.