Brainstorming for design of messages

[lwjohnst86]

Put ideas here. Obviously we are constrained by what is possible in Python, but this is to discuss some ideas for improving the messages of the issues detected.

[martonvago]

Initial thoughts:

Things to include in a message about a single failed check:

1. Which check failed. In jsonschema each check has a name / type (required, pattern, type, etc.), and custom checks can also be named when they are defined. This is useful for filtering checks. I would find it useful to include this in the message as well because it would make it possible to scan all the messages quickly to get an overview of what failed.
2. Which field in the descriptor failed the check. I still think that a string JSON path (e.g. $.resources[0].schema.fields[0].description) is a pretty transparent way of locating the error unambiguously, and this is what jsonschema uses in its output as well. I understand if it is too technical though. We could replace $ for the root node with package, for example? Or maybe something like this: package > 1st resource > schema > 1st field > description? Not sure if this is that much easier to understand though...
3. The offending value itself. Not sure if it is always possible/useful to provide this. E.g., the value could be missing or it could be a full dictionary object with its own structure, which is difficult to print nicely.

All of these could be spliced together into a sentence or two.
For example:

type check failed: the value 1 at location package.resources[0].schema.fields[0].description is not of type string.

Or is this still too technical?

One inconvenience here is that some bits of information may not be exposed by jsonschema as separate properties and can only be extracted from their error message. The only way to know is to go through all JSON schema error types that are possible in the Data Package schema.

[lwjohnst86]

Hmm, yea, still too technical/complicated/not-helpful. An idea (building off of your example):

We identified an issue at:

    "description": 1
                   ^

Which is nested under `package > resources > 1st entry > schema > fields > 1st entry` (or as a JSON path, it would be `$package.resources[0].schema.fields[0].description`).

This item should be a string, but we identified it as a number. Please fill it in with actual text, for example:

    "description": "An explanation of this field in the schema."

Not sure I love the item > subitem, but it is better than the JSON path (which only makes sense if you are familiar with JSON paths).

[martonvago]

Sounds good to me!

The main challenge I can foresee is that it will probably be quite faffy to go through all the possible error types output by jsonschema and rewrite the error messages.

The only other idea I have for displaying the nesting is treating it like a directory structure, i.e.:

package
├─ resources
│  ├─ my-resource-name
│  │  ├─ schema
│  │  │  ├─ fields
│  │  │  │  ├─ my-field-name
│  │  │  │  │  ├─ description

(Could be rendered in a nicer way though.)

[lwjohnst86]

The file tree structure could work too! And yea, it might take some wrangling to get them to output nicely. I guess we for now only need to search through what is in the data package standard.

[joelostblom]

I was reading through the guide on excluding checks and noticed that understanding json syntax such as $, ., and [0] is required to write exclusions. Given that we expect readers of this guide to be able to learn json syntax in that context, it seems reasonable to also expect them to learn it in the context of reading the error messages? Particularly since both sprout and check-datapackage are geared toward technical roles such as data engineers.

I personally find that the other suggestions here would have a tendency to create rather verbose error messages, which can become an annoyance when reading many of them. Therefore, I'm wonder if instead of reformatting each error message, it would be more beneficial to clearly explain how to interpret them in the guide for those that don't understand them on their own. Again, I think most data engineers will quickly understand what is going on in those messages (without specific json experience, which many might actually even have); the only possibly obscure-looking detail is the leading dollar sign which could possibly be omitted (either way works I think).

[lwjohnst86]

I think any time you start configuring things, you naturally need to have more understanding for stuff. I don't think we need to assume someone who is just using this package as is needs to know JSON path syntax. If they want to customize things, then of course, you need to learn more things about how the checks work. But if not, the output shouldn't depend on them reading the guide just to use the basic functionality.

As for my own suggestion, that was based on how Rust outputs its error messages, which I found very helpful when trying to understand how to fix problems. (rather than having to know to look for and in a guide).

[lwjohnst86]

As for assuming data engineers/other technical people would know. I had no idea what JSON path syntax was and I'm a pretty technical person. I won't be the only one who won't know.