Error info content is not language-agnostic

[gibiansky]

I'm not sure whether this should be addressed or how to address it, but the standard error messaging mechanism in the messaging spec is not particularly language agnostic. Specifically, it requires that the kernel provide an error name, an error value, and an error traceback – while these things make sense in many contexts, many environments and languages might not have distinct concepts of error name vs error value, or even have a traceback to include.

(For example, until very recently Haskell did not provide stack traces, so tracebacks were not relevant; I'm not even sure that interpreted Haskell as in IHaskell can currently provide stack traces.)

Whether or not this is significant enough to be worth fixing I'm not sure, but figured I'd point it out for future discussion. I'm not sure what's gained by separating these fields rather than just having the error message include a display_data mimebundle, in the same way that results tend to do.

[takluyver]

I've been thinking for a while that in a future version of the message spec, we may deprecate error messages and the error fields in execute_reply. There are really two bits of indicating an error:

• Indicate to the frontend that an error occurred - use status: error in the reply. We don't really need a machine-readable indication of the kind of error.
• Display error information to the user - this can be done with normal output messages like display_data; there's no obvious reason a different message type is needed.

[gibiansky]

Ah, good to know. FYI that is actually how IHaskell has always worked – although it caused slightly different behaviour it simply did not use any error messages and sent extra display data messages.

[takluyver]

I think the only thing where the frontend actually uses the error information for anything other than display is when you do 'run all' in the notebook, to stop after an error.

[gibiansky]

Makes sense – I was thinking about this recently and I think in that case IHaskell actually does the wrong thing (and will run all the cells) as a result of the fact that errors don't get reported via error messages.

[takluyver]

I forget whether it uses the error message or the status: error in the execute_reply - if we do follow my idea, it would obviously have to be the status.

[minrk]

Since errors are output and should be displayed by all frontends, I think we have to keep the error IOPub message. We can move those to the mimebundle-style output to let kernels take care of rendering, or try to come up with a generic structured-traceback that we've talked about many times, allowing nice links to lines containing errors, etc.

[takluyver]

Since errors are output and should be displayed by all frontends,

We could display in all frontends with a display_data message; do you think it's important that all frontends have a machine-readable representation of the error?

[minrk]

It's been discussed and planned in vague terms since 0.12, so it's worth considering. I'm not sure how feasible a language-agnostic structured errors are, though.

[rgbkrk]

What if, instead of trying to create one generic language-agnostic structured error, there were media types with JSON payloads for each language? It would be up to the kernel authors to define the spec for their errors (for a frontend to consume).

[minrk]

@rgbkrk how does the kernel add to the frontend the ability to consume their custom error data (where frontend includes nbconvert, nteract, notebook, jupyterlab, etc.)?

[rgbkrk]

And the jupyter console!

how does the kernel add to the frontend the ability to consume their custom error data (where frontend includes nbconvert, nteract, notebook, jupyterlab, etc.)?

From a high level, some type of schema for their error messages and a resulting implementation in the frontend. The frontend would have an implementation of rendering/handling errors across many kernels with different error schemas or there is some way to register those dynamically. I'd rather bundle support for many languages directly, similar to codemirror modes.

I'd assume that there's a generic error reporting mechanism that any frontend can consume and a richer optional component that a frontend can ignore.

[Carreau]

Things I have been reading recently:

https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html
http://elm-lang.org/blog/compiler-errors-for-humans

There is probably a pattern we can find in there that should work in the 95% case and make error friendly for beginners. I think minimally error likely can have a "short version" and and "long version". The long version being often the long traceback that scare people away.

[minrk]

I guess my first question is where do we want to aim in between:

1. structured traceback data, which the frontend interprets (complex, powerful, probably hard to spec in a language-agnostic way)
2. mimebundles for errors, just like display

The mimebundles will let you have arbitrarily fancy representation in different contexts, but would be hard to do much-sought-after things like linking to the line in the cell (or external source file??) where the error occurred.

As a protocol maintainer, I like mimebundles because they are well defined and easy to support/implement. But they are punting on the problem a bit (just like we already did for inspection), and probably limit the scope of what can be accomplished.