Error reporting

[silvanshade]

This PR adds error diagnostics based on @alanz's implementation mentioned here and is intended to fix #255.

I haven't had a chance to actually experiment with the results yet but I just wanted to get this PR in place so that there is some visibility for this. It would really be nice to have some sort of functionality like this ready for 1.0 (see #930).

Ultimately, I think it may be more useful to have JSON output of these error diagnostics (at least as an option) rather than plain strings. I will probably try to add that to this PR before marking it as ready.

/cc @maxbrunsfeld @alanz

[maxbrunsfeld]

👋 I think this is not quite the right way to report errors. To generate a useful error message, I think we need to actually indicate the location where the error was detected. Right now, this information is not stored: when recovering from an error, we pick out an arbitrary sequence of nodes surrounding the error-detection-point, and wrap those nodes in an ERROR.

What we need to do is tweak the error recovery procedure (during parsing) so that it records where the error initially occurred, and store this information on the ERROR node. I think it could be stored in terms of a child index (e.g. the first node after the error-detection-point is the nth child of the ERROR).

Then, we would be able to compute what parse state we were in at the time the error was detected, and look up the set of tokens that would be valid in that parse state.

[silvanshade]

@maxbrunsfeld After having experimented with this some, I agree with you. I was thinking the output would perhaps be in a tree form or something, but it's basically what you'd get from walking the tree (with a cursor) and collecting errors, which is something I already do in my language servers.

I'd prefer to have something more like what you are describing where the actual parser state is described when an error occurs.

I'll try to take a look at the parser to see if I can figure out how to do what you described. Any further suggestions you might have along those lines are appreciated.