Better Python Errors

[johnc1231]

This PR is based on discussion here: https://dev.hail.is/t/better-python-error-messages-idea/201/9 The intention is to create a system to give better error messages from python in a generic way. Tim's work in #7792 does a good job introducing behavior like this this specifically for ArrayRef nodes, but I want to add three things on top of that:

1. I don't want to have to do as much custom per IR node work
2. I don't want to send the entire python stack trace over py4j to scala for every node
3. I don't want the user to see a Java stack trace in this scenarios.

This first PR is a proof of concept that adds this behavior for the Die node, which will catch any errors generated by uses of CaseBuilder.or_error. Follow up PRs should change ArrayRef to work this way, as well as catch things like looking up a key in a dictionary but not finding it. In an ideal future, we'd bolt on some extra mechanism to give types to these errors, and we could throw a proper IndexError in the ArrayRef case or KeyError in the dictionary case.

It feels a little bit messy right now, open to suggestions. I don't love using -1 as the "no error" situation, but I thought it was probably easier than dealing with optionals between python and scala.

To give an example of what it looks like, the error message for this script:

import hail as hl

ht = hl.utils.range_table(10)
ht = ht.annotate(foo = hl.nd.array([[1], [2], [3]]))
ht = ht.annotate(bar = ht.foo[0:4, 12])
ht.collect()

is

Traceback (most recent call last):
  File "better_error_test.py", line 6, in <module>
    ht.collect()
  File "<decorator-gen-1103>", line 2, in collect
  File "/Users/johnc/Code/hail/hail/python/hail/typecheck/check.py", line 614, in wrapper
    return __original_func(*args_, **kwargs_)
  File "/Users/johnc/Code/hail/hail/python/hail/table.py", line 1903, in collect
    return Env.backend().execute(e._ir)
  File "/Users/johnc/Code/hail/hail/python/hail/backend/spark_backend.py", line 325, in execute
    raise HailUserError(message_and_trace) from None
hail.utils.java.HailUserError: Error summary: HailException: Index 12 is out of bounds for axis 1 with size 1
------------
Hail stack trace:
  File "better_error_test.py", line 5, in <module>
    ht = ht.annotate(bar = ht.foo[0:4, 12])

  File "<decorator-gen-707>", line 2, in __getitem__

  File "/Users/johnc/Code/hail/hail/python/hail/expr/expressions/typed_expressions.py", line 3709, in __getitem__
    hl.str("Index ") + hl.str(s) + hl.str(f" is out of bounds for axis {i} with size ") + hl.str(dlen)

  File "<decorator-gen-1299>", line 2, in or_error

  File "/Users/johnc/Code/hail/hail/python/hail/expr/builders.py", line 311, in or_error
    die_ir.save_error_info()

(Note that ndarray bounds checking internally uses a case builder, which is why this example works).

[johnc1231]

One other question: Is it a breaking change to change the type of error we throw?

[patrick-schultz]

This is great. Overall design looks good, just a few minor things.

[patrick-schultz]

I think this can be simplified to

filt_stack = [
    candidate for candidate in stack[i:]
              if not any(phrase in candidate for phrase in forbidden_phrases)
]

[patrick-schultz]

Extraneous @

[patrick-schultz]

You added the constructor that sets the errorId to -1 implicitly; did you not want to use that here (and several other places)? That would keep the -1 magic number hard coded in fewer places.

[johnc1231]

Ah, I originally added the numeric argument everywhere manually because I didn't want to miss anywhere where I had to copy the argument along. I can go back now and remove the extras.

All addressed

[johnc1231]

Thanks for the review, all addressed now.

[tpoterba]

delete this?

[johnc1231]

Oh, that was a mistake where I typed "Invalid" instead of "Illegal", must have autoimported.

[tpoterba]

yeah, I do that all the time!

Fixed