support custom exceptions from retries #530
Description
The retry wrapper currently raises a RetryError when the retry deadline is exhausted, with the last exception encountered as the cause. There are a couple issues with this approach:
DeadlineExceeded may be a more clear better exception to raise in some circumstances
The details of the retryable errors are lost, with only the most recent exception encountered. This could mean the loss of valuable debugging information. With Python 3.11's exception groups, all transient errors could be passed along to the user as a group
For python-bigtable, I attempted to build an exception group out of the RetryError by tracking exceptions encountered in the on_error callback, and then catching RetryErrors and converting the error to a new type within the exception handler code. But this was messy, and resulted in performance drops even when the rpc did not encounter errors.
Instead, I propose adding a new exception_factory parameter to the internal retry functions. This is a callback that accepts some details about the error, and returns a source_exception and cause_exception to be raised by the retry wrapper. We can provide a default implementation that maintains the status quo, but library developers can also use this as an extension point to customize the error with minimal performance overhead
def _build_retry_error(
exc_list: List[Exception], is_timeout: bool, timeout_val: float
) -> Tuple[Exception, Optional[Exception]]:
"""
Args:
- exc_list (list[Exception]): list of exceptions that occurred during the retry
- is_timeout (bool): whether the failure is due to the timeout value being exceeded,
or due to a non-retryable exception
- timeout_val (float): the original timeout value for the retry, for use in the exception message
Returns:
- a tuple of the exception to be raised, and the cause exception if any
"""
if is_timeout:
# return RetryError with the most recent exception as the cause
src_exc = exc_list[-1] if exc_list else None
return (
exceptions.RetryError(
"Timeout of {:.1f}s exceeded".format(timeout_val),
src_exc,
),
src_exc,
)
elif exc_list:
# return most recent exception encountered
return exc_list[-1], None
else:
# no exceptions were given in exc_list. Raise generic RetryError
return exceptions.RetryError("Unknown error", None), NoneImplementation
I implemented this parameter in the #495. If approved, I can add this functionality to the existing unary retry wrappers as well.