Hi,
Below is a repost of an issue I raised on stackoverflow and I was advised this is a better place to go into more details.
I am submitting a dask distributed task with submit() and placing an add_done_callback on the future. In the current dask implementation, the callback is called in a separate thread from the one where the callback was placed. Sometimes an exception can occur in this thread and I'd like to throw this exception in the main thread that placed this callback.
My workaround solution is to sandwich all the code in this callback between a try/except and just print a notification that a problem occurred. Ideally I would like to have access to the whole exception and find out where the problem occurred.
My questions are:
- Is there a better way to handle this problem?
- Is it practical or possible to run the done callback in the main thread that placed it similar to how ipyparallel is handling this?
Thank you!
Hi,
Below is a repost of an issue I raised on stackoverflow and I was advised this is a better place to go into more details.
I am submitting a dask distributed task with submit() and placing an add_done_callback on the future. In the current dask implementation, the callback is called in a separate thread from the one where the callback was placed. Sometimes an exception can occur in this thread and I'd like to throw this exception in the main thread that placed this callback.
My workaround solution is to sandwich all the code in this callback between a try/except and just print a notification that a problem occurred. Ideally I would like to have access to the whole exception and find out where the problem occurred.
My questions are:
Thank you!