Swallow Python exceptions better on workflow GC from eviction #341
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sushisource
approved these changes
Jul 10, 2023
Comment on lines
+659
to
+660
| logger.debug( | ||
| "Ignoring exception while deleting workflow", exc_info=True |
There was a problem hiding this comment.
Very nit but IMO this is more like trace level
There was a problem hiding this comment.
Was a tough call, but figured since I remove the two primary cases where this happens, the rest I think are fairly rare. And in fact, this can show a sign of a code smell, because it might mean you are swallowing an exception in a finally if it's not one of these two exception types.
# Conflicts: # temporalio/workflow.py
…k-python into generator-exit
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What was changed
Python raises
GeneratorExitwhen GC'ing coroutines that are beingawaited on. So when our workflow is evicted, users were seeing uncaught exception warnings. Also, if they hadfinallylogic that did something with the workflow, the event loop is already unset so that error is raised too, causing Python warning, sometimes swallowingGeneratorExit(which causes another type of Python exception).This no longer logs
GeneratorExitor no-workflow-runtime exceptions when deleting, and all other exceptions during delete are just logged at the debug levelChecklist