Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-843] Store exceptions on task_instance #2135

Merged
merged 2 commits into from
Oct 16, 2018

Conversation

thesquelched
Copy link
Contributor

Store exceptions encountered executing a task on the task instance
object, making it available for on_failure_callback handlers.

Dear Airflow Maintainers,

Please accept this PR that addresses the following issues:

Testing Done:

  • Added unit test to check that running a bash operator that returns a non-zero exit code stores the appropriate exception on the task instance

@mention-bot
Copy link

@thesquelched, thanks for your PR! By analyzing the history of the files in this pull request, we identified @mistercrunch, @bolkedebruin and @saguziel to be potential reviewers.

@thesquelched
Copy link
Contributor Author

Tests fail on unrelated LDAP issue; should be good to merge

@alexaltair
Copy link

Just went to Issues to file this exact thing. Happy to see someone's on it!

@codecov-io
Copy link

codecov-io commented Nov 29, 2017

Codecov Report

Merging #2135 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2135      +/-   ##
==========================================
+ Coverage   75.92%   75.93%   +<.01%     
==========================================
  Files         199      199              
  Lines       15954    15956       +2     
==========================================
+ Hits        12113    12116       +3     
+ Misses       3841     3840       -1
Impacted Files Coverage Δ
airflow/models.py 91.99% <100%> (+0.04%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a581cba...e584190. Read the comment docs.

Copy link
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better late then never on the review, eh?

Happy to pick this up and make the changes (keeping you as the commiter) if you don't have time to make these changes.

@@ -1574,6 +1574,8 @@ def dry_run(self):
def handle_failure(self, error, test_mode=False, context=None, session=None):
self.log.exception(error)
task = self.task
session = settings.Session()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this line for? It seems like it does nothing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cruft from testing, I suppose; I'll remove

@@ -1574,6 +1574,8 @@ def dry_run(self):
def handle_failure(self, error, test_mode=False, context=None, session=None):
self.log.exception(error)
task = self.task
session = settings.Session()
self.exception = error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a property to the TaskInstance object feels the wrong place for this - the context object feels like a more appropriate place.

@thesquelched
Copy link
Contributor Author

I've just been lazy; I'll look at this again

@xnuinside
Copy link
Contributor

@thesquelched, do you have any plans for updating this PR?

Store exceptions encountered executing a task on the task instance
object, making it available for on_failure_callback handlers.
@thesquelched
Copy link
Contributor Author

@xnuinside sorry for the long wait; rebased and addressed PR comments

@thesquelched
Copy link
Contributor Author

@ashb looks good now?

@ashb
Copy link
Member

ashb commented Oct 16, 2018

Perfect, thanks! We'll get this in to 1.10.1

@ashb ashb merged commit 74c613b into apache:master Oct 16, 2018
@thesquelched thesquelched deleted the AIRFLOW-843 branch October 16, 2018 17:23
ashb pushed a commit that referenced this pull request Nov 6, 2018
…2135)

Store exceptions encountered executing a task in the context dict,
making it available for on_failure_callback handlers.
galak75 pushed a commit to VilledeMontreal/incubator-airflow that referenced this pull request Nov 23, 2018
…pache#2135)

Store exceptions encountered executing a task in the context dict,
making it available for on_failure_callback handlers.
aliceabe pushed a commit to aliceabe/incubator-airflow that referenced this pull request Jan 3, 2019
…pache#2135)

Store exceptions encountered executing a task in the context dict,
making it available for on_failure_callback handlers.
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Jan 23, 2019
Store exceptions encountered executing a task in the context dict,
making it available for on_failure_callback handlers.
wmorris75 pushed a commit to modmed/incubator-airflow that referenced this pull request Jul 29, 2019
…pache#2135)

Store exceptions encountered executing a task in the context dict,
making it available for on_failure_callback handlers.
rajatsri28 pushed a commit to rajatsri28/airflow that referenced this pull request Jan 6, 2020
…pache#2135)

Store exceptions encountered executing a task in the context dict,
making it available for on_failure_callback handlers.
msumit pushed a commit to twitter-forks/airflow that referenced this pull request Jan 7, 2020
…back (apache#2135) (#28)

Store exceptions encountered executing a task in the context dict,
making it available for on_failure_callback handlers.

Co-authored-by: Scott Kruger <scott@chojin.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants