Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle soft fail for s3keysensor #1161

Merged
merged 3 commits into from Jun 5, 2023
Merged

Handle soft fail for s3keysensor #1161

merged 3 commits into from Jun 5, 2023

Conversation

pankajastro
Copy link
Contributor

@pankajastro pankajastro commented Jun 2, 2023

if soft_fail is set to true then raise AirflowSkipException. We may need to replicate this to other sensor too

closes: #1149

@codecov
Copy link

codecov bot commented Jun 2, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (8c80e16) 98.58% compared to head (2cfbae5) 98.58%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1161   +/-   ##
=======================================
  Coverage   98.58%   98.58%           
=======================================
  Files          90       90           
  Lines        5368     5377    +9     
=======================================
+ Hits         5292     5301    +9     
  Misses         76       76           
Impacted Files Coverage Δ
astronomer/providers/amazon/aws/sensors/s3.py 100.00% <100.00%> (ø)
astronomer/providers/amazon/aws/triggers/s3.py 100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Collaborator

@pankajkoti pankajkoti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we paste a screenshot of a DAG run when the sensor failed and the task was marked skipped. Looks good to me otherwise. Have a minor comment wrt. to the exception message in the case of poke.

astronomer/providers/amazon/aws/sensors/s3.py Outdated Show resolved Hide resolved
tests/amazon/aws/sensors/test_s3_sensors.py Outdated Show resolved Hide resolved
tests/amazon/aws/sensors/test_s3_sensors.py Outdated Show resolved Hide resolved
tests/amazon/aws/sensors/test_s3_sensors.py Outdated Show resolved Hide resolved
@pankajastro
Copy link
Contributor Author

Can we paste a screenshot of a DAG run when the sensor failed and the task was marked skipped. Looks good to me otherwise. Have a minor comment wrt. to the exception message in the case of poke.

Screenshot 2023-06-05 at 6 39 17 PM

@pankajastro
Copy link
Contributor Author

Merging this but feel free to raise any concerns I will address them separately

@pankajastro pankajastro merged commit 91eb7a4 into main Jun 5, 2023
10 checks passed
@pankajastro pankajastro deleted the handle_soft_fail branch June 5, 2023 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

S3KeySensorAsync does not respect soft_fail
2 participants