Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix redacting secrets in context exceptions. #17618

Merged
merged 2 commits into from Aug 14, 2021

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Aug 14, 2021

Secret masking did not work in implicit and
explicit context exceptions (see
https://www.python.org/dev/peps/pep-3134/)
When there was a try/except/raise sequence,
or raise ... from exception - the original
exceptions were not redacted.

Related: #17604


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@potiuk
Copy link
Member Author

potiuk commented Aug 14, 2021

@kaxil @jhtimmins - we might want to include that in 2.1.3

@potiuk potiuk added this to the Airflow 2.1.3 milestone Aug 14, 2021
@potiuk potiuk force-pushed the fix-secret-masking-in-context-exceptions branch from 00981d2 to 2a047fd Compare August 14, 2021 18:25
@potiuk potiuk changed the title Fixes redacting secrets in context exceptions. Fix redacting secrets in context exceptions. Aug 14, 2021
@potiuk potiuk force-pushed the fix-secret-masking-in-context-exceptions branch from 2a047fd to b8e43d9 Compare August 14, 2021 18:27
@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Aug 14, 2021
@github-actions
Copy link

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

airflow/utils/log/secrets_masker.py Show resolved Hide resolved
airflow/utils/log/secrets_masker.py Show resolved Hide resolved
@ashb
Copy link
Member

ashb commented Aug 14, 2021

try:
     try:
         raise Exception("user:pass")
     except Exception as e:
         # This is how you get an exception with a __cause__
         raise Exception("two") from e
 
except Exception as e:
     err = e

Secret masking did not work in implicit and
explicit context exceptions (see
https://www.python.org/dev/peps/pep-3134/)
When there was a `try/except/raise` sequence,
or `raise ... from` exception - the original
exceptions were not redacted.

Related: apache#17604
@potiuk potiuk force-pushed the fix-secret-masking-in-context-exceptions branch from b8e43d9 to b3f878a Compare August 14, 2021 19:51
@potiuk potiuk requested a review from ashb August 14, 2021 19:51
@potiuk
Copy link
Member Author

potiuk commented Aug 14, 2021

Added __cause__ handling. Good call @ashb!

@potiuk
Copy link
Member Author

potiuk commented Aug 14, 2021

Interesting thing... Actually that would work even before latest change, because in this case, the "cause" is also in "context":

try:
     try:
         raise Exception("user:pass")
     except Exception as e:
         # This is how you get an exception with a __cause__
         raise Exception("two") from e
 
except Exception as e:
     err = e

But this wuldn't work:

exception = None
try:
     try:
         raise Exception("user:pass")
     except Exception as e:
         # This is how you get an exception with a __cause__
         exception =  e
try:
    raise Exception("x") from exception
except Exception as e:
     err = e

@ashb
Copy link
Member

ashb commented Aug 14, 2021

Ahh cause and context are the same object in some cases:

In [12]: err.__context__ is err.__cause__
Out[12]: True

Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
@potiuk potiuk merged commit 6df3ee7 into apache:main Aug 14, 2021
@potiuk potiuk deleted the fix-secret-masking-in-context-exceptions branch August 14, 2021 20:49
potiuk added a commit that referenced this pull request Aug 14, 2021
* Fix redacting secrets in context exceptions.

Secret masking did not work in implicit and
explicit context exceptions (see
https://www.python.org/dev/peps/pep-3134/)
When there was a `try/except/raise` sequence,
or `raise ... from` exception - the original
exceptions were not redacted.

Related: #17604

(cherry picked from commit 6df3ee7)
@potiuk
Copy link
Member Author

potiuk commented Aug 14, 2021

I cherry-picked that one to v2-1 @jhtimmins @kaxil

kaxil pushed a commit that referenced this pull request Aug 17, 2021
* Fix redacting secrets in context exceptions.

Secret masking did not work in implicit and
explicit context exceptions (see
https://www.python.org/dev/peps/pep-3134/)
When there was a `try/except/raise` sequence,
or `raise ... from` exception - the original
exceptions were not redacted.

Related: #17604

(cherry picked from commit 6df3ee7)
jhtimmins pushed a commit that referenced this pull request Aug 17, 2021
* Fix redacting secrets in context exceptions.

Secret masking did not work in implicit and
explicit context exceptions (see
https://www.python.org/dev/peps/pep-3134/)
When there was a `try/except/raise` sequence,
or `raise ... from` exception - the original
exceptions were not redacted.

Related: #17604

(cherry picked from commit 6df3ee7)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:logging full tests needed We need to run full set of tests for this PR to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants