Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redact conn secrets in webserver logs #16579

Merged
merged 2 commits into from Jun 22, 2021
Merged

Conversation

msumit
Copy link
Contributor

@msumit msumit commented Jun 22, 2021

Found sensitive connection attributes in webserver logs while testing a connection from UI.

Before:

[2021-06-22 12:00:32,341] {base.py:80} INFO - Using connection to: id: VmAyCbqf. Host: https://www.httpbin.org/, Port: None, Schema: , Login: admin, Password: admin, extra: {'access_token': '123456', 'foo': 'bar'}

After:

[2021-06-22 12:04:14,162] {base.py:80} INFO - Using connection to: id: 4P0GvhP3. Host: https://www.httpbin.org/, Port: None, Schema: , Login: admin, Password: ***, extra: {'access_token': '***', 'foo': 'bar'}

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@boring-cyborg boring-cyborg bot added the area:core-operators Operators, Sensors and hooks within Core Airflow label Jun 22, 2021
@msumit msumit requested a review from ashb June 22, 2021 06:39
Copy link
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah I'd changed it to only redact the Task logs.

The other option would be to have the redact filter apply to all of the webserver logs, but that feels heavy weight.

Is the Test Connection button available in 2.1 or is it new in main/2.2? Yup, just checked, new in 2.2.

@msumit
Copy link
Contributor Author

msumit commented Jun 22, 2021

Ah, yeah I'd changed it to only redact the Task logs.

The other option would be to have the redact filter apply to all of the webserver logs, but that feels heavy weight.

Is the Test Connection button available in 2.1 or is it new in main/2.2? Yup, just checked, new in 2.2.

Yeah, I too thought it to apply for all logs, but then it feels like overburdening web server, hence used specifically.

@msumit msumit merged commit 2a59de3 into apache:main Jun 22, 2021
@msumit msumit deleted the redact_conn branch June 22, 2021 09:10
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 22, 2021
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 23, 2021
(cherry picked from commit 2a59de3)
(cherry picked from commit 9ac87a9)
kaxil pushed a commit to astronomer/airflow that referenced this pull request Jun 23, 2021
(cherry picked from commit 2a59de3)
(cherry picked from commit 9ac87a9)
(cherry picked from commit 836803b)
Jorricks pushed a commit to Jorricks/airflow that referenced this pull request Jun 24, 2021
@uranusjr
Copy link
Member

Since BaseHook.get_connection() is public API, this should probably still be backported to 2.1. See #17021.

@potiuk potiuk added this to the Airflow 2.1.3 milestone Jul 16, 2021
@potiuk
Copy link
Member

potiuk commented Jul 16, 2021

God call. Marked it as 2.1.3

jhtimmins pushed a commit that referenced this pull request Aug 9, 2021
jhtimmins pushed a commit that referenced this pull request Aug 13, 2021
kaxil pushed a commit that referenced this pull request Aug 17, 2021
jhtimmins pushed a commit that referenced this pull request Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core-operators Operators, Sensors and hooks within Core Airflow
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants