[AIRFLOW-1836] airflow uses OAuth Provider keycloak #2799

fisher-monkey · 2017-11-20T11:30:28Z

Dear Airflow maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-1836

Description

Here are some details about my PR, including screenshots of any UI changes:

Tests

My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"

codecov-io · 2017-11-20T11:58:07Z

Codecov Report

Merging #2799 into master will decrease coverage by 3.14%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2799      +/-   ##
==========================================
- Coverage   76.67%   73.52%   -3.15%     
==========================================
  Files         199      158      -41     
  Lines       16186    11958    -4228     
==========================================
- Hits        12410     8792    -3618     
+ Misses       3776     3166     -610

Impacted Files	Coverage Δ
airflow/operators/email_operator.py	`0% <0%> (-100%)`	⬇️
airflow/hooks/pig_hook.py	`0% <0%> (-100%)`	⬇️
airflow/operators/slack_operator.py	`0% <0%> (-97.37%)`	⬇️
airflow/operators/s3_file_transform_operator.py	`0% <0%> (-96.23%)`	⬇️
airflow/operators/redshift_to_s3_operator.py	`0% <0%> (-95.46%)`	⬇️
airflow/hooks/mssql_hook.py	`6.66% <0%> (-66.67%)`	⬇️
airflow/hooks/hdfs_hook.py	`32.5% <0%> (-60%)`	⬇️
airflow/operators/hive_operator.py	`41.02% <0%> (-45.52%)`	⬇️
airflow/hooks/S3_hook.py	`56.97% <0%> (-37.35%)`	⬇️
airflow/hooks/hive_hooks.py	`39.52% <0%> (-33.9%)`	⬇️
... and 196 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e703d6b...4edc73c. Read the comment docs.

Fokko · 2017-11-21T06:39:20Z

Hi @fisher-monkey, thanks for contributing. Could you open a Jira ticket and add a description about the implementation that you've did?

Fokko · 2017-11-21T06:40:58Z

airflow/contrib/auth/backends/keycloak_auth.py

+
+    def get_ghe_user_profile_info(self, ghe_token):
+        #resp = self.ghe_oauth.get(self.ghe_api_route('/userinfo'),token=(ghe_token, ''))
+        #resp=self.ghe_oauth.get('http://10.110.13.147:9090/auth/realms/brent/protocol/openid-connect/userinfo',token=(ghe_token,''))


Please remove the comments

I have create a Jira ticket and simply describe the implementation, and this is the link: https://issues.apache.org/jira/browse/AIRFLOW-1836

the comments have been removed.

Fokko · 2017-11-21T06:41:15Z

airflow/contrib/auth/backends/keycloak_auth.py

+
+            username, email = self.get_ghe_user_profile_info(ghe_token)
+        except AuthenticationError:
+            _log.exception('')


An empty log line

Fokko · 2017-11-21T06:43:13Z

airflow/contrib/auth/backends/keycloak_auth.py

+from airflow.configuration import AirflowConfigException
+
+import os
+os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'


I think this should be disabled by default

ron819 · 2018-10-03T13:08:12Z

@fisher-monkey @Fokko is there any progress with this?

c4milo · 2018-10-04T20:23:40Z

Would it be possible to make this more generic and support any OpenIDConnect/OAuth2 identity provider? being KeyCloak one of them.

Fokko · 2018-10-12T09:46:16Z

Very good suggestion @c4milo

ron819 · 2018-11-14T12:31:52Z

@Fokko Does the suggestion to make it generic prevents from this to be merged or it's a feature request for future development? We do have other cases where non-generic PRs have been submitted. Example: #3941

Fokko · 2018-11-18T17:32:49Z

I think it is for #3941 hard to implement it generic, i.e. have some logic to test the connection of an arbitrary connection. Implementing OAuth makes perfect sense. What do you suggest @ron819 ?

stale · 2019-01-02T21:02:40Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

weldpua2008 · 2020-08-12T22:19:30Z

@Fokko / @fisher-monkey Could you please inform if keycloak is supported in Airflow?
We started to use keycloak and I need to configure it.

mik-laj · 2020-08-12T23:29:33Z

@weldpua2008 You can configure any OAuth2 provider, but I am working on a Keycloak-proxy integration as it is much more secure. It's not public yet, but I have plans to share it with the community.

farisabdulraheem · 2020-10-20T10:53:07Z

How can i implement keycloak auth with airflow? @fisher-monkey @Fokko is there a simple way for that, any sample code available ?

mik-laj · 2020-10-20T10:57:22Z

this PR refers to the old and unsupported user interface. We support RBAC (Flask AppBuilser based) only now.

farisabdulraheem · 2020-10-20T11:02:07Z

@mik-laj so how can i use it with keycloak using rbac

weldpua2008 · 2021-01-27T12:49:40Z

Any news?

airflow uses OAuth Provider keycloak

4d37771

Fokko reviewed Nov 21, 2017

View reviewed changes

Update keycloak_auth.py

4edc73c

ashb changed the title ~~airflow uses OAuth Provider keycloak~~ [AIRFLOW-1836] airflow uses OAuth Provider keycloak Nov 14, 2018

stale bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Jan 2, 2019

stale bot closed this Jan 10, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AIRFLOW-1836] airflow uses OAuth Provider keycloak #2799

[AIRFLOW-1836] airflow uses OAuth Provider keycloak #2799

fisher-monkey commented Nov 20, 2017 •

edited by ashb

codecov-io commented Nov 20, 2017 •

edited

Fokko commented Nov 21, 2017

Fokko Nov 21, 2017

fisher-monkey Nov 21, 2017

Fokko Nov 21, 2017

Fokko Nov 21, 2017

ron819 commented Oct 3, 2018

c4milo commented Oct 4, 2018

Fokko commented Oct 12, 2018

ron819 commented Nov 14, 2018

Fokko commented Nov 18, 2018

stale bot commented Jan 2, 2019

weldpua2008 commented Aug 12, 2020

mik-laj commented Aug 12, 2020

farisabdulraheem commented Oct 20, 2020

mik-laj commented Oct 20, 2020

farisabdulraheem commented Oct 20, 2020

weldpua2008 commented Jan 27, 2021

[AIRFLOW-1836] airflow uses OAuth Provider keycloak #2799

[AIRFLOW-1836] airflow uses OAuth Provider keycloak #2799

Conversation

fisher-monkey commented Nov 20, 2017 • edited by ashb

JIRA

Description

Tests

Commits

codecov-io commented Nov 20, 2017 • edited

Codecov Report

Fokko commented Nov 21, 2017

Fokko Nov 21, 2017

Choose a reason for hiding this comment

fisher-monkey Nov 21, 2017

Choose a reason for hiding this comment

Fokko Nov 21, 2017

Choose a reason for hiding this comment

Fokko Nov 21, 2017

Choose a reason for hiding this comment

ron819 commented Oct 3, 2018

c4milo commented Oct 4, 2018

Fokko commented Oct 12, 2018

ron819 commented Nov 14, 2018

Fokko commented Nov 18, 2018

stale bot commented Jan 2, 2019

weldpua2008 commented Aug 12, 2020

mik-laj commented Aug 12, 2020

farisabdulraheem commented Oct 20, 2020

mik-laj commented Oct 20, 2020

farisabdulraheem commented Oct 20, 2020

weldpua2008 commented Jan 27, 2021

fisher-monkey commented Nov 20, 2017 •

edited by ashb

codecov-io commented Nov 20, 2017 •

edited