Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add backend for Hashicorp Vault (alternative approach) #669

Merged
merged 2 commits into from
Jun 10, 2022

Conversation

candlerb
Copy link
Contributor

Proposed changes

This is a more generic solution to PR #668 - for consideration.

In this PR, the OpenIdConnectAuth class is enhanced so that it can be instantiated directly, as a "generic" OIDC connector.

To enable this, it also makes the OAuth2 authentication mechanism configurable (i.e. form post or HTTP Basic), and uses OIDC discovery to select this automatically.

Vault then becomes an empty subclass of OpenIdConnectAuth, since it's an out-of-the-box OIDC provider.

Types of changes

Please check the type of change your PR introduces:

  • Release (new release request)
  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Code style update (PEP8, lint, formatting, renaming, etc)
  • Refactoring (no functional changes, no api changes)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Build related changes (build process, tests runner, etc)
  • Other (please describe):

I believe this shouldn't break existing code, and all tests pass, but it would be wise to check the other OpenIdConnectAuth subclasses just in case.

Checklist

Put an x in the boxes that apply. You can also fill these out after creating
the PR. If you're unsure about any of them, don't hesitate to ask. We're here to
help! This is simply a reminder of what we are going to look for before merging
your code.

  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works

Other information

This PR also includes a fix for okta test which surfaced as part of this change. I believe this should only fix the test, not the actual behavior of the okta backend.

@codecov
Copy link

codecov bot commented Feb 23, 2022

Codecov Report

Merging #669 (f92ac0d) into master (36f25c4) will increase coverage by 0.19%.
The diff coverage is 94.80%.

@@            Coverage Diff             @@
##           master     #669      +/-   ##
==========================================
+ Coverage   76.95%   77.15%   +0.19%     
==========================================
  Files         317      321       +4     
  Lines        9647     9739      +92     
  Branches     1034     1042       +8     
==========================================
+ Hits         7424     7514      +90     
- Misses       2071     2072       +1     
- Partials      152      153       +1     
Flag Coverage Δ
unittests 77.15% <94.80%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
social_core/backends/open_id_connect.py 91.60% <80.95%> (-0.88%) ⬇️
social_core/backends/oauth.py 85.97% <100.00%> (+0.42%) ⬆️
social_core/backends/vault.py 100.00% <100.00%> (ø)
social_core/tests/backends/test_okta.py 97.61% <100.00%> (+0.32%) ⬆️
social_core/tests/backends/test_open_id_connect.py 99.09% <100.00%> (+0.15%) ⬆️
social_core/tests/backends/test_vault.py 100.00% <100.00%> (ø)
social_core/tests/strategy.py 93.84% <0.00%> (-1.54%) ⬇️
social_core/backends/saml.py 80.31% <0.00%> (ø)
social_core/backends/keycloak.py 100.00% <0.00%> (ø)
social_core/tests/backends/test_musicbrainz.py 100.00% <0.00%> (ø)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 36f25c4...f92ac0d. Read the comment docs.

@nijel nijel mentioned this pull request Mar 24, 2022
Copy link
Contributor

@davidhalter davidhalter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for trying to contribute @candlerb

I'm personally against this big list of changes that affect everything a lot of providers. I think we should go one step smaller and deal with individual problems as they arise. What is the issue for Vault here? What are you trying to solve?

The project is in maintenance mode and I feel like changes like this one are dangerous if there isn't a clear benefit to them. I won't merge for now and I would ask other maintainers not to merge as without discussing this first.

@candlerb
Copy link
Contributor Author

candlerb commented Mar 24, 2022

The specific problem is that you can't configure PSA to connect to an arbitrary standards-compliant OpenID Connect provider: you have to write Python code to subclass OpenIdConnectAuth every time. A self-hosted Vault is an example of such a provider, and #668 is an explicit backend for this. But it would be nice to be able to connect to any OIDC provider (and Vault then wouldn't need its own backend).

There are a few things such a generic backend needs to do:

  1. Have the appropriate configuration knobs to select (at minimum) the OIDC_ENDPOINT, KEY and SECRET
  2. Configuration knob to use HTTP Basic Auth instead of Form parameters for the client ID and secret
  3. If (2) not set explicitly then auto-detect from the .well-known/openid-configuration

The most flexible place to do (2) was in BaseOAuth2 as it could then be used by other subclasses, potentially simplifying them later.

However I take your point about not touching working code. I could create a new subclass of OpenIdConnectAuth, let's say GenericOpenIdConnectProvider, which overrides methods as necessary, and then I wouldn't have to touch any underlying layers. Would that be preferable?

@candlerb
Copy link
Contributor Author

candlerb commented Mar 24, 2022

Now I remember the other problem. In OpenIdConnectAuth, OIDC_ENDPOINT is a class variable, not a setting.

You can spoof it by turning it into a property (which is what I did in #668 to minimise other changes):

    @property
    def OIDC_ENDPOINT(self):
        return self.setting('OIDC_ENDPOINT')

This works, but it seems pretty ugly to me. It would probably be better just to have a fresh GenericOpenIdConnectProvider which doesn't inherit from OpenIdConnectAuth at all, even though that involves duplicating code.

The nearest existing backend I found with configurable endpoint URL is the Keycloak backend, which gets around this problem because it doesn't subclass from OpenIdConnectAuth, it subclasses BaseOAuth2. However that means it loses all the auto-discovery: you can't just point at Keycloak's base url and pick up .well-known/openid-configuration. Instead you have to configure a load of settings by hand (PUBLIC_KEY, AUTHORIZATION_URL, ACCESS_TOKEN_URL)

@davidhalter
Copy link
Contributor

I have personally had the same issue. I was able to work around that by using a function that generates those classes and writes them to globals(). It's really ugly, but in general it's still the right approach with the code we currently have. I originally also wanted to go with the approach you are taking, but ultimately it's probably the wrong approach for what we have here.

If you are just writing an OpenID Connect compatible software that only needs OIDC, I would probably recommend to not use this library, because it's not very well maintained. There are probably other libraries that only do OpenID Connect and are therefore more correct when it comes to the standard.

In general I'm just worried about touching the OIDC pieces here, because they are already not that well maintained and almost nobody around here understands OIDC on a good level. I certainly do not, because it's just hard to understand and I'm kind of maintaining that part :-/.

@candlerb
Copy link
Contributor Author

If you are just writing an OpenID Connect compatible software that only needs OIDC, I would probably recommend to not use this library, because it's not very well maintained. There are probably other libraries that only do OpenID Connect and are therefore more correct when it comes to the standard.

It's the other way round: I wanted to take an application which already has python-social-auth integration (Netbox), and connect it to an OIDC provider.

Netbox does offer an alternative way of doing this, via remote authentication with HTTP headers, so I can instead use an Apache reverse proxy with mod_auth_openidc. I just thought that in fixing this use case for Netbox, it would be nice to make it available to other consumers of python-social-auth.

@nijel
Copy link
Member

nijel commented Apr 30, 2022

I don't see anything breaking in this (though my OpenID Connect experience is very limited). I think having generic OpenID Connect provider is good for reasons described by @candlerb - we use python-social-auth at Weblate and users do not have the choice for the authentication library (I agree that there might be better choices out there these days), but they want to have the choice in the authentication service. Therefore, I'd go and merge this.

It would be great, if @candlerb could prepare matching docs pull request in https://github.com/python-social-auth/social-docs/

@davidhalter
Copy link
Contributor

Since I feel very much responsible for OpenID Connect stuff, please do not merge (otherwise I might have to revert). I will try to review some things again next week. My feeling still is that this is a bad approach (and probably breaks backwards compatibility), but I do not have the time to explain right now.

@nijel
Copy link
Member

nijel commented May 11, 2022

I've just gone again through this patch and I don't see anything breaking compatibility there. I think approach in having generic OpenID Connect backend is desired. For example IdentityServer could be integrated this way also.

@davidhalter
Copy link
Contributor

I don't see how this is generic. If I wanted a generic OpenID Connect backend, I would want multiple ones. This is pretty common actually. The current approach is already generic, you just have to instantiate a class.

I still have to review the code better, though.

@nijel
Copy link
Member

nijel commented May 13, 2022

Yes, it's limited to a single OpenID Connect provider, but this is really what I'm looking for. Our users typically have internal authentication service and want to use that. Being able to specify that only in configuration without implementing a subclass for every service would be great.

@davidhalter
Copy link
Contributor

Yes, it's limited to a single OpenID Connect provider, but this is really what I'm looking for. Our users typically have internal authentication service and want to use that.

I feel like the difference of inheriting from a class and setting a few attributes on it versus setting a few variables in a global namespace is minimal. Both of these ways can be documented, will be around the same line count and with inheritance it is even possible to use multiple OIDC backends if that is ever wanted.


@candlerb

I kind of understand your approach with managing variables better now. Is there a way you could provide me with a way where in the OIDC standard you were looking up the basic auth stuff? I feel like the big problem is that we currently do not support that and the OIDC standard is really hard to get right...

@candlerb
Copy link
Contributor Author

Is there a way you could provide me with a way where in the OIDC standard you were looking up the basic auth stuff? I feel like the big problem is that we currently do not support that and the OIDC standard is really hard to get right...

Sure. When you want to exchange a code for an ID token (at the Token endpoint), OIDC provides two ways of the client authenticating itself: via HTTP Basic Authentication, or via client_id and client_secret fields in the POST request body.

This is described in section 3.1.3.2 and section 9 of the OIDC Core 1.0 specification.

This in turn refers to section 2.3.1 of the OAuth 2.0 specification, RFC 6749. Here it is made explicit that the authorization server must implement HTTP Basic Auth, and may also implement the POST body fields (but the latter is not recommended).

By supporting only basic auth but not POST, Vault is compliant to the standard. However, prior to this patch, PSA's oauth.py (and hence also open_id_connect.py) implements only the POST option.

There are some subclasses of BaseOAuth2 which override some methods to do basic auth - see for example RedditOAuth2.

Copy link
Contributor

@davidhalter davidhalter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay again. I'm pretty busy and this is not easy to understand.

I feel like I understand the OpenID Connect part now, thanks for linking the relevant documents.

I feel like since @nijel wants the settings changes, this is also fine (although I personally prefer the subclassing stuff). Is there documentation needed @nijel ?

social_core/backends/open_id_connect.py Show resolved Hide resolved
@nijel
Copy link
Member

nijel commented Jun 2, 2022

Yes, documentation for new backends is definitely needed in https://github.com/python-social-auth/social-docs/

@candlerb
Copy link
Contributor Author

candlerb commented Jun 2, 2022

I made some docs a while ago here: python-social-auth/social-docs@master...candlerb:candlerb/oidc

I didn't make a PR as it was unclear whether these oidc changes were accepted, either in current form or differently.

* Modify OpenIdConnectAuth so it can be directly instantiated
  (i.e. as SOCIAL_AUTH_OIDC_xxx) for use with a generic
  OpenID provider

* Update BaseOAuth2 so that it can perform HTTP Basic Auth for
  completing the auth process, configured from OIDC discovery

* Fix tests for okta, which previously were not calling
  oidc_config() from OktaMixin
@davidhalter
Copy link
Contributor

To me this looks good now, thanks a lot. I think @nijel should approve all the documentation stuff, I have no idea about that.

@nijel
Copy link
Member

nijel commented Jun 7, 2022

I've made the PR for the docs here: python-social-auth/social-docs#109

@nijel nijel merged commit 8a3ec88 into python-social-auth:master Jun 10, 2022
@nijel
Copy link
Member

nijel commented Jun 10, 2022

Merged, thanks for your contribution!

@nijel nijel self-assigned this Jun 10, 2022
nijel added a commit to nijel/social-core that referenced this pull request Jun 28, 2022
It leads to crash with Django.

This reverts a tiny portion of python-social-auth#669.

Fixes python-social-auth#690
nijel added a commit that referenced this pull request Jun 28, 2022
It leads to crash with Django.

This reverts a tiny portion of #669.

Fixes #690
jbronn added a commit to radiant-maxar/openmaps-auth that referenced this pull request Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants