Decide how to Authenticate Users of the Turing BinderHub

[sgibson91]

Summary

Decide on the most appropriate method of authenticating users of the Turing BinderHub.

Currently have GitHub authentication working with the following config:

config:
  BinderHub:
    use_registry: true
    image_prefix: sgibson91/BHtest2-dev-
    hub_url: http://<redacted>
    auth_enabled: true

jupyterhub:
  cull:
    # don't cull authenticated users
    users: False

  hub:
    services:
      binder:
        oauth_redirect_uri: "http://<redacted>/oauth_callback"
        oauth_client_id: "binder-oauth-client-test"
    extraConfig:
      hub_extra: |
        c.JupyterHub.redirect_to_server = False

      binder: |
        from kubespawner import KubeSpawner

        class BinderSpawner(KubeSpawner):
          def start(self):
            if 'image' in self.user_options:
              # binder service sets the image spec via user options
              self.image = self.user_options['image']
            return super().start()
        c.JupyterHub.spawner_class = BinderSpawner

  singleuser:
    # to make notebook servers aware of hub
    cmd: jupyterhub-singleuser

  auth:
    type: github
    github:
      clientId: "<redacted>"
      clientSecret: "<redacted>"
      callbackUrl: "http://<redacted>/hub/oauth_callback"

Adapted from these docs:

• https://binderhub.readthedocs.io/en/latest/authentication.html
• https://zero-to-jupyterhub.readthedocs.io/en/latest/authentication.html#github

And with the addition of the following config, GitHub users would need to be a member of the Alan Turing Institute organisation on GitHub to gain access (not tested yet). Adapted from here: https://zero-to-jupyterhub.readthedocs.io/en/latest/authentication.html#giving-access-to-organizations-on-github

auth:
    type: github
    github:
      orgWhitelist:
        - "alan-turing-institute"
    scopes:
      - "read:user"

This would allow any member of the Turing GitHub organisation to access the BinderHub and launch any public repo.

However, there is also the option to authenticate using Active Directory, see the following link:

• https://zero-to-jupyterhub.readthedocs.io/en/latest/authentication.html#example-active-directory-configuration

What needs to be done?

• Pros/Cons of GitHub and Active Directory Authentication
• Have authentication work for HTTPS wed addresses
• Get OAuth with GitHub organisations working

Who can help?

• Anyone with knowledge of Authentication protocols, either GitHub or Azure specific

---

Updates

07/03/2019 - Successfully got organisational authentication for BinderHub working using auth: scopes: read:org. Some reading to convince ourselves this is only accessing organisation/team memberships is required. Docs here.

[martintoreilly]

I think the Github organisation member check is good enough for an initial MVP. It sounds like we stand a decent chance of getting this working for the Build a BinderHub workshop, which is not possible for the Azure Active Directory option.

I vote yes for Github membership check.

[martintoreilly]

If we get it working in time for the workshop, can we make creation of a GittHub organisation to use for membership control part of the demo? Not all attendees will have an existing Github org and it would be great to let people spin up all the required pieces from scratch to demo this "back at the ranch".

[martintoreilly]

I think both Github organisation and Active Directory authentication and membership restrictions are valuable contributions to make easy to set up, so let's not lose the one we don't pick for the MVP. Let's spin out an issue for both once we pick.

[sgibson91]

If we get it working in time for the workshop, can we make creation of a GittHub organisation to use for membership control part of the demo? Not all attendees will have an existing Github org and it would be great to let people spin up all the required pieces from scratch to demo this "back at the ranch".

I can absolutely try. But I have just broken it... 😞

[martintoreilly]

Anything I can help with?

[sgibson91]

Not sure yet. I didn't get the webpage redirection I was expecting after I added the orgWhitelist and purged the User Access Token on the GitHub OAuth app. So I tried generating a new clientId and clientSecret, still no luck. So I removed the orgWhitelist and couldn't get back to sign in with any GitHub account. I think I'm going to come back at it with fresh eyes and brain in the morning.

[sgibson91]

I got standard GitHub sign-in working again, I think I had an extra slash in my homepage URL when setting up the GitHub OAuth app. I get a nice little authentication page and then I get my own username on JupyterHub to work in.

[sgibson91]

I would like to get this working with HTTPS connection as well.

[sgibson91]

If we want to really control who we allow access the BinderHub, we could change auth: scopes: - "read:user" to auth: scopes: - "read:org" in config.yaml, which would mean users would have to request access from the owner of the GitHub organisation and they would verify over email. See zero-to-jupyterhub-k8s/issues/687.

This is potentially overkill, but just documenting that it's an option.

[sgibson91]

Updated config.yaml to require organisation login.

• @r-j-arnold who is not a member of the Turing GitHub organisation was not granted access. Received a 403 : Forbidden page
• @LouiseABowler who is a member of the Turing GitHub organisation was also not granted access.

Experiments:

• Tried changing orgWhitelist from "alan-turing-institute" to "The Alan Turing Institute": This caused a 500 : Internal Server Error for both myself and @LouiseABowler .
• Tried chainging orgWhitelist to "TheAlanTuringInstitute" - still receiving 500 : Internal Server Error
• Tried revoking all user tokens on the GitHub OAuth App so as to force myself through the authentication flow again (this does not generate a new client ID or secret) but I'm now taken straight to the binder page and not asked to sign in at all, and I'm definitely signed out of my GitHub account (very disconcerting -- clearing my browsing history fixed it)
• @jemrobinson also tested for orgWhitelist: - "alan-turing-institute", he was taken to the sign in page and then received 403 : Forbidden error

[sgibson91]

Useful commands for accessing the JupyterHub logs:

• kubectl get pod -n <namespace>: List the pods, find the one beginning hub-
• kubectl logs hub-<random-stuff> -n <namespace>: prints the logs to the terminal

Output of JupyterHub logs for a failed authentication using read:user scope:
This claims the user is not a member of the GitHub organisation when in fact they are.

[W 2019-03-07 15:43:16.919 JupyterHub github:149] User <redacted> is not in org whitelist
[W 2019-03-07 15:43:16.920 JupyterHub base:504] Failed login for unknown user
[W 2019-03-07 15:43:16.922 JupyterHub log:158] 403 GET /hub/oauth_callback?code=[secret]&state=[secret] (@10.244.0.1) 325.12ms

[sgibson91]

On the advice of the Binder team, tried switching read:user to read:org. Apparently, requesting access for the app to read users organisation membership is only required once.

People who accessed the binder page after the app was granted access by the alan-turing-instute organisation were me, @LouiseABowler and @KirstieJane. Whereas @r-j-arnold was denied access. So it seems that the filtering of membership is working as expected.

I think further exploration is required into exactly what permissions are being granted here. The approval email from alan-turing-institute reads:

An organization owner has approved the following application to access private data in the The Alan Turing Institute organization: . Because you are a member of the The Alan Turing Institute organization and have already granted this application access to your personal account, this app can now access The Alan Turing Institute organization resources on your behalf.

What does organization resources mean here?

The authorisation step from my end read:

Read only access. This application will be able to read your organization and team membership.

Documentation for GitHub OAuth scopes here.

[martintoreilly]

Doing sone digging on the Github users API, I feel we should be able to read a user's org memberships using https://api.github.com/users/<username>/orgs with the user:read scope. Let's experiment and find out.

[sgibson91]

Tried to write an access token to my GitHub user account using the following:

curl -H 'Content-Type: application/json' -H 'Accept: application/json' -X PUT -u sgibson91 https://api.github.com/authorizations/client/<redacted> -d '{"client_secret": "<redacted>", "scopes": ["read:user"]}'

Result:

{
  "message": "Not Found",
  "documentation_url": "https://developer.github.com/v3"
}

[manics]

These two discussions on GitHub org scopes might be helpful:

• Expected behavior with org_whitelist ? jupyterhub/zero-to-jupyterhub-k8s#687
• GitHubOAuthenticator: check org_whitelist using check-membership api jupyterhub/oauthenticator#227

[sgibson91]

Test ran with @nbarlowATI :
GitHub OAuth app owned by the whitelisted organisation (binderhub-test-org in this case). Scope "read:user" will work if people's membership to the GitHub organisation is public, but not if it's private (we'd need "read:org" scope). @nbarlowATI could log in with public organisation visibility, I could not with private.

What are the downsides of asking users who wish to use the BinderHub to have their membership be public? What reasons would they not want their membership to be public? (I feel like there are good ones, they're just not coming to me.)

The token, once granted to the user, remains so long as neither the user revokes the access of the app and the owner of the app doesn't revoke all user access tokens (i.e. forcing the authentication flow again). So a user would only need to have their membership be public the first time they log on to the BinderHub and could be reverted afterwards.

[sgibson91]

This suggests that using "read:user" should work for private organisation visibility providing the app is owned by the same organisation, but the test above disproves this.

Update: This only means that the organisation owner doesn't have to approve access of the app, "read:org" is still required.

[manics]

What are the downsides of asking users who wish to use the BinderHub to have their membership be public?

I did this with a publicly accessible jupyterhub, the biggest problem was ensuring people could find the relevant setting to switch to public, it's not very intuitive: https://help.github.com/en/articles/publicizing-or-hiding-organization-membership

[sgibson91]

I did this with a publicly accessible jupyterhub, the biggest problem was ensuring people could find the relevant setting to switch to public, it's not very intuitive: https://help.github.com/en/articles/publicizing-or-hiding-organization-membership

Yes, it is awkward to find. I'm just struggling to find a reasonable middle-ground between "read:user" not working for private memberships and the private/Third party access requirements for "read:org".

[KirstieJane]

(I haven't read this whole issue, I'm just swinging by in the middle of a slack meeting to say that I think having public membership is a fine price to pay for accessing Hub23!)

[sgibson91]

This is interesting jupyterhub/zero-to-jupyterhub-k8s#886

https://github.com/InfuseAI/primehub

[sgibson91]

See #410 Solved.

[sgibson91]

Plan is to transform the current Turing BinderHub into a JupyterHub, since that is probably more useful to a research community than a BinderHub, and it will use Azure Active Directory authentication so auth will be provided via a turing.ac.uk account, rather than relying on the GitHub org membership model which has dodgy permission scopes.

• Moving from BinderHub to Zero-to-JupyterHub alan-turing-institute/hub23-deploy#426