Feature: Shibboleth/SAML SSO

[bradenmacdonald]

Description
This is the new Shibboleth/SAML SSO support integrated into third_party_auth (TPA). See the technical spec for a lot more background and discussion.

Dependencies

• edx/configuration#2002
• #8077 - Keyed ConfigurationModel Support
• onelogin/python-saml#66 (external upstream PR)
• onelogin/python-saml#67 (external upstream PR) - Rejected but compromise commit onelogin/python-saml@3a5847b will work for us and is now in python-saml master
• onelogin/python-saml#68 (external upstream PR)
• onelogin/python-saml#69 (external upstream PR)
• omab/python-social-auth#616 (external upstream PR)
• #8262 Fix for CRI-9 (not part of this feature branch)

Constituent PRs

• #8018 - Updates dependencies, adds SAML support
• #8155 Moves provider configuration to the database, implements metadata fetching
• #8518 - New tests
• #8591 Ability to specify default third_party_auth provider via query param
• #8603 UX changes: Ability to configure specific SSO providers as "secondary" providers
• #8599 Bump python-social-auth to upstream (trivial change, not required - just nice)
• #8600 - postponed

Sandbox

• http://sandbox5.opencraft.com/login
• http://sandbox5.opencraft.com/auth/saml/metadata.xml

Upgrade Issues
The following are actually caused by #8262 which was merged separately from this feature branch, but I'm leaving them here as a reminder:

1. When merged, this PR does cause a problem for existing installations: python-social-auth at one point converted from syncdb to south/django migrations, and bumping the version of python-social-auth that we use will result in this error next time paver update_db is run:

django.db.utils.DatabaseError: table "social_auth_usersocialauth" already exists

To fix this (on devstack), the user must manually run this command:

./manage.py lms migrate default 0001_initial --fake --settings=devstack

Unfortunately, I'm not aware of any way to avoid this issue other than this manual fix. New installs are not affected.
2. The new version of python-social-auth uses the Google+ API for obtaining user details. Any instance administrator with Google third party authentication enabled will need to go to https://console.developers.google.com/ and make sure that the "Google+ API" is enabled for their account.

[edx-webhook]

Thanks for the pull request, @bradenmacdonald! It looks like you're a member of a company that does contract work for edX. If you're doing this work as part of a paid contract with edX, you should talk to edX about who will review this pull request. If this work is not part of a paid contract with edX, then you should ensure that there is an OSPR issue to track this work in JIRA, so that we don't lose track of your pull request.

To automatically create an OSPR issue for this pull request, just visit this link: http://openedx-webhooks.herokuapp.com/github/process_pr?repo=edx%2Fedx-platform&number=8140

[bradenmacdonald]

@cpennington I need to make sure this feature branch makes the Cypress cut off. Will it need a separate review, or can we just merge it later this week (whether or not all the remaining subtasks listed above have been incorporated yet)?

CC @antoviaque

[cpennington]

I don't think it needs a separate review, given that all of the constituent PRs have had two thumbs.

[RobDolinMS]

@bradenmacdonald The latest version of Python Social Auth is required for this change to subsume the functionality in the old PR from Vin Bhalerao. Please make sure #8599 is included.

#8599 Bump python-social-auth to upstream (trivial change, not required - just nice)

[antoviaque]

@sarina Do you have any recommendations about the documentation part, for the changes to SSO? Will we add instructions in http://edx.readthedocs.org/projects/edx-installing-configuring-and-running/en/latest/ ? I'm not sure if we have any documentation currently about it, besides John Cox' blog post - @bradenmacdonald do you confirm?

If we do want to add a section to that page, how would it work? Should we submit something, or would someone from the documentation team write it?

[sarina]

@antoviaque that's a good question. My understanding is that @mhoeber is going to document this, but after it merges. You should sync up with him and see if there's a way your team can provide assistance.

[RobDolinMS]

@sarina, @antoviaque, @mhoeber My team and I would be interested in helping with docs here if help is needed.

[bradenmacdonald]

@mhoeber Please let us know how you'd like to go about getting this documented. The old instructions for setting up third_party_auth (e.g. Jon Cox's series of blog posts) will no longer work at all and the new approach, though simpler in many respects, is currently undocumented.

[mhoeber]

@bradenmacdonald - @lamagnifica will be working on this, mostly after 7/1. At a really high level we will need info on any config steps for Cypress release; and any information Cypress and Edg learners need to know.

We can use notes and links here, and anything you can contribute in a PR here would be great: https://github.com/edx/edx-documentation/tree/master/en_us/install_operations/source

[RobDolinMS]

@mhoeber Great! Colleagues and I will work on a PR at the link you provided; and looking forward to working with @lamagnifica after 7/1. @bradenmacdonald - Is it OK if we take some screenshots of your sandbox for illustrating the showing the UI?

[bradenmacdonald]

@RobDolinMS Yes feel free to take screenshots, but please use http://sandbox4.opencraft.com/login rather than sandbox 5, since it is more representative of the production UI. It's best to wait until this has merged, since the UI may change.

[bradenmacdonald]

Alright, I think this feature branch is ready to land! I've squashed it down to one commit per PR and rebased it.

I will merge it once the tests pass and I've manually verified the rebase didn't introduce any obvious regressions (there were no conflicts though).

[bradenmacdonald]

🎉

[RobDolinMS]

@bradenmacdonald This is great! BIG KUDOS on your good work!

[bradenmacdonald]

@fredsmith FYI, this PR has migrations. Configuration of third_party_auth is moved from django settings to ConfigurationModels. The migrations include a data migration that will copy any existing Google/Facebook third party provider settings to the new config table, so no special actions should be necessary.

[levalencia]

Hello Guys, can you please explain me how to configure openedx with office 365 single sign on?

I havent found any documentation about it.

[vinhub]

We have used these instructions for adding Office 365 SSO to Open edX. Hope they help you.

1. Follow these instructions to create an application in Azure AD for your edx installation: https://msdn.microsoft.com/en-us/office/office365/howto/add-common-consent-manually

   • For the sign-on URL and App ID URI, you can need to specify http://:8000/login
   • For the Reply URL, you can specify: http://:8000/auth/complete/azuread-oauth2/
   • In the "Permissions to Other Applications" section,
     • for Windows Azure Active Directory, select the "Sign in and read user profile" permission
     • for Sharepoint Online, select "Read user profiles" permission
     • Save changes and remember the client ID and secret

2. ssh into youe edx server

3. In lms.env.json, under "FEATURES", set "ENABLE_THIRD_PARTY_AUTH"="true"

4. In lms.env.json, add the following (this is the master list of all providers your site can support, you can decide whether to include the others in the list):

   "THIRD_PARTY_AUTH_BACKENDS": [
   "social.backends.google.GoogleOAuth2",
   "social.backends.linkedin.LinkedinOAuth2",
   "social.backends.facebook.FacebookOAuth2",
   "social.backends.twitter.TwitterOAuth",
   "third_party_auth.saml.SAMLAuthBackend",
   "social.backends.azuread.AzureADOAuth2"
   ],

5. Run db migration script (e.g. for devstack: "paver update_db --settings=devstack")

6. Start lms

7. Navigate to http://:8000/admin/ and login as admin

8. Navigate to Third_Party_Auth -> Provider Configuration (Oauth2)

9. Add Provider Configuration (OAuth2)

   • Name: Office 365
   • Backend: azuread-oauth2
   • Client ID: enter client ID from Azure AD portal
   • Client Secret: enter client secret from Azure AD portal
   • Other settings: { "RESOURCE": "https://-my.sharepoint.com" }
   • This will be the URL to the sharepoint site created for you by Office 365
   • icon: fa-sign-in (there is no font awesome icon for Office 365)
   • Enabled = true

10. You may select to skip registration form or email verification if desired (optional)

11. Save

12. Now, you can log out and then go to the login page and you will see the Office 365 login button. Clicking on it will start the OAuth2 SSO flow.

[levalencia]

sorry for the noob question, as I am not a linux expert, how do I start lms? also, which one is the default admin pass? I am using WIndows Azure Image with Ubuntu and Open EDX Cypress (from Bitnami)

[bradenmacdonald]

@levalencia Pull requests like this are meant to be for technical discussions of code, not support requests. Please join us on slack or one of the mailing lists to get support from the community.

To answer your question though: I would suggest you read https://openedx.atlassian.net/wiki/display/OpenOPS/Managing+OpenEdX+Tips+and+Tricks . If you find the production install on a remote VM is not convenient, you can also start with https://github.com/edx/edx-platform/wiki/Developing-on-the-edX-Developer-Stack which runs on your computer and has a lot more things that work "out of the box".