User/Group Management Without SCIM

[allquixotic]

Leela & AWS Team,

While attempting to roll out AWS SSO, my organization is encountering a number of AWS SSO challenges that exist outside the current feature-set boundary of SSO Extensions for Enterprise.

The original Problem 2 from this issue has been moved to #46

This issue is about User/Group Management Without SCIM.

Assumptions:

Note: Not all of these assumptions must be true for a potential solution to have value.

• Using External IdP (e.g. Okta) with AWS SSO.
• The external IdP does not support SCIM, or SCIM is gated behind a license that your organization does not have.
• Entering users and groups by hand in the AWS Management Console is too much administrative overhead, due to the sheer number of users/groups.
• Okta's backend is an on-prem Active Directory (i.e., AD is the system of record for user accounts).
• AWS has a Managed AD instance that receives a one-way user/group sync from on-prem AD using Microsoft Identity Manager.

Problem:

• Without another solution, we would have to either (1) purchase a license supporting SCIM on the External IdP side, or (2) manually duplicate all user CRUD operations (new user, delete user, update user, change user group assignments) both on the AD side, and on the AWS SSO side.
• SSO is supposed to reduce maintenance burden, or at the very worst, keep it the same - not increase it!
• Purchasing can be a challenging prospect for some organizations, even if the money is technically available. Okta may not even be the only IdP that requires additional purchases for SCIM.

Ad-Hoc Automation Solution:

• Developing a Python script (soon to be a cron schedule-driven Lambda)
• Script reads users from Managed AD, extracting email, name fields, username, and memberOf (the multi-valued list of every group the user is a member of) and stores the user data in an in-memory data structure.
• Script issues REST calls to the AWS SSO SCIM endpoint, at a minimum creating non-existent users/groups, but (ideally) also updating out of date information.
  • Example: User legally changes their name; joins or leaves AD groups.
  • Most common calls are GET and POST to /scim/v2/Users, /scim/v2/Groups - sometimes with filters. Code is fairly formulaic and intuitive if one understands the AWS SSO SCIM limitations as documented.

Possible Variations:

• Getting the user list from a different source, e.g. from Okta API natively, or some other IdP or database.
• Perhaps some organizations might only want to implement a subset of the users in AD, not all. The import process should provide a hook to allow arbitrary code to run and accept/reject a potential user import, or even include whitelist/blacklist functionality.

What Could SSO Extensions Do:

• Implement some kind of extensible variation of my ad-hoc automation solution idea.
• Take care of the AWS SSO SCIM operations (CRUD) and only require the user to provide a "data source" (where are you getting the users from?). Could bundle a few common data sources.
• Leverage your excellent CodePipeline infrastructure to auto maintain the new functionalities.
• Optionally support configuring the solution at runtime in S3 or via API.

---

Are these possible future directions for SSO Extensions? (Would you accept pull requests if I ever figure out the internals of SSO-Ex enough to directly implement this using SSO-Ex's existing infrastructure? Would you consider / are you already working on this yourself?)

Thanks for reading...

[leelalagudu]

Hi @allquixotic ,

Many thanks for bringing these 2 issues to our attention. And, a special thanks for the very thorough rationale setting and the solution approach you've described here. This helps our team with exactly understanding the background for the ask.

After reading through both the problem statements, here's my first impressions. I/team will update here with other feedback as we discuss this internally next week.

Could I ask you to break down both the problems into 2 issues please? Given the scope of these issues, it would be more helpful to tackle them as individual issues. With this assumption, I am providing the updates here for the first problem statement.

Also, to answer your last question - no, we did not think of these pain points until now , thank you for bringing this to our attention. Yes, we always welcome collaboration from the community and accept PR's.

Problem Statement 1: User/Group management without SCIM

• The business context, feasibility and the pain points described in this problem statement are clearly understood, and I agree that bridging this gap would be a very useful feature of the solution.

• I am trying to break down this into logical components , and the model I could come up with is as follows:

1. Fast-Fed based SCIM middle-ware

• perform all SSO user/group provisioning operations as SCIM operations supported by SSO SCIM API
• implemented through API gateway + lambda (and have an S3 path as mirror alternative)
• generic for any identity source customer has
• needs updates when SSO SCIM API are updated

2. Connector modules for a beginner set of non SCIM identity sources

• These would be fully packaged solutions that work for a non SCIM identity source and aim to handle user/group changes dynamically if feasible, and fall back to bulk scheduled mode otherwise
• The connector modules look up data from the non SCIM identity source and transform this into a format acceptable by SCIM middle-ware
• Managed AD would be a good first candidate to write a connector module

3. Utilities

• These would include functional components that do schema validation, identity source integration, batching, parallelisation, logging, error handling

• From an end user perspective, the flow would be as follows:

1. In the configuration file, they would set provisionSCIM to true
2. They would also set identitySource to one of the supported connectors we provide. for ex, ManagedAD
3. Depending on value in (2) , they would then fill in details specific to the identity source. This would include mode of import from identity source as well i.e. dynamic/bulk scheduled. If the value in (2) is not a supported source, then the user should mandatorily specify what interface they would choose for SCIM middle ware - API/S3.
4. They would then deploy the pipeline, which would either deploy the connector and start importing users/groups based on the mode the connector supports and the user choose in their config file (or) provide the user with S3/API interface details for SCIM middle ware. If the user choose a non supported connector, then they could script a mechanism that matches their identity source with the target set of that script being - make an API call / upload an object to S3.

What are your first thoughts on this?

Thanks,
Leela

[allquixotic]

Separated out the second problem to #46.

[allquixotic]

My comments are inline. Thank you for your hard work on this.

Hi @allquixotic ,

Many thanks for bringing these 2 issues to our attention. And, a special thanks for the very thorough rationale setting and the solution approach you've described here. This helps our team with exactly understanding the background for the ask.

After reading through both the problem statements, here's my first impressions. I/team will update here with other feedback as we discuss this internally next week.

Could I ask you to break down both the problems into 2 issues please? Given the scope of these issues, it would be more helpful to tackle them as individual issues. With this assumption, I am providing the updates here for the first problem statement.

Also, to answer your last question - no, we did not think of these pain points until now , thank you for bringing this to our attention. Yes, we always welcome collaboration from the community and accept PR's.

Problem Statement 1: User/Group management without SCIM

* The business context, feasibility and the pain points described in this problem statement are clearly understood, and I agree that bridging this gap would be a very useful feature of the solution.

* I am trying to break down this into logical components , and the model I could come up with is as follows:


1. [Fast-Fed](https://openid.net/specs/fastfed-scim-1_0-02.html#rfc.section.4) based SCIM middle-ware


* perform all SSO user/group provisioning operations as SCIM operations supported by [SSO SCIM API](https://docs.aws.amazon.com/singlesignon/latest/developerguide/supported-apis.html)

* implemented through API gateway + lambda (and have an S3 path as mirror alternative)

* generic for any identity source customer has

* needs updates when [SSO SCIM API](https://docs.aws.amazon.com/singlesignon/latest/developerguide/supported-apis.html) are updated

Agreed. Providing S3 and API driven management of the users will make it a most flexible solution, just like the current design patterns for permission sets and links.

2. `Connector` modules for a beginner set of non SCIM identity sources


* These would be fully packaged solutions that work for `a non SCIM identity source` and aim to handle user/group changes dynamically if feasible, and fall back to bulk scheduled mode otherwise

* The `connector` modules look up data from the non SCIM identity source and transform this into a format acceptable by SCIM middle-ware

* Managed AD would be a good first candidate to write a `connector` module

This is excellent.

The only thing I would add: to avoid weird conflicts, if the user enables a pre-packaged connector, the setting for using S3 or API driven for user and group management should probably be ignored. When a connector is enabled, to avoid weird conflicts and unexpected behavior, the SCIM middleware should always be set up for API access.

A user could, at their option, both use a connector and custom API calls to the SCIM middleware, but I'm not sure if that would have any weird corner cases that make it hard to manage on the SCIM middleware backend.

I have a hard time rationalizing in my head how the S3 approach would work in conjunction with a connector. Maybe if you can see how to resolve that design ambiguity, you could allow S3 to be used at the same time as a connector.

For my organization's purposes, we would just use the Managed AD connector, and we would not add our own code to call the SCIM middleware API, nor would we need additional user management in S3. We would treat our Managed AD as the single source of truth for SSO user and group management.

3. Utilities


* These would include functional components that do schema validation, identity source integration, batching, parallelisation, logging, error handling

* From an end user perspective, the flow would be as follows:


1. In the configuration file, they would set `provisionSCIM` to `true`

This is good so far..

2. They would also set `identitySource` to one of the supported connectors we provide. for ex, `ManagedAD`

Or none (or omit identitySource entirely)? :) My organization would use ManagedAD, but a flexible design will help the most organizations use this.

3. Depending on value in (2) , they would then fill in details specific to the identity source. This would include mode of import from identity source as well i.e. dynamic/bulk scheduled. If the value in (2) is not a supported source, then the user should mandatorily specify what interface they would choose for SCIM middle ware - API/S3.

Ah, this solves some of my earlier concerns.

4. They would then deploy the pipeline, which would either deploy the `connector` and start importing users/groups based on the mode the connector supports and the user choose in their config file (or) provide the user with S3/API interface details for SCIM middle ware. If the user choose a non supported connector, then they could script a mechanism that matches their identity source with the target set of that script being - make an API call / upload an object to S3.

Excellent. To follow existing design patterns, we'd want to introduce another option flag for preserving existing users who were manually created in AWS SSO but may not exist in the identitySource. Something analogous to ImportCurrentSSOConfiguration, but applying to the identity source connector.

Update 01/29/2022 23:41 UTC

The desired behavior for this flag would be:

true --> If principal exists in AWS SSO before SSO Extensions is deployed, and they do not exist in the identitySource, then leave them there.

false --> If principal exists in AWS SSO before SSO Extensions is deployed, and they do not exist in the identitySource, then delete the principal from AWS SSO.

I do not think it makes sense to update (write to) identitySource with this solution.

What are your first thoughts on this?

Thanks, Leela

[allquixotic]

One implementation concern I have... This may not actually require modifications to this design spec...

Our OrgMain account does not currently have network level access to our Managed AD. The Managed AD sits in a member account on a private VPC. Currently we have our entire SSO Extensions solution sitting in OrgMain in the same region.

I think if you put the SCIM middleware and the identity source connector into the target account (target being a technical term used in the design of SSO Extensions), we'll be able to change our SSO Extensions deployment so that the target account is the same account as the Managed AD. This is probably a better design for us long term, anyway, and it would solve the networking problem without needing a TGW from the member account back to OrgMain.

[leelalagudu]

Hi @allquixotic ,

Regarding the account structure, while moving to the recommended multi account model is a good idea and should be done, this won't be a blocker however. I am planning to treat the account hosting Managed AD as a separate account and communicate through cross account/region SNS topics. This is the same design pattern implemented currently between OrgMain, SSO and target accounts.

Regarding the logical design and flow details, I am working with the rest of the team on a structure and will update here shortly.

Also, to set the correct timeline expectations, we are currently working on another feature we've prioritised for 2 of our customers that would enable them to do a region switch with their AWS SSO. This is a stand-alone component of the solution that a customer could use to automate moving their permission sets and account assignments over to the new region. In terms of our current development, we are aiming to get this out in 3.1.0 in the next 2 weeks, and will focus on this enhancement as the next priority.

Hope these timelines work for you.

Thanks,
Leela

[allquixotic]

The other feature request I logged recently is actually more urgent than this one. The good news about this issue is that I've already written code that solves the immediate need for my organization, and can continue to customize it to suit my needs. It isn't as elegant as a baked-in solution with SSO Extensions, but it's better than nothing.

[leelalagudu]

@fotinosk will update the solution design docs with the pattern he's building for SCIM support and link the docs here for reference.