Fixes #26312 - fix default taxonomy for external users

[rabajaj0509]

This PR makes sure that the external users are assigned default taxonomy on creation.

@tbrisker can you please take a look at this one?

[theforeman-bot]

Issues: #26312

[tbrisker]

This changes the behavior in an unexpected and possibly dangerous way - new users will be auto created from an external auth source, even if the administrator doesn't want that to occur, and there is no way to disable this setting any more.
Also, I don't understand how exactly does this fix setting a taxonomy for external users?

[rabajaj0509]

I had a word with @ares also about the design of external users org/loc so adding him to the thread.

@tbrisker I am not sure if I understand this correct but I would like to describe my case here.

1. There are exactly 4 ways of authentication as per my knowledge: INTERNAL, EXTERNAL, HIDDEN, and LDAP, none of the others except EXTERNAL is affected by this change. So it is harmless to the rest of the auth flows.

2. In the seeds https://github.com/theforeman/foreman/blob/develop/db/seeds.d/030-auth_sources.rb#L10 we have mentioned that unless the parameter authorize_login_delegation_auth_source_user_autocreate is set, we do not assign org/loc to the external users. Therefore, with this change we can overcome that. During an upgrade or installation of the application, the migrations will run before the seeds and therefore an org/loc will be assigned to the external user.

3. New users, which are only external will get this assigned to themselves. If the administrator does not want this, he can go to the settings and make the field authorize_login_delegation_auth_source_user_autocreate as nil.

Let me know your thoughts.

[ares]

Note that EXTERNAL auth source can be used for other means of authentication, not just keycloak. E.g. if you configure kerberos, IPA, client cert authentication etc. EXTERNAL auth source means, we rely on Apache to set REMOTE_USER. Therefore this change affects other auth flows as @tbrisker mentions.

I understand what you're trying to achieve, but this rather seems as a workaround. It seems our REST for external auth source actually allows setting org/loc. Can that be a manual step then? It seems we don't have hammer command for that.

[rabajaj0509]

Note that EXTERNAL auth source can be used for other means of authentication, not just keycloak. E.g. if you configure kerberos, IPA, client cert authentication etc. EXTERNAL auth source means, we rely on Apache to set REMOTE_USER. Therefore this change affects other auth flows as @tbrisker mentions.

I am still missing something :( I am unable to understand how this would affect the other EXTERNAL auth source means. In any case the admin has to set this value to EXTERNAL. For Apache also, I checked, the admin has to set the field to External -> https://theforeman.org/manuals/1.23/index.html#5.7.5Populateusersandattributes

Also, looking at the code base, the setting authorize_login_delegation_auth_source_user_autocreate is used in the function self.find_or_create_external_user(attrs, auth_source_name) in user.rb wherein the setting is passed as the second parameter i.e auth_source_name, which has to be External for external users and should not be something like REMOTE_USER. The field has to be a AuthSource object https://github.com/theforeman/foreman/blob/develop/app/models/user.rb#L293

Let me know your thoughts on this!

I understand what you're trying to achieve, but this rather seems as a workaround. It seems our REST for external auth source actually allows setting org/loc. Can that be a manual step then? It seems we don't have hammer command for that.

@ares yes, makes sense. But I am still keen on this PR, we can add a hammer command for the admin though.

There seems to be few loopholes here, that I am unable to understand, @ares can you please elaborate?

[ares]

I am still missing something :( I am unable to understand how this would affect the other EXTERNAL auth source means. In any case the admin has to set this value to EXTERNAL. For Apache also, I checked, the admin has to set the field to External -> https://theforeman.org/manuals/1.23/index.html#5.7.5Populateusersandattributes

Also, looking at the code base, the setting authorize_login_delegation_auth_source_user_autocreate is used in the function self.find_or_create_external_user(attrs, auth_source_name) in user.rb wherein the setting is passed as the second parameter i.e auth_source_name, which has to be External for external users and should not be something like REMOTE_USER. The field has to be a AuthSource object https://github.com/theforeman/foreman/blob/develop/app/models/user.rb#L293

Let me know your thoughts on this!

This patch sets this setting value to External by default. Something users needed to do explicitly before. That means if the Apache is configured to set REMOTE_USER even mistakenly, it will be considered valid authentication now. I was refering to

foreman/app/services/sso/apache.rb

Line 39 in ce12cd9

return false unless User.find_or_create_external_user(attrs, Setting['authorize_login_delegation_auth_source_user_autocreate'])

. If auth source with such name is not found (today default) the authentication will fail. Now it will be set to External, which I think would then be found and the authentication can be performed. That is the impact of the change, unless I'm missing something.

The API part I mentioned was for setup, not for authentication itself. If I understood correctly, we need External auth source to exist and assign taxonomies to it. And that can be done through API today. The UI part would make it easier (but big to cherry-pick), the hammer is missing.

I'm not against this PR if it proves to have no effect to other auth methods. But especially if you want this to be cherry-picked, it shouldn't impact anything else.

[ekohl]

Can we even unset a value if there's a default? At least the help text doesn't match its function anymore since keep implies it's the default.

[rabajaj0509]

@ekohl thanks for the correction in the message :) and yes, you can unset the value!

[rabajaj0509]

@ares I am not sure about whether or not this PR brings any drawback. If the user sets REMOTE_USER mistakenly and he does not provide the correct credentials, then he/she will as it is fail to login. I think, there are enough number of checks in each SSO method to validate the users coming from different sources.

Although, I would like to bring your attention to this Bugzilla, https://bugzilla.redhat.com/show_bug.cgi?id=1599908. So right now, if a user logs in as an external user using any of the methods like kerberos, etc, then the user does not get assigned to any org/loc.

I think this should go in the core but if you still feel the slightest hesitation towards this PR then I will be happy to do the hammer part :) Let me know your thoughts.

[ares]

I think the linked BZ is the same cause. Correct me if I'm wrong, but this PR basically causes External auth source be present by default and have default taxonomies assigned. That seems reasonable, but the original concern is, we will now auto create and login users if the web server sets REMOTE_USER to something. We fully trust the value (hence external authentication). If users misconfigured Apache, e.g. having client cert authentication enabled for the whole vhost (we do this by default btw), it can start creating users which is probably unexpected. You only need to have yubikey inserted to your drive and have it's CA trustted.

I will let @tbrisker to clarify his concern. But I'm proposing a workaround for both the BZ you linked as well as keycloak - set taxonomies on external auth source manually. If we can find out remediation for unexpected change, I'm happy to have External auth sources created as assigned correctly on default instance.

[ekohl]

If users misconfigured Apache, e.g. having client cert authentication enabled for the whole vhost (we do this by default btw), it can start creating users which is probably unexpected. You only need to have yubikey inserted to your drive and have it's CA trustted.

AFAIK when you provide credentials but the REMOTE_USER header as a user it's also trusted since from the app we can't distinguish between headers set by Apache and by the user. That would mean by default we create a security hole that makes our entire authentication useless. However, I have not verified this.

[ares]

Is there a way user can set REMOTE_USER env variable from headers? AFAIK Apache will prevent this by prefixing user headers with HTTP_. If you spoof REMOTE_USER you'll find the value under HTTP_REMOTE_USER.

[rabajaj0509]

Sorry I don't have much insight on this 🤦‍♂️ but I still think checks would not let users to just login if External is set to the authorize_login_delegation_auth_source_user_autocreate setting.

[rabajaj0509]

@ares although as suggested I am adding a hammer command to assign default taxonomies for external users.

[ares]

My concern is dismissed by the fact we have authorized_login_delegation setting which still defaults to no. Let's if that also answers @tbrisker concern.

[ekohl]

After some reading it appears that REMOTE_USER should be an environment variable, not a HTTP header which translates in REMOTE_USER vs HTTP_REMOTE_USER. However, when we move to a reverse proxy and Puma we don't have environment variables anymore so that will be a problem then.

[tbrisker]

Let me clarify my main concern here:
Currently, my understanding is that if you have external auth source (such as apache sso or keycloak) configured, if this setting is nil, that means you have to manually create a user in foreman before that user can log in using the external auth source. This is useful if you only want to give specific users access to foreman by using their external login.
if you change this to setting to External, that means that every user who can sign in to the external auth source also gets access to Foreman immediately, since their user gets auto-created when they log in. This is usually not what you would expect to happen unless you manually set this setting. In fact, changing this behaviour would cause a significant security issue for those who did not want to grant their entire organization access to Foreman.
When you remove the setting from the BLANK_ATTRS list, that means that users can no longer set it to empty, meaning that they can't even disable this behaviour.

The other concern I have that you are relying on changing a runtime setting (which should be changed at runtime) to alter the behaviour of the seeding. The creation of the external auth source in the seed is done to make sure that if this setting has been set by the user, the AuthSourceExternal entry does exist in the database with the correct name, so that it would not fail when an unknown user tries to log in. I am not sure of why this was added, but the fact that it causes taxonomies to be assigned to the auth source is a side effect, not the goal of the seed.
The reason that occurs is that during the initial taxonomy seeding, the default taxonomies get assigned to all existing taxable objects - and since the auth source seed runs earlier, the external auth source is assigned to them. This means that the external auth source would only be assigned to the default taxonomies on a new install, but on existing installations/upgrades it would still remain unassigned.

So in short - I agree with @ares that it is better to work around it by setting the taxonomy explicitly via API rather than modifying this setting to get the side affect. Long term the correct solution is to allow properly managing the auth source from the UI like we do for LDAP.

[rabajaj0509]

Let me clarify my main concern here:
Currently, my understanding is that if you have external auth source (such as apache sso or keycloak) configured, if this setting is nil, that means you have to manually create a user in foreman before that user can log in using the external auth source. This is useful if you only want to give specific users access to foreman by using their external login.

AFAIK, you can not have an external auth source without the setting being set to External.

if you change this to setting to External, that means that every user who can sign in to the external auth source also gets access to Foreman immediately, since their user gets auto-created when they log in. This is usually not what you would expect to happen unless you manually set this setting. In fact, changing this behaviour would cause a significant security issue for those who did not want to grant their entire organization access to Foreman.

If the setting is set to External by default, it will only mean that users can login via the external auth. There are checks and other settings that will not allow a user to just login without the correct source and intent provided. Getting access to Foreman immediately would mean that the user would be able to access only the resources that he/she is allowed to.

When you remove the setting from the BLANK_ATTRS list, that means that users can no longer set it to empty, meaning that they can't even disable this behaviour.

I am not sure about the internals, but by lookign at it, I am being able to unset the attribute to nil.

The other concern I have that you are relying on changing a runtime setting (which should be changed at runtime) to alter the behaviour of the seeding. The creation of the external auth source in the seed is done to make sure that if this setting has been set by the user, the AuthSourceExternal entry does exist in the database with the correct name, so that it would not fail when an unknown user tries to log in. I am not sure of why this was added, but the fact that it causes taxonomies to be assigned to the auth source is a side effect, not the goal of the seed.

Looking at the seed this line https://github.com/theforeman/foreman/blob/develop/db/seeds.d/030-auth_sources.rb#L11 is never being executed at all. Not at the time of installation and also not at the time of upgrade.

external_name.present? && AuthSourceExternal.find_by_name(external_name).nil?

This condition imo does not run in any case. When there is a new installation the external_name.present? is false. If the setting is set to External and if an external user is created then AuthSourceExternal.find_by_name(external_name).nil? will be false. In both the conditions, it will be false.

The reason that occurs is that during the initial taxonomy seeding, the default taxonomies get assigned to all existing taxable objects - and since the auth source seed runs earlier, the external auth source is assigned to them. This means that the external auth source would only be assigned to the default taxonomies on a new install, but on existing installations/upgrades it would still remain unassigned.

So in short - I agree with @ares that it is better to work around it by setting the taxonomy explicitly via API rather than modifying this setting to get the side affect. Long term the correct solution is to allow properly managing the auth source from the UI like we do for LDAP.

Right now we only have a :PUT request to update the Auth Source(theforeman/hammer-cli-foreman#461), but that would come into picture only when the AuthSourceExternal is present. To make that present we need to add an API endpoint that makes it possible to create an AuthSourceExternal. IMO, this should go into the core.

@ares @tbrisker let me know your thoughts. I might be wrong here, just trying to get an idea of what we need to do to get this done with :)

[tbrisker]

Let me clarify my main concern here:
Currently, my understanding is that if you have external auth source (such as apache sso or keycloak) configured, if this setting is nil, that means you have to manually create a user in foreman before that user can log in using the external auth source. This is useful if you only want to give specific users access to foreman by using their external login.

AFAIK, you can not have an external auth source without the setting being set to External.

That is incorrect. You can have this setting set to nil and Authorize login delegation set to true. That means users have to be manually created with external auth source set before they can log in to the system. This is most likely the more common use case in fact, as you only want to grant specific users access to foreman and not every user in your organization.

if you change this to setting to External, that means that every user who can sign in to the external auth source also gets access to Foreman immediately, since their user gets auto-created when they log in. This is usually not what you would expect to happen unless you manually set this setting. In fact, changing this behaviour would cause a significant security issue for those who did not want to grant their entire organization access to Foreman.

If the setting is set to External by default, it will only mean that users can login via the external auth. There are checks and other settings that will not allow a user to just login without the correct source and intent provided. Getting access to Foreman immediately would mean that the user would be able to access only the resources that he/she is allowed to.

This is again incorrect. This setting allows any user who is authenticated by external auth source to get a user in Foreman. Remember there are multiple implementations of external auth, not just keycloak.

When you remove the setting from the BLANK_ATTRS list, that means that users can no longer set it to empty, meaning that they can't even disable this behaviour.

I am not sure about the internals, but by lookign at it, I am being able to unset the attribute to nil.

The other concern I have that you are relying on changing a runtime setting (which should be changed at runtime) to alter the behaviour of the seeding. The creation of the external auth source in the seed is done to make sure that if this setting has been set by the user, the AuthSourceExternal entry does exist in the database with the correct name, so that it would not fail when an unknown user tries to log in. I am not sure of why this was added, but the fact that it causes taxonomies to be assigned to the auth source is a side effect, not the goal of the seed.

Looking at the seed this line https://github.com/theforeman/foreman/blob/develop/db/seeds.d/030-auth_sources.rb#L11 is never being executed at all. Not at the time of installation and also not at the time of upgrade.

external_name.present? && AuthSourceExternal.find_by_name(external_name).nil?

This condition imo does not run in any case. When there is a new installation the external_name.present? is false. If the setting is set to External and if an external user is created then AuthSourceExternal.find_by_name(external_name).nil? will be false. In both the conditions, it will be false.

This covers the (probably rare) condition where the setting has been set to "External" (or anything else - could also be e.g. "apache" or "myCoolAuthSource") but no user has been auto created by it until the seeds run again. That is the only case in which the auth source would indeed be created by the seed. In any case, relying on this setting to get the auth source created makes no sense to me.

The reason that occurs is that during the initial taxonomy seeding, the default taxonomies get assigned to all existing taxable objects - and since the auth source seed runs earlier, the external auth source is assigned to them. This means that the external auth source would only be assigned to the default taxonomies on a new install, but on existing installations/upgrades it would still remain unassigned.
So in short - I agree with @ares that it is better to work around it by setting the taxonomy explicitly via API rather than modifying this setting to get the side affect. Long term the correct solution is to allow properly managing the auth source from the UI like we do for LDAP.

Right now we only have a :PUT request to update the Auth Source(theforeman/hammer-cli-foreman#461), but that would come into picture only when the AuthSourceExternal is present. To make that present we need to add an API endpoint that makes it possible to create an AuthSourceExternal. IMO, this should go into the core.

@ares @tbrisker let me know your thoughts. I might be wrong here, just trying to get an idea of what we need to do to get this done with :)

I would be fine with seeding an External Auth source always, but this requires a bit more thought than just removing the condition in the seed. For example, what do we do if there is already an auth source with a different name? What if they had more than one created (i.e. used one name, created it and changed the name)?

[rabajaj0509]

@tbrisker @ares so would you suggest to create an API in the core to create the AuthSourceExternal object with taxonomy assigned to it till the time we have the UI in place?

[ares]

I'd say the update is enough if our long term plan is to seed it automatically. Note there should always be just one external auth source. Edit API is there already. So we need CLI and UI for the same.

[rabajaj0509]

I'd say the update is enough if our long term plan is to seed it automatically. Note there should always be just one external auth source. Edit API is there already. So we need CLI and UI for the same.

@ares The only issue I see with this approach is that when the Foreman is initially installed, at that time the admin will not be able to use the update API because at that point in time there is no AuthSourceExternal created. Once an external user is created only then the admin can make use of the update API.

One update: the PR that consumes the update API is merged in hammer.

[ares]

@ares The only issue I see with this approach is that when the Foreman is initially installed, at that time the admin will not be able to use the update API because at that point in time there is no AuthSourceExternal created. Once an external user is created only then the admin can make use of the update API.

What I'm trying to say is - let's seed the auth source by default. Let's also add UI and Hammer for updating taxonomies for existing external auth source that is created as part of the seed. But let's do that in 1.25/2.0 as this is potentially breaking. If we do both changes, create API is not necessary. Introducing that only for 1.24 does not feel very valuable if there's a way to do it manually and if we're improving that in next version.

[rabajaj0509]

@ares that looks like a reasonable solution to me. Let me try a work around form the Keycloak side in 1.24.

What I propose is: send the org/loc information in the JWT from Keycloak, once decoded, we can use that to assign it to the appropriate user!

Let me know your thoughts on this. @ares @tbrisker

[ares]

How would you like to do the mapping? Or how keycloak knows about Foreman's orgnization so that it can put it to the token?

[rabajaj0509]

@ares in Keycloak we have an Attributes field. If we pass the org/loc in the attributes field these attributes will be added to the JWT token. Once that is achieved, we can assign this org/loc to the user inside Foreman. The only drawback here would be that admin for Keycloak should know the org/loc for the usergroup that is created.

Few questions that I answered to myself:

1. Does the admin need to add User attributes for each user?
   ans: No, he can assign the user to a usergroup and we can pass the org/loc in group attributes.

2. How will the attributes look like?
   ans: Attributes is a key:pair value, the admin will add Organization as the key and an array for the value like ['default_organization', 'myorg']. After we decode the token we can use it.

Again this is in theory, I am not sure if this would work as explained but this is what I would propose.

This would be a time-consuming PR, for now, do we agree on seeding the external users by default?

[ekohl]

I have trouble following the entire discussion since I'm largely unfamiliar with external auth. I may be missing things.

I have a preference for only enabling external auth when the (web)server is actually configured to use that rather than enabling by default. In theforeman/puppet-foreman#779 I've started code to configure the system for keycloak. Since the installer is aware of the entire system and able to manage settings, we can do it there. It's using foreman-rake config under the hood so authentication shouldn't be an issue. As an alternative we can also set it in the YAML file which would mean the user can't change it in the UI anymore.

Does this help this discussion?

[rabajaj0509]

I have trouble following the entire discussion since I'm largely unfamiliar with external auth. I may be missing things.

I have a preference for only enabling external auth when the (web)server is actually configured to use that rather than enabling by default. In theforeman/puppet-foreman#779 I've started code to configure the system for keycloak. Since the installer is aware of the entire system and able to manage settings, we can do it there. It's using foreman-rake config under the hood so authentication shouldn't be an issue. As an alternative we can also set it in the YAML file which would mean the user can't change it in the UI anymore.

Does this help this discussion?

If I understand it correctly, @ekohl somehow when an admin enables SSO(via keyclaok) in installer, the setting would be set to External and then the seeds would run. Then it is a win-win situation. This way we dont need to make the enable the setting by default.

[ekohl]

the setting would be set to External and then the seeds would run.

At this point I think foreman::settings runs after seeding, but if we set it in the YAML file then it happens before seeding. The more I think about it, including it in the YAML file feels like the correct thing to do. Users shouldn't be able to change the setting via the UI if their webserver has been set up to do external auth anyway.

[tbrisker]

I have a preference for only enabling external auth when the (web)server is actually configured to use that rather than enabling by default.

There are two parts to this: one is creating the ExternalAuthSource object in the database, the other is actually enabling it (via authorize_login_delegation and authorize_login_delegation_api settings). We agreed (or at least it is my understanding) that the correct way to implement this is always seeding an external auth source and allowing users to edit it via ui/api/cli.
There is still a need to handle existing auth sources that users have created already, and consider if it's possible to set up more than one external auth source.
I'm not sure about the part of changing the setting(s) from the installer depending on the installer option. Keep in mind it is also used by other external auth methods, not just OIDC. I tend to prefer keeping them as settings the user needs to decide if they want to enable or not.

If I understand it correctly, @ekohl somehow when an admin enables SSO(via keyclaok) in installer, the setting would be set to External and then the seeds would run. Then it is a win-win situation. This way we dont need to make the enable the setting by default.

As was mentioned multiple times before, setting authorize_login_delegation_auth_source_user_autocreate to External and relying on the current seed is not the way to implement this.
This setting has it's own purpose, which is described in it's description: "Name of the external auth source where unknown externally authentication users (see authorize_login_delegation) should be created (keep unset to prevent the autocreation)". It has nothing to do with creation of the auth source or assignment of taxonomies to it, both of which are side effects of a portion of the seed that makes sure the setting is pointing to a valid auth source if it is set.

This would be a time-consuming PR, for now, do we agree on seeding the external users by default?

I don't think this is worth the effort and I don't think that we should rely on the external auth source to set location/organization in foreman. I don't expect the keycloak admin to need to know the foreman internals.

[ares]

FTR, fully agree with @tbrisker's reply

[ekohl]

I'm not sure about the part of changing the setting(s) from the installer depending on the installer option. Keep in mind it is also used by other external auth methods, not just OIDC. I tend to prefer keeping them as settings the user needs to decide if they want to enable or not.

If keycloak is set via the installer, it must be set to external. Otherwise the installer shouldn't set it and let the user change it. That's at least my understanding.

Edit: Just to be clear: right now I don't know how to proceed with the installer because it's not clear to me how & what settings to set. Should I wait with that until this PR has come to a resolution?

[tbrisker]

right now I don't know how to proceed with the installer because it's not clear to me how & what settings to set.

Perhaps as a first step the installer will only set up the apache mod needed for it to work and leave the settings for the user to manually configure themselves? or does the mod need also some settings for it to work?

[ekohl]

No, I can leave the settings out. I just don't know how to properly verify it works end to end.

[rabajaj0509]

Closing this PR as we decided to create a rake task for AuthSourceExternal and admins could then change org/loc from hammer command. PR for the rake task: https://github.com/theforeman/foreman/pull/7264/files