Implement OIDC authentication

[s3lph]

This PR implements login to Semaphore via OpenID Connect.

Multiple OIDC providers can be configured in config.json:

{
  # ...
  "oidc_providers": {
    "mysso": {  # The ID of the provider, is used as a URL path component in the redirect URL
      "display_name": "Sign in with MySSO",  # Text on the additional login button
      "provider_url": "https://keycloak.example.org/realms/mysso",  # Root URL of the OIDC provider, expects /.well-known/openid-configuration below this URL
      "client_id": "556e8e0a-bba8-49e8-af80-eae6db863b23",
      "client_secret": "ad497288-34bf-4452-bff6-2c218992f906",
      # "redirect_url": "${web_host}/api/auth/oidc/${provider}/redirect",  # default value, the OIDC provider redirects back here
      # "scopes": ["openid", "profile", "email"],  # default value, OIDC scopes
      # "username_claim": "preferred_username",  # default value, id_token claim to use as the username
      # "name_claim": "preferred_username",  # default value, id_token claim to use as the display name
      # "email_claim": "email"  # default value, id_token claim to use as the email address
      # If the OIDC provider does not offer a /.well-known/openid-configuration, the endpoints can be
      # configured manually.  In this case, the "provider_url" must be omitted.
      # "endpoint": {
      #   "issuer": "https://keycloak.example.org/realms/mysso",
      #   "auth": "https://keycloak.example.org/realms/mysso/protocol/openid-connect/auth",
      #   "token": "https://keycloak.example.org/realms/mysso/protocol/openid-connect/token",
      #   "userinfo": "https://keycloak.example.org/realms/mysso/protocol/openid-connect/userinfo",
      #   "jwks": "https://keycloak.example.org/realms/mysso/protocol/openid-connect/certs",
      #   "algorithms": ["HS256", ...]
      # }
    }
  }
}

For each of the configured providers, an additional login button is added to the login page:

If a user clicks the second button, they are redirected via /api/auth/oidc/${provider}/login (I couldn't find a nice way to do this without redirecting through an API endpoint) to the OIDC provider. From there, they are redirected back to /api/auth/oidc/${provider}/redirect, which validates the response and performs the backchannel token exchange. On success, a user session is created and the user is redirected on to /, on failure the user is sent back to /auth/login.

Same as with LDAP authentication, an external user account is created if one does not exist yet.

I tested this against Keycloak 21 using the code authentication flow.

[s3lph]

See also #973

[mnestor]

I can't wait for this to get merged. I would say though that since Gorilla Toolkit is in archive mode you probably shouldn't use anything from it.

[s3lph]

I can't wait for this to get merged.

@mnestor neither can i 🙃

I would say though that since Gorilla Toolkit is in archive mode you probably shouldn't use anything from it.

I did not choose to use gorilla/mux. Its what Semaphore uses for request routing (this is my first contribution to Semaphore, I'm not a maintainer). I think you'd be best off opening a separate issue for finding an alternative to gorilla/mux.

[mnestor]

Doh! Missed that

[ragavsathish]

Am looking forward on this to to be merged !

[aaronnad]

+1 on getting this added for extra authentication mechanisms alongside LDAP

[fiftin]

@s3lph , @binaryfire sorry for so long delay. Currently I have time to work on the project. And I will review the MR ASAP.

[fiftin]

Hi @s3lph You code looks very clear, all Dredd test passed, thank you!

I have no experience with OpenID, anybody can help me to test this? :-D

[fiftin]

@s3lph Thanks for your work on this. Will there be a way of disabling email login if SSO is enabled?

I think we will do this after testing period of SSO auth.

[lafayetteduarte]

Hi @s3lph You code looks very clear, all Dredd test passed, thank you!

I have no experience with OpenID, anybody can help me to test this? :-D

Hi @fiftin .
I have experience with openid and I could help with the testing.

How would you like to conduct the tests with this feature?

I think it's relatively simple to write docs and dockerfile examples using the most common openid providers out there.

[fiftin]

Hi @lafayetteduarte

This prerelease includes the feature: https://github.com/ansible-semaphore/semaphore/releases/tag/v2.8.91

No documentation exists yet, only what @s3lph written.

Thank you!

[aaronnad]

Is there anyway to get this config in through docker environment variables?
I want to be able to test and deploy this from an end users perspective passing through docker environment configuration.

Can we get the environment variables config pr raised in the interim to edit the config,json?

[fiftin]

Hi @aaronnad
sorry for delay.

Currently it is not available for docker. I don't know how to pass so many parameters vis ENV vars.

[fiftin]

@s3lph cloud you write some documentation for the feature?

Docs for OpenID: https://github.com/semaphoreui/ansible-semaphore-docs/blob/main/administration-guide/openid.md

[fiftin]

Hi @lafayetteduarte and others. Anybody can write a doc how to setup Semaphore OpenID to work with Amazon OpenID: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html#manage-oidc-provider-console

or GitLab: https://docs.gitlab.com/ee/integration/openid_connect_provider.html

?

[fiftin]

Wow, I did this! I connected Semaphore to GitLab OpenID!

It is really easy, thank you @s3lph :)

I will write tutorial how to do this.

[lafayetteduarte]

Hi @fiftin , tomorrow I will take a look at this.
The Amazon link seems to refer to a wrapper around IAM .
Almost like a proxy capable to emit tokens with access to Aws resources.
As you've experienced with gitlab the setup should be easy enough.
I was thinking about writing examples of this using common identity providers such as Google, azure Ad , and other solutions like keycloak and identity server.

I've seen you are planning for roles in semaphore.
When it's done should be easy enough to map one or more claims from the openid token to semaphore roles . I think that could be useful.

[fiftin]

Hi @lafayetteduarte

I wrote a small tutorial how to setup OpenID with GitLab: https://www.ansible-semaphore.com/blog/openid-authentication/

if you write a tutorial, I could post it on the website, with a link to your profile. Thank you!

[lafayetteduarte]

Hi @fiftin , will do.
Im working on a set o examples using self hosted solutions for openid.
how would you guys feel about an example folder on the project with instructions and docker-compose files to run all the parts involved? ( semaphore, the openid server , the database, etc).

i have some ldap example on the works ive used to debug the behaviour where the space in the parameters breaks the json generation. need to work on seeding the ldap database and write a compose as complete example.

I forked the dev branch . As soon as its done i will pull it to my profile and send you a link so you can evaluate the idea.
Thanks

[fiftin]

@lafayetteduarte It would be cool!

[lafayetteduarte]

Hi @fiftin

Just to let you know I'm running late on this.
Stumbled on a wired network issue with the docker-compose that I'm debugging.
Also , Included an example on building a custom image to install python dependencies such as pywinrm for managing windows hosts and VMware dependencies for dynamic inventories I've seen on some issues.

Will get back to the debug as soon as possible.

Will send the fork link when I'm done

[lafayetteduarte]

@fiftin ,
Got it working last night.

fork with the examples folder, here if you want to review-it
https://github.com/lafayetteduarte/semaphore/tree/develop/examples

Working examples:

• OpenID with keycloack
• ldap auth with openldap

this weekend i will add more examples for google, microsoft, facebook , linkedin and whatnots.

Let me know if i'm missing something

Thank you

[aaronnad]

Hi,

I'm unable to get this working with Authentik,

Using the following configuration

        "oidc_providers": {
          "authentik": {
            "display_name": "Sign in with Authentik",
            "provider_url": "https://auth.my.tld/application/o/semaphore/",
            "client_id": "wQyFi6xHdtz0NFSWq1Rg4WKoOEYXdFUrTbzb2Lf4",
            "client_secret": "fzFhvq56LB9455fFowxSmgE8jDghkQcGdSOnk61AFJkfJOvRjCB5iGmM2o352hWhh3N4HZ3y9QTl0bwxhPYI4UQKO1LLApfd9QBotnUjmjZcKNyJG6YM3L9i64lSTHpY",
            "redirect_url": "https://semaphore.my.tld/api/auth/oidc/authentik/redirect"
                }
        },

I tried at first
"provider_url": "https://auth.my.tld/application/o/semaphore", Notice the missing trailing /
However this didn't work as authentik is expecting the /

time="2023-07-28T19:56:27Z" level=error msg="oidc: issuer did not match the issuer returned by provider, expected \"https://auth.my.tld/application/o/semaphore\" got \"https://auth.my.tld/application/o/semaphore/\""

So i then re-tried it with the trailing / and i was still getting a redirect to the login page and not the application.

So i then tried to manually specify everything in case this would work, but again, authentik expects the trailing / at the end of each url

	"oidc_providers": {
	  "authentik": {
	    "display_name": "Sign in with Authentik",
	    "client_id": "wQyFi6xHdtz0NFSWq1Rg4WKoOEYXdFUrTbzb2Lf4",
	    "client_secret": "fzFhvq56LB9455fFowxSmgE8jDghkQcGdSOnk61AFJkfJOvRjCB5iGmM2o352hWhh3N4HZ3y9QTl0bwxhPYI4UQKO1LLApfd9QBotnUjmjZcKNyJG6YM3L9i64lSTHpY",
	    "redirect_url": "https://semaphore.my.tld/api/auth/oidc/authentik/redirect",
	    "endpoint": {
	      "issuer": "https://auth.my.tld/application/o/semaphore/",
	      "auth": "https://auth.my.tld/application/o/authorize/",
	      "token": "https://auth.my.tld/application/o/token/",
	      "userinfo": "https://auth.my.tld/application/o/userinfo/",
	      "jwks": "https://auth.my.tld/application/o/semaphore/jwks/",
	      "algorithms": ["HS256", "ES256"]
		    }
		}
	},

Still when I click sign in with Authentik and sign in, i just get looped round and stay on the login screen but my user does get created in the database.

Before sign-in with Authentik

mysql> select * from user;
+----+---------------------+----------+------------+------------------------+--------------------------------------------------------------+-------+----------+-------+
| id | created             | username | name       | email                  | password                                                     | alert | external | admin |
+----+---------------------+----------+------------+------------------------+--------------------------------------------------------------+-------+----------+-------+
|  1 | 2023-07-28 20:31:57 | admin    | localadmin | appadmin@tld.net       | $2a$11$1g0rJYlsdX9REcaZ.1Vzh.yFbUMCCihBKciuJ/8sMCeysLevlvBhG |     0 |        0 |     1 |
+----+---------------------+----------+------------+------------------------+--------------------------------------------------------------+-------+----------+-------+
1 row in set (0.00 sec)

After sigin with Authentik

mysql> select * from user;
+----+---------------------+----------+------------+-------------------------+--------------------------------------------------------------+-------+----------+-------+
| id | created             | username | name       | email                   | password                                                     | alert | external | admin |
+----+---------------------+----------+------------+-------------------------+--------------------------------------------------------------+-------+----------+-------+
|  1 | 2023-07-28 20:31:57 | admin    | localadmin | appadmin@mytld.net      | $2a$11$1g0rJYlsdX9REcaZ.1Vzh.yFbUMCCihBKciuJ/8sMCeysLevlvBhG |     0 |        0 |     1 |
|  2 | 2023-07-28 20:43:31 | aaron    | aaron      | aaron@testemail.com     |                                                              |     0 |        1 |     0 |
+----+---------------------+----------+------------+-------------------------+--------------------------------------------------------------+-------+----------+-------+
2 rows in set (0.01 sec)

There are no log outputs on the docker container saying any errors were caused either - this is a completely fresh installation.

Edit: Debugging further, as soon as I added the oidc configuration, my local users are also no longer able to progress through to login - using the docker installation

[aaronnad]

So upon further testing, If I go direct to the docker container and not through a reverse-proxy, the local admin works just fine.

At present my semaphore installation is available at https://semaphore.my.tld. Trying to use that instead of the direct IP has a negative effect. Going direct to http://192.168.50.2:3001 then allows the login.

It would appear that this configuration does not work/play well with reverse proxies.

Edit: See screenshot below. It would appear that having the default security enabled for HTTP-Only Cookies on the VMWare NSX ALB (formerly AVI loadbalancers) does not work. Disabling this setting enables this to work fine.

[lafayetteduarte]

@aaronnad , is the http- only cookie related to the username / password flow ?
I've had some trouble with the cookie policy in my sso implementation when the new rule came out in chrome.
Does it happen without the openid configured ? I think I have a reference post that helped me to troubleshoot the issue back then . Will check my messy favourite folders and try to find it .

[aaronnad]

Hi @lafayetteduarte ,

The HTTP-Only cookie issue happens without the OpenID Configuration present. I just get a re-direct back to the /auth/login path once I’ve entered my credentials.

Some applications I’ve worked with have a similar issue, just took me a little while to remember it in this case for Semaphore.

Secure cookies can be enabled on my reverse proxy still which is great.