initial setup

[travisghansen]

Creating this issue to simply provide a forum for discussing the initial setup.

[zackpollard]

Hey there, love the sound of what you're working on, I'm currently using authelia for my setup however I'm interested in trying out keycloak but couldn't get it working before. If you need testers I'm definitely up for giving you a hand with that! Saw your posts over on issue #593 on the traefik github page :)

[travisghansen]

@zackpollard wasn't aware of that project so thanks for sending the link! I'd love to get more eyes on it so how can I help get you going?

[zackpollard]

Sorry, wasn't perfectly clear with what I said there. My current setup is Traefik using Authelia as my authentication provider. They just submitted a PR that makes this work nicely together, however Keycloak seemed like a more feature-rich auth provider and is what I tried to get working with Traefik in the first place, however had no luck. I'm interested in helping you test this with Traefik + Keycloak as I want to give keycloak a go to see how it compares with Authelia :)

[travisghansen]

@zackpollard so do you have what you need currently? Or do you need some assistance in how to configure keycloak etc?

[zackpollard]

I won't be able to get around to testing until Monday night/Tuesday as am away, however I can configure keycloak myself just need some guidance towards how this all interacts between traefik and keycloak. I believe in the other issue you mentioned you were going to produce some sort of documentation with example labels etc, that'd be a lot of help :)

[travisghansen]

@zackpollard ah ok! Are you running in kubernetes or something else?

[zackpollard]

Running in docker with compose setup for all my services.

[travisghansen]

@zackpollard ok, ultimately (after deploying the server somewhere) with traefik you should only need to set the forward auth url and auth response headers. There is an example of a static configuration doing this in the examples folder. My guess is that will equate to some simple labels in docker/composer.

[kevinoconnor7]

Hey @travisghansen!

Thank you for you work on this project! I've been messing around with it this weekend. The biggest hurdle I have had thus far is that the user is always redirected back to the same URL and the validation payload is passed through URL params.

This is problematic since many OpenIDC providers require callback URLs to be fixed and whitelisted. This unfortunately makes it largely unusable for me :(

[travisghansen]

@kevinoconnor7 hey thanks for the feedback. Actually my last commit before you even mentioned this I added support for that: https://github.com/travisghansen/oauth-external-auth-server/blob/master/bin/generate-config-token.js#L61

Try it out and let me know how it works out for you! I did my testing with gitlab (https://gitlab.com/gitlab-org/gitlab-ce/issues/48707) and confirmed it worked in my scenario.

[kevinoconnor7]

Oh very interesting! I played around with it a bit tonight and it seems to work! The only thing I'm not sure on is what I should host at that static url.

[travisghansen]

@kevinoconnor7 great to hear! It's not documented very well but essentially nothing is needed to be hosted at the URL. As long as the URL goes back to the target service and the target service is configured to externally authenticate the URL/path to oeas it should just work. It's one of the benefits of how it's oeas is programmed as it sort of acts like a seamless overlay on top the actual service. I'm currently using a magical GET parameter that let's the oeas service know it's the primary target of the request rather than the backing service.

In short, assuming you're authenticating a full domain service.example.com with external auth and that's what you've allowed in your provider is the same value service.example.com simply use that same value as the configuration parameter when generating the config_token.

If you don't mind my asking can you share what environment you're running in (which reverse proxy, kubernetes?, etc, etc)? Also, any feedback (good or bad) you can share for the next little while is greatly appreciated. If we find any deficiencies we'll see what we can address. Thanks!

[kevinoconnor7]

Got it! I'm using traefik as my reverse proxy so I just fired up a nginx container that just always returns 200. There's nothing too fancy here, it's just traefik + a few docker compose files.

So now the problem I have is that I got redirected back to ?__oeas_handler__=authorization_callback and then I get this displayed on the page/logs:

Error: error:0606506D:digital envelope routines:EVP_DecryptFinal_ex:wrong final block length
    at Object.decrypt (/app/src/utils.js:51:11)
    at app.get (/app/src/server.js:182:27)
    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
    at next (/app/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/app/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
    at /app/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/app/node_modules/express/lib/router/index.js:335:12)
    at next (/app/node_modules/express/lib/router/index.js:275:10)
    at cookieParser (/app/node_modules/cookie-parser/index.js:57:14)

This only shows up once though. If I go through the flow again (without clearing cookies), I just end up getting a 503 response from oeas with no error message in the response or logs.

BTW: can we make sure that errors don't get displayed on the page?

[travisghansen]

@kevinoconnor7 that's pretty strange, it seems to indicate the provider isn't sending the state back. Do you have the full request URL from the traefik logs? What happens is the 'relying party' (oeas in this case) sends the user agent to the provider with some state data that is supposed to be returned by the provider via GET parameter back to the redirect_uri.

Can you share who your provider is? If so can you send the discover url too?

[kevinoconnor7]

I'm using AWS Cognito.

Looking at the logs a bit closer, we do indeed have state returned as I see a log entry for:

parent request info: {
  uri: "<snip>",
  "parsedQuery": {
    ...
    state: "<snip>",
    ...
  },
  ...
}

What is interesting though is that that the state in the uri does not appear to be url encoded. So the character + is present in the uri but the decoded state turned those into spaces.

This only appears to be the case for the initial redirect back from AWS Cognito though. On subsequent tries the state in the uri appears to match the state in the parsedQuery.

The issue with the 503 response might be my fault. It looks like oeas returns that if the csrf check fails. I think this is do to me not redirecting back to the service directly. Basically I have a bunch of services that are *.mydomain.com and I redirect back to auth-proxy.mydomain.com. I think I can fix this simply by updating the config to set the cookie host to the top-level domain. I'll try that tonight.

[travisghansen]

@kevinoconnor7 ok, thanks for the patience trying to work through the quirks! I think what you've described is probably OK regarding the encoding but perhaps not. I need to have a little more info regarding your redirect approach though...I think that may be causing some issues.

Let's assume the following:

• You are running a target service at app1.mydomain.com
• You are running oeas at oeas.mydomain.com
• Your auth provider is configured to allow redirects to appN.mydomain.com (this is critical, it should not point to oeas.mydomain.com directly)

Assuming the above, if you want to cover several services with the same session (this is mostly untested but should work by the way...I'll test it out this week some time) you should:

• set the cookie.domain config option to mydomain.com
• set the redirect_uri config option to the appropriate appN.mydomain.com
• for each config_token you generate (basically 1 for each appN, make sure to use the same aud value in the config_token.

I'm pretty sure I can address the shortcomings of the above and make it so you can always use the same config_token for each appN service IF they all (each appN AND oeas.mydomain.com) are hosted under the same parent domain. I'll put some thought into it and see what can be done. The approach would allow setting your allowed redirects at the provider to oeas.mydomain.com instead of each individual appN needing to be listed. That does not work currently though, you must redirect back to the appN URL not to oeas directly at the moment, otherwise you'll see weird stuff.

[kevinoconnor7]

I was considering a slightly different setup:

• I have n services living at appN.mydomain.com
• oeas is running at say oeas.mydomain.com (in practice this doesn't exist, it's only accessible via an internal docker network.
• I have an nginx server that always returns an empty 200 response living at auth-redir.mydomain.com.
• AWS Cognito is living on auth,mydomain.com

Ideally I would want for any appN:

1. Traefik attempts the forward auth with redirect_uri set to auth-redir.mydomain.com.
2. oeas responds with a redirect to https://auth.mydomain.com/oauth2/authorize?redirect_uri=https://auth-redir.mydomain.com/?__oeas_handler__=authorization_callback&...
3. AWS Cognito kicks me back to the redirect_uri
4. oeas validates and kicks me back to appN.mydomain.com.

So far this all seems to work since the state payload that oeas decodes does have the correct redirect. AFAICT the only thing that's broken currently is the CORS check, but that should be solvable with setting the cookie host to mydomain.com.

Is there something else that I'm missing?

[kevinoconnor7]

Oh, with regard to the unencoded state payload, I think there is actually an issue there. I believe the state is meant to contain literal + characters, they should not be decoded to a space. I assume this is why the decrypt is failing.

[travisghansen]

@kevinoconnor7 the CORS cookie isn't using the cookie.domain setting currently. I'm still struggling to catch the full picture of your setup currently. Any chance you're on Freenode IRC or kubernetes slack channels? Might be easier to have a quicker feedback loop to get to the bottom of it and then put our findings here for others and better document things.

[kevinoconnor7]

I am not, unfortunately. I'm also about to leave for vacation so I won't be able to get back to this until next week.

--

Ah yes, I did overlook the CORS cookie... I was looking at the code for the wrong cookie.

I think the basic idea that I'm going for is that I want to minimize the configuration I need to do per service. Ideally I want to be be able to add any docker container labeled for traefik and everything should just work. At a high-level this should work as-is, but the limitation I hit is that AWS Cognito doesn't allow any wildcards in redirect_uri.

I think your intention for the redirect_uri config option was for it to be set per-service (ex. appN.mydomain.com). I think I want something that goes a step further that allows redirecting to a more common location like common.mydomain.com so that I don't have to reconfigure anything per-service.

[travisghansen]

@kevinoconnor7 yeah, I'm gonna punch at that idea for a bit. Pretty sure it's feasible (and I actually already had code for it at one point but ripped it out I think).

In essence you want:

1. The verify process to be a sub-request as it relates to oeas (external/forward auth) configured on each host/frontend
2. The callback process to directly (it may go through a reverse proxy, but it won't be a sub-request) access the oeas service.

In essence, you want to share the config_token across services, which I think is a valid use-case. There are a lot of assumptions in that context:

• all services use the same session (ie: if you authenticated to 1 you've authenticated to all)
• implied by the above is all services share the same cookie
• if all services share the same cookie you must be running all the services under the same parent domain (at some level)
• oeas must be accessible at or below that same parent domain
• the cookie.domain must be set as appropriate in the config/token (to the parent domain)

There are lots of nuanced scenarios in-between but does that align with your approach generally?

[kevinoconnor7]

That sounds correct to me.

[travisghansen]

@kevinoconnor7 ok, enjoy your vacation, I'll start covering the use-case for you to try when you get back :)

[travisghansen]

@kevinoconnor7 ok, I've just wrapped up support for your use case. Do the following:

• set the cookie.domain to the desired value (or leave blank)
• expose oeas somewhere the user-agent can access directly (ie: oeas.example.com)
• set redirect_uri to https://oeas.example.com/oauth/callback
• configure your provider to allow https://oeas.example.com/oauth/callback as a redirect_uri
• annotate traefik appropriately to set the forward auth url, etc (ie: https://oeas.example.com/oauth/verify?config_token=CONFIG_TOKEN_HERE)

That should do it. I'm using kubernetes so my setup is slightly different than your but it should all function the same. I've tested using a generic cookie.domain across several services and it worked perfectly.

Also note, I actually built it in such a way that the domain where oeas is directly accessible from is irrelevant (ie: you could use https://foo.bar.baz.example2.com) to the cookie.domain setting. The target service and oeas do NOT need to share a parent domain in other words. That should alleviate the need to keep adding more and more redirect_uris with the provider.

Try it out and let me know how it goes.

[travisghansen]

Just a heads up to those following along, I've decided that I want to expand the scope of the project to be a generic external authentication server. openid will be 1 plugin with ldap following shortly (nearly ready).

[kevinoconnor7]

Awesome! This update seems to be working for my use case.

The only remaining issue I have is that AWS Cognito is URI decoding the state when redirecting back to oeas which leads to decrypt errors. I'm fairly certain this is an issue on their end though, unless oeas is not properly transmitting the state in the first place.

[travisghansen]

@kevinoconnor7 wow great to hear! That sounds really odd that AWS is doing that. So when you get redirected it's an invalid URL? I'm guessing the other providers I'm using are working cause that state is critical to functioning setup..

[kevinoconnor7]

It's not invalid, just not properly encoded. So when oeas sends me to AWS with the state abc+def, it does so with ?state=abc%2Bdef.

When AWS redirects back to oeas it sets ?state=abc+def. This is problematic since oeas sees the state as abc def.

If I manually encode that component and refresh then it works.

[travisghansen]

@kevinoconnor7 ok, I'll have a look and see what I can find.

[travisghansen]

@kevinoconnor7 yeah, I think you've found a bug with AWS's handling of the state. I'm wondering if we can solve it via other measures. Right now I base64 encrypted data which subsequently get URL encoded. If I skip the base64 with something URL friendly that doesn't necessarily need URL encoding the issue would go away. I'm not sure if hex has any reserved URL characters but I'll play around with that for a bit..

[travisghansen]

@kevinoconnor7 it appears hex will work nicely. It's a slightly longer string but given the overall amount of data still relatively small. I've made the change but probably still a few days out since I have the code base ripped up right now to support multiple providers etc.

My next commit will include lots of new stuff. Support for openid and ldap (and possibly oauth2) and also allowing multiple plugins to used simultaneously (ie: check for basic auth and openid cookie with the same config token).

[kevinoconnor7]

Sounds good, I'll try to reach out to AWS to see if they'll confirm the bug on their end.

[travisghansen]

Soliciting a little feedback: I'm implementing a sort of pipeline of configured plugins (ie: test openid/oauth, basic auth, url param, etc). The order matters as the first one to succeed allows the request through. However, if all of them fail what feels more natural from a configuration standpoint, return the response for the first or the last plugin in the pipeline?

PS: I've gutted the code-base and got everything back in working order with openid specifically. I'm implementing a couple of crude plugins mostly for demonstration/testing purposes. After those are done I'll be tidying up ldap and looking into oauth2.

[travisghansen]

OK, I've landed a massive overhaul (pleaes re-read the docs, regenerate the tokens, update the foward auth endpoint to /verify?config_token=, etc).

The service now implements 6 authentication plugins including ldap, oidc (open id connect), ouath2, htpasswd (basic auth) and others.

Also, as mentioned in the previous comment the authentication process is now a pipeline of plugins. It's now possible to secure an endpoint with, for example, both ldap/basic auth and oauth2. It processes the plugins in a linear fashion and as soon as one responds 2XX the request is allowed. If all of them fail the default is to return the response from the last plugin in the config_token. That can be overriden on a service-by-service basis however by specifying the desired plugin in the forward auth URL.

Feedback welcome.

Beware lots of logging currently including the config tokens so beware of secrets logging. I'll get that cleaned up next.

[travisghansen]

@kevinoconnor7 state is hex now FYI.

[travisghansen]

OK, just landed support for custom assertions and infrastructure to support provider-specific userinfo with oauth2 plugins.

This does 2 broad things:

1. With oidc you can do custom assertions on any data in the id_token or the userinfo data.
2. With support for gathering provider specific userinfo on oauth2 it affords the same control.

So far I've only implemented userinfo for github in oauth2 which means you can for example do assertions based on teams or organizations or any other user data.

Proper logging is now in place and cleaned up. Secrets should not be getting logged now unless you've turned on debug level logging.

Server-side sessions expiration has been implemented for oauth2/oidc. This allows for expiring sessions to services which provide non-expiring tokens.

Userinfo expiration has been implemented as well (meaning after X period of time it is considered stale and must be re-retrieved). When used with userinfo assertions this becomes a powerful combination. You can ensure continued validation of users over a long period of time.

[travisghansen]

I've landed some better documentation and explanation of custom assertions now (still not great, but it's a start).

• https://github.com/travisghansen/external-auth-server/blob/master/ASSERTIONS.md
• https://github.com/travisghansen/external-auth-server/blob/master/PLUGINS.md
• https://github.com/travisghansen/external-auth-server/blob/master/HOWTO.md

I've implemented a jwt plugin as well which facilitates authenticating jwt Bearer tokens.

At this point I've knocked everything off the list I really care to for the 0.1.0 release.

[travisghansen]

@kevinoconnor7 any luck with AWS after the switch to hex state values?

@zackpollard did you ever give it a go?

[kevinoconnor7]

Sorry, haven't gotten a chance to try again. Hopefully I'll have some time this weekend. Thanks again for your help and active development. I did open a thread on the AWS forums about the issue, but the forum doesn't appear to have a lot of activity from the AWS team.

[travisghansen]

@kevinoconnor7 yeah ok, I'm hoping the issue will simply be gone now :D

[kevinoconnor7]

Wired it up this morning; the rename from oeas --> eas caused a few issues since I had to update a bunch of configurations but...

everything works! This is perfect! Thank you so much!

[travisghansen]

@kevinoconnor7 awesome to hear! I've got some more goodness coming shortly :) I'm adding a feature I call pcb (pipeline circuit breaker) which allows the auth pipeline to be broken using dynamic assertions.

For example you could force requests which have an Authorization header to not attempt the oauth plugin. Or sniff browser/user-agent strings and target specific plugins etc.

As part of it I'm going to upgrade the assertion stuff a little bit which may or may not need a new config to be generated if you're using assertions as an FYI.

Also I just committed a forward plugin yesterday which allows the service to reference other external auth services (essentially an external auth proxy plugin).

As a side note, I'd love if you could send over your config minus any secrets/etc so I can start collecting a series of examples with different providers.

Thanks again for the feedback!