GitHub oauth

[runningman84]

Do you really need a full openid provider or would a GitHub oauth application also work?

[travisghansen]

You need openid currently but I intend to look into pure oauth shortly. Another option would be to run keycloak with GitHub as an identity provider.

I'm working on making the scope of the project a generic external auth service (I haven't committed the changes yet) but with the changes I've made supporting additional services will be much easier and plugin based (I've already implemented ldap in the new code). Given how closely related oauth and openid are I don't think it will require much to get it going.

[runningman84]

That sounds great. My usecase are just a bunch of home or dev clusters which should not have any stateful software like keycloak...

[travisghansen]

Ok, note that this project stores data in memory or redis so there's a stateful component to even this.

Also note, the authorization code flow (primary focus of this and similar projects) doesn't work so great with SPAs (single page apps) so depending on what you're running it may or may not be useful. Works great for traditional server-side apps though :)

[runningman84]

Yes but even if you clear the redis data or memory you will just loose the active sessions. In case of keycloak you will loose all users which are stored in postgres.

I just want to secure applications like Prometheus and other internal tools.

[travisghansen]

True. And even if the session goes away typically you would still be authenticated at the provider so it wouldn't require a full login per se.

Prometheus (or presumably grafana) is a SPA and I can try it out for ya to see how well it works. Basically this project would work fine for those use cases as well if you turn off some of the validity checks (ie: expired tokens) and enable cookie expiration.

I can give further explanation if you'd like.

[runningman84]

What especially would not work with SPA sites?

I think of these applications:

• Prometheus / Alertmanager
• Grafana
• Graylog
• Traefik Dashboard
• Kubernetes Dashboard
• Influxdb Admin
• Homeassistant
• Kibana
• Uchiwa
• Jenkins

[travisghansen]

Ok, just landed a massive commit which implements pure oauth2.

To answer your question, unless the token/session does not expire it's not great for SPA. In the case of github my testing appears to show the tokens do NOT have an expiration so you should be good generally.

If not, I've now implemented a pipeline process so you could service the request with Basic auth using htpasswd data if you wanted as well. That way any system-to-system tools could also authenticate/access the resources.

[travisghansen]

@runningman84 as an FYI I'm implementing some infrastructure to support fetching user info with provider specific configuration using the oauth2 plugin. When combined with token assertions (not implemented yet) you'll then be able to ensure the target service is only available to you personally and not everyone who can login to github.

[runningman84]

Additionaly it would be great to have support for GitHub organizations and corresponding teams.

[runningman84]

This is the corresponding oauth proxy config:
https://github.com/bitly/oauth2_proxy/blob/master/README.md#github-auth-provider

[travisghansen]

Done and done.

[travisghansen]

@runningman84 looking for a little feedback here if you think this will help your use-case. Basically my approach with oauth2 is to implement provider specific userinfo plugins (that's what oidc calls it). In the case of github it looks like this in the config token:

...
features: {
  ...
  userinfo: {
    provider: "github",
    config: {
      fetch_teams: true,
      fetch_organizations: true,
      fetch_emails: true
    }
  }
}

Subsequently, in the assertions block you would put something like this:

userinfo: [
  {
    path: "$.login",
    //path: "$.emails[*].email",
    rule: {
      method: "eq",
      value: "travisghansen",

      //method: "regex",
      //value: "/^travis/",// "/pattern/[flags]"

      //method: "in",
      //value: ["travisgh", "travisghansen"],
      //value: ["travisgh", "travisghanse"],

      //method: "contains",
      //value: "travisghansen@yahoo.com2",

      //negate: true,
      //case_insensitive: true
    }
  }
]

Does this make sense? I'm basically taking the approach of let you assert on whatever field(s) you want with various methods (so far eq, regex, in, and contains).

Does it fulfill the need?

[runningman84]

That sounds good. Can you already give me a full example config how to use traefik with your auth provider and GitHub integration?

[travisghansen]

@runningman84 yeah of course. Can you share the assertion(s) you like to use on the user/orgs/teams? I'll build those in.

[runningman84]

I have these use cases:

• single user (username eq runningman84)
• org everyone (orgname eq arvatoaws)
• org specific team (orgname eq arvatoaws and team eq devops)

[travisghansen]

@runningman84 got it. Do you want the sessions to ever expire? Independent of that, how frequently would you like the userinfo refreshed to ensure continued validity to the assertions?

[runningman84]

In a perfect world both values can be configured. Session expire never and refresh one hour sounds like good defaults.

[runningman84]

Btw. maybe sometimes you want to allow multiple teams or multiple individual users.

[travisghansen]

@runningman84 both are configurable yes. And I do have queued up assertion methods of contains-any and contains-all.

Multiple individual users is already supported. Depending on what you mean by multiple teams it should be covered by one of the above.

[travisghansen]

@runningman84 are you using kubernetes or some other environment? Just getting ready to draft up a doc..

[runningman84]

I am using kubernetes

[travisghansen]

@runningman84 OK, I've written a howto using github as the example. It should be right in the direction you need. I omitted including custom assertions in the example config_token but they have been documented in the link below. I can help you craft those to the various needs you described above but let's start with basic functionality first and get that up and going. It's very easy to tweak later.

Thanks for the feedback and willingness to try it out!

• https://github.com/travisghansen/external-auth-server/blob/master/HOWTO.md
• https://github.com/travisghansen/external-auth-server/blob/master/ASSERTIONS.md

[runningman84]

That sounds great I will test that next week and provide you with feedback

[runningman84]

@travisghansen the auth server seems to work. But I do not really understand how to configure the assertions.

Your default config looks like

        assertions: {
          /**
           * assert the token(s) has not expired
           */
          exp: true

Where do I store the custom config as described here?
https://github.com/travisghansen/external-auth-server/blob/master/ASSERTIONS.md

exp: true is not really documented...

Maybe you can improve your example to contain a simple username matching?

[travisghansen]

@runningman84 good timing, I just landed expanded support for assertions (slightly updated syntax).

assertions: {
    ...
    userinfo: [
            {
              query_engine: "jp",
              query: "$.login",
              rule: {
                method: "eq",
                value: "someusername"
                //negate: true,
                //case_insensitive: true
              }
            }
          ]
    ....

exp is asserting the token itself is not expired (ie: basic jwt verification). It has no effect with github from what I can tell as github doesn't expire the tokens.

The documentation definitely needs some help :) Just letting the dust settle a bit before going too wild with it.

[travisghansen]

I forgot to mention, make sure to pull that latest code/image to use the above syntax.

[travisghansen]

I had a little bug in the version I mentioned to pull in the last comment. It's been cleaned up FYI and I'm very close to tagging a release finally.

[runningman84]

I just changed the config to include the userinfo like this:

        assertions: {
          /**
           * assert the token(s) has not expired
           */
          exp: true,
          userinfo: [
            {
              query_engine: "jp",
              query: "$.login",
              rule: {
                method: "eq",
                value: "runningman84"
                //negate: true,
                //case_insensitive: true
              }
            }
          ]
        },

But now every protected service throws a 500 http error.

Maybe my container is too old?

I also do not understand your helm chart, you are using a imagePullPolicy config but this is commented out in the values.yaml.

[travisghansen]

@runningman84 can you send over the logs to review? It could be an older image yes.

The imagePullPolicy is commented out in values.yaml which basically means, "let kubernetes decide" which pull policy to apply.

Kubernetes' logic is:

• if image tag is latest then Always
• otherwise IfNotPresent

So if you deployed with the chart and left that value alone, then simply deleteing the pod(s) currently running should force the latest image to pull when they restart.

[runningman84]

I just deleted the pod... but the error is still the same.

my logs are quite limited:

$ kubectl logs external-auth-server-5d46fb979-mk622 -n kube-system                                                                                                                  

> external-auth-server@0.1.0 start /app
> node --nouse-idle-notification --expose-gc --max-old-space-size=8192 src/server.js

store options: {"store":"memory","max":0,"ttl":0}
{"service":"external-auth-server","level":"info","message":"starting server on port 8080"}

traefik does not have any log at this timeframe....

[travisghansen]

If the request is erroring 500 you should see some error spit out on the container logs. Unless the traefik server is failing before it ever makes it there.

Can you try to revert the config token back to what it was and tell me if it starts working again?

[runningman84]

I reverted some sites back to the old config but the error is still there. Does the external auth server provider some debug logs?

[travisghansen]

Can you try with a private/incognito browser? I experienced something similar while developing where all the cookies data combined was quite large (cookie data from other services etc all combined) and it silently failed. Wondering if you've hit the same issue..

Related, I originally designed the server to store all session data (ie: stateless server side) in the cookie but quickly hit browser limits which is why I changed the design to store the sessions server side (redis/memory) and make the cookie simply be a session ID.

[runningman84]

incognito mode does not change anything :/

[travisghansen]

Must not be that then. If you deployed with the chart then you can set the logLevel value:

# set the logging level
# WARN: debug or above will log secrets
#
# error, warn, info, verbose, debug, silly
logLevel: "info"

I mean, you should see some data logging already if the server is receiving requests etc. Are you seeing anything in the logs at all? Might be good to setup a screenshare/conference to see if that's helpful at all..

Can you send the full response you're getting at the browser?

[runningman84]

Ok I have fixed the problem. My traefik service got a new loadbalancer ip and my fritzbox was still forwarding the packets to the old ip.

How does the communication work? Does traefik only internally talk to the ingress.kubernetes.io/auth-url? Or is the ingress.kubernetes.io/auth-url also accessed by the client browser?

[travisghansen]

@runningman84 auth-url value should point the to /verify endpoint of the service. It's assumed that endpoint is always being triggered via the reverse proxy itself (ie: sub request of a real request).

For the oauth2 and oidc plugins the service additionally exposes /oauth/callback which is meant to be hit directly by the browser (ie: not a auth request/sub request).

Does that make sense?

• https://github.com/travisghansen/external-auth-server#endpoints

[travisghansen]

Did you get the assertions to work?

[runningman84]

I guess yes but you can do the final test. Just check your mails.

[runningman84]

Do you have an example for these use cases?

• org everyone (orgname eq arvatoaws)
• org specific team (orgname eq arvatoaws and team eq devops)

[travisghansen]

@runningman84 this is a bit nuanced depending on what you really want. But here are a couple examples using team/org IDs (I'd recommend that over names). For your first use-case there are a couple ways:

  {
    query: "$.organizations[*].id",
    rule: {
      method: "contains",
      value: 1

      //negate: true,
      //case_insensitive: true
    }
  }

For the second use-case (I'm assuming team IDs are globally unique):

  {
    query_engine: "jp",
    query: "$.teams[*].id",
    rule: {
      method: "contains",
      value: 1

      //negate: true,
      //case_insensitive: true
    }
  }

If you want to allow to be part of a list of teams/orgs change the method to contains-any and set the value to an array of values like value: [1, 3, 9].

If fetching organizations, teams and emails is turned on the data for each is added to the userinfo data with the respective names/keys:

userinfo.organizations = [...]
userinfo.teams = [...]
userinfo.emails = [...]

• https://developer.github.com/v3/orgs/#list-your-organizations
• https://developer.github.com/v3/teams/#list-user-teams
• https://developer.github.com/v3/users/emails/#emails

Lastly, remember that all the assertions added are LOGICAL AND, meaning ALL of them must pass assertion or the result is a failure.

[travisghansen]

Any luck with these?

[runningman84]

I suppose they will work but I cannot really test it because the browser complains about a too long url.

[travisghansen]

Can you give me more detail? Something I can look into?

[runningman84]

Github throws this error:

414 Request-URI Too Large
nginx

The url looks like this:
https://github.com/login/oauth/authorize?response_type=code&client_id=07fcc7b7c3696fee3c8a&redirect_uri=https%3A%2F%2Feas.example.com%2Foauth%2Fcallback%3F__eas_oauth_handler__%3Dauthorization_callback&scope=user&state=273377c96c3e52fe3f4ad.......................

[travisghansen]

That's pretty strange by itself. Even more so that adding assertions would impact it at all.

Do you only get that with the added assertions or does it do that generally to you now?

[runningman84]

I haven't had time to reproduce it myself. A team member tried it. Do you have an ETA for the server side configuration?

[travisghansen]

It's the next big item for me. Wrapping up some header work and then on to that.

Mostly struggling how to configure that and keep it flexible. Store them in redis? SQL based storage? Or something else altogether...like hitting some other url and let it be completely managed externally? Any feedback is welcome on that front :)

[runningman84]

I would like to store the config in a configmap using helm. I would rather have the auth server without any persistent storage. Imagine there is some kind of storage problem and you cannot access some admin services because their auth relies on storage too.

[travisghansen]

Wise words!

[travisghansen]

I'm very close to landing server-side token support. I dreamed up a structure I really like that I think will be flexible and sane. I'm guessing I'll have it landed in the next branch before mid week as an FYI.

Guessing 0.2.0 will land sometime the week following with 3 primary features:

• server-side tokens
• custom service headers (ie: downstream headers based off of userinfo/token data/etc)
• firebase plugin

[travisghansen]

Just landed server-side tokens among other things. See CONFIG_TOKENS.md for an idea. Doc is still sparse but may be enough to get you going.

[travisghansen]

Anything else you need on this issue?

[runningman84]

@travisghansen I got this error while using your github org example:

{"message":"starting verify pipeline","level":"info","service":"external-auth-server"}
{"service":"external-auth-server","level":"info","message":"starting verify for plugin: oauth2"}
{"service":"external-auth-server","level":"warn","message":"failed assertion: {\"query_engine\":\"jp\",\"query\":\"$.organizations[*].id\",\"rule\":{\"method\":\"contains\",\"value\":12345678}} against value: []"}

My config looks like this:

let config_token = {
  /**
   * future feature: allow blocking certain token IDs
   */
  //jti: <some known value>

  /**
   * using the same aud for multiple tokens allows sso for all services sharing the aud
   */
  //aud: "some application id", //should be unique to prevent cookie/session hijacking, defaults to a hash unique to the whole config
  eas: {
    // list of plugin definitions, refer to PLUGINS.md for details
    plugins: [
      {
        type: "oauth2",
        issuer: {
          authorization_endpoint: "https://github.com/login/oauth/authorize",
          token_endpoint: "https://github.com/login/oauth/access_token"
        },
        client: {
          client_id: "aaaaaaaaaaaaaaa",
          client_secret: "bbbbbbbbbbbbbbbbbbbbbbbbb"
        },
        scopes: ["user"],
        /**
         * static redirect URI
         * if your oauth provider does not support wildcards place the URL configured in the provider (that will return to this proper service) here
         */
        redirect_uri: "https://eas.example.com/oauth/callback",
        features: {
          /**
           * if false cookies will be 'session' cookies
           * if true and cookies expire will expire with tokens
           */
          cookie_expiry: false,

          userinfo_expiry: 86400, // 24 hours

          /**
           * sessions become a floating window *if* tokens are being refreshed or userinfo being refreshed
           */
          session_expiry: 604800, // 7 days

          /**
           * if session_expiry is a number and this is set then sessions become a 'floating window'
           * if activity is triggered in this amount of time *before* preceeding the end of the
           * session then the expiration time is extended + session_expiry
           */
          session_expiry_refresh_window: 86400, // 24 hours

          /**
           * will re-use the same id (ie: same cookie) for a particular client if a session has expired
           */
          session_retain_id: true,

          /**
           * if the access token is expired and a refresh token is available, refresh
           */
          refresh_access_token: true,

          /**
           * fetch userinfo and include as X-Userinfo header to backing service
           */
          fetch_userinfo: true,

          userinfo: {
            provider: "github",
            config: {
              fetch_teams: true,
              fetch_organizations: true,
              fetch_emails: true
            }
          },

          /**
           * which token (if any) to send back to the proxy as the Authorization Bearer value
           * note the proxy must allow the token to be passed to the backend if desired
           *
           * possible values are access_token, or refresh_token
           */
          //authorization_token: "access_token"
        },
        assertions: {
          /**
           * assert the token(s) has not expired
           */
          exp: true,           
          userinfo: [
            {
              query_engine: "jp",
              query: "$.organizations[*].id",
              rule: {
                method: "contains",
                value: 12345678
                //negate: true,
                //case_insensitive: true
              }
            }
          ]
        },
        cookie: {
          name: "_eas_github_session_", //default is _oeas_oauth_session
          domain: "dev.example.com" //defaults to request domain, could do sso with more generic domain
          //path: "/",
        }
      }
    ]
  }
};

Do you have any idea how to fix this issue? (12345678 is our redacted github org id)

[runningman84]

I spend some time debugging this issue. It looks like the organizations is empty for my users. Funny enough my users have an team array which also contain an organisation id. This is from the debug logs:

{
    "iat": 1563454644,
    "data": {
        "login": "testuser",
        "id": 12345678,
        "node_id": "MDQ6VXNlcjE2OTkxMjg=",
        "avatar_url": "https://avatars2.githubusercontent.com/u/12345678?v=4",
        "gravatar_id": "",
        "url": "https://api.github.com/users/testuser",
        "html_url": "https://github.com/testuser",
        "followers_url": "https://api.github.com/users/testuser/followers",
        "following_url": "https://api.github.com/users/testuser/following{/other_user}",
        "gists_url": "https://api.github.com/users/testuser/gists{/gist_id}",
        "starred_url": "https://api.github.com/users/testuser/starred{/owner}{/repo}",
        "subscriptions_url": "https://api.github.com/users/testuser/subscriptions",
        "organizations_url": "https://api.github.com/users/testuser/orgs",
        "repos_url": "https://api.github.com/users/testuser/repos",
        "events_url": "https://api.github.com/users/testuser/events{/privacy}",
        "received_events_url": "https://api.github.com/users/testuser/received_events",
        "type": "User",
        "site_admin": false,
        "name": "xxx yyy",
        "company": null,
        "blog": "",
        "location": "Germany",
        "email": null,
        "hireable": null,
        "bio": null,
        "public_repos": 48,
        "public_gists": 14,
        "followers": 12,
        "following": 6,
        "created_at": "2012-05-02T14:14:03Z",
        "updated_at": "2019-05-29T20:47:04Z",
        "private_gists": 1,
        "total_private_repos": 4,
        "owned_private_repos": 4,
        "disk_usage": 2425,
        "collaborators": 0,
        "two_factor_authentication": true,
        "plan": {
            "name": "free",
            "space": 976562499,
            "collaborators": 0,
            "private_repos": 10000
        },
        "organizations": [],
        "teams": [
            {
                "name": "example-Bootcamp",
                "id": 423425345,
                "node_id": "MDQ6VGVhbTIxNjcxNTU=",
                "slug": "example-bootcamp",
                "description": "",
                "privacy": "closed",
                "url": "https://api.github.com/teams/423425345",
                "html_url": "https://github.com/orgs/example/teams/example-bootcamp",
                "members_url": "https://api.github.com/teams/423425345/members{/member}",
                "repositories_url": "https://api.github.com/teams/423425345/repos",
                "permission": "pull",
                "created_at": "2016-10-24T08:54:01Z",
                "updated_at": "2016-10-24T08:54:01Z",
                "members_count": 11,
                "repos_count": 1,
                "organization": {
                    "login": "example",
                    "id": 12345678,

[travisghansen]

Interesting, I don't know enough about the GitHub API to say one way or another but glad you got it worked out! If you need further help I can dig a little deeper just reopen and let me know.