Documenation for multiple upstreams?

[ekristen]

I've read through the README and looked at the code, although I'm still pretty new to Go. I don't understand how multiple upstreams is suppose to work? Can the upstreams be multiple different domains? Or only unique uri?

"If multiple, routing is based on path" is a little vague.

Any help would be appreciated.

[ploxiln]

It's true, the documentation of multiple upstreams should be improved.

The upstreams can be on multiple domains.

The path is always passed directly through the proxy, it isn't munged. If you include a path in the upstream, like --upstream='http://127.0.0.1:8080/wiki/', it doesn't prepend "/wiki/" to the path from the client's request, instead it looks for "/wiki/" at the beginning of the path in the client's request, and only uses that upstream if it's there. It's a matching rule mixed right in with proxy upstream. So I see how that can be confusing.

So, for example, if google_auth_proxy was listening on port 80 and had --upstream='http://127.0.0.1:8080/wiki/' --upstream='http://127.0.0.1:8081/pages/', then a request to http://127.0.0.1:80/wiki/stuff would proxy to the first upstream, and a request to http://127.0.0.1:80/pages/john would proxy to the second upstream.

[neurogenesis]

@ploxiln, this would work well if everything is being served from the same FQDN. Is there currently a way to get this to work with virtual domains?

For example if I have the following virtual servers configured in nginx:

host1.mydomain.com > localhost:8081
host2.mydomain.com > localhost:8082
...

how would I proxy multiple domains in such a way as to retain the routing after the client is redirected to the OAuth callback URL?

host1.mydomain.com > localhost:4180 > localhost:8081
host2.mydomain.com > localhost:4180 > localhost:8082

[ploxiln]

You're right, the current mechanism only supports distinguishing based on path, not based on host.

[neurogenesis]

i'm presently starting multiple proxies to handle multiple virtual servers for the same host, to ensure the URLs are portable across hosts. i get the feeling this won't scale well though.

thank you for the clarification though.

[skippy]

hey folks,
I think I figured out a way to make this work... any thoughts on how to simplify are greatly welcome! I hope this makes sense :)

1. in https://console.developers.google.com, choose your app, go to APIs and Auth => Credentials and set your REDIRECT URIS and JAVASCRIPT ORIGINS to https://auth.internal.example.com/oauth2/callback
2. for nginx, I have the following config files (with 2 internal apps; in fact this is protecting many more services, each with their own unique subdomain)

upstream google-auth {
  least_conn;
  server localhost:4180;
}

server {
  listen *:8080;
  server_name ~^auth.(?<domain>internal.*)$;
  location = /oauth2/callback {
    proxy_pass http://google-auth;
  }
  location ~/(?<sub>[^/]+)(?<remaining_uri>.*)$ {
    rewrite ^ $scheme://$sub.$domain$remaining_uri;
  }
}

server {
  listen *:8080;
  server_name ~^(.+).internal.*;
  location = /oauth2/start {
    proxy_pass http://google-auth/oauth2/start?rd=%2F$1;
  }
  location / {
    proxy_pass http://google-auth;
  }
}

# send to elasticsearch
upstream es {
  least_conn;
  server localhost:9200
}

server {
  listen *:8081;
  server_name es.internal.*;
  location / {
    proxy_pass  http://es;
    proxy_redirect     off;
  }
}

# send to consul
upstream consul {
  least_conn;
  server localhost:8500
}

server {
  listen *:8081;
  server_name consul.internal.*;
  location / {
    proxy_pass  http://consul;
    proxy_redirect     off;
  }
}

1. google-auth-proxy commands are:

  --upstream="http://localhost:8081" \
  --redirect-url="https://auth.internal.example.com/oauth2/callback" \
  --google-apps-domain="example.com" \
  --cookie-domain=".internal.example.com" \
  --client-id=$GOOGLE_AUTH_CLIENT_ID \
  --client-secret=$GOOGLE_AUTH_CLIENT_SECRET

1. lets wire this stuff up! Setup your env as needed, so that external traffic is coming in to port 8080. (My setup is a bit complex: AWS ELB (TCP passthrough for 443) => HAProxy (with SSL termination) => HAProxy sends all requests to *.internal.example.com to nginx servers on port 8080). At this point, we do some internal proxy redirection:
   • already authenticated: es.internal.:8000 => processed by google-auth => sent back to nginx and matches es.internal.:8081 => success!
   • auth needed: es.internal.:8000 => google-auth redirects to /oauth2/start => little regex magic in nginx to insert our subdomain => google-auth sends to google => after google authorizes, comes back to auth.internal./oauth2/callback, which redirects to auth.internal./es, which redirects back to es.internal. => and finally, success!

sorry, this is a bit hinky. I hope I rewrote the steps in a generic manner and that it isn't tied to my particular setup (aws, elb, docker, consul, consul-template, etc.) let me know if you have questions or thoughts on how to simplify this!

[ayashjorden]

Great solution @skippy !!!
Works like a charm.

Thanks,
Yarden

[ayashjorden]

Hi all (again),
Should ALL traffic be proxy-ed through oauth-proxy? event after authenticating ?

BTW, is there an IRC/google-groups/slack/other channels for discussion ?

Thanks,
Yarden

[ploxiln]

All traffic needs to be proxied through oauth2_proxy. Otherwise, what decides what traffic has already authenticated? oauth2_proxy looks at a cookie in the request in order to know.

[jrallison]

Hey all,

I'm finding the current upstream configuration options fairly limiting. For example, my current use-case is using oauth2-proxy as a single proxy for all our company's internal apps:

https://internal.mycompany.com/metrics -> internally built rails app
https://internal.mycompany.com/jenkins -> jenkins, open source CI software
https://internal.mycompany.com/rabbit -> rabbitmq, open source messaging/queuing interface
etc, etc

Any apps we built internally, it's easy enough to add a prefix, so that as oauth2-proxy routes all traffic from https://internal.mycompany.com/metrics/... to https://internal:8000/metrics/... the internal app serves it's root at /metrics rather than /

For jenkins, we've found configuration options to also add a prefix.

For other open source projects, like rabbitmq, it's not as simple to force it to serve it's root with a prefix, and we have to resort to wrapping it with an additional proxy to strip out the prefix.

Perhaps having an option to specify upstreams with a hash rather an array could make the routing more flexible while still being fairly simple?:

upstreams = {
  "/metrics" => "http://internal:1234/metrics",
  "/jenkins" => "http://jenkins.internal:8080/jenkins",
  "/rabbit" => "http://rabbit.internal:8080/",
  ...
}

[Tenzer]

As part of #142, I added some documentation to the README file on how the upstreams can be configured, you can find it under the Upstreams Configuration header. If that's sufficient, I guess this issue can be closed.

[jehiah]

📝++

[estelora]

A solution that worked for me:

1. Oauth2_proxy Configuration

./oauth2_proxy
  --cookie-domain=".internal.example.com" \ --upstream="http://localhost:8081"\
  --cookie-name="oauth2_proxy" \ --cookie-secret="<cookie-secret>" \
  --client-id="<client-id>.apps.googleusercontent.com" \
  --client-secret="<client-secret>" \
  --redirect-url="https://auth.internal.example.com/oauth2/callback" \
  --email-domain="example.com" \ --request-logging=true \ --pass-host-header=true \
  --tls-cert="/path/to/my.crt" \
  --tls-key="/path/to/my.key" \

2. Nginx Configurations

Sites-enabled: Google-auth Configuration

upstream google-auth  {
  least_conn;
  server 127.0.0.1:443;
}

server {
  rewrite_log on;
  listen *:8080;
  server_name ~^auth.(?<domain>internal.*)$;
  location = /oauth2/callback {
    proxy_pass https://google-auth;
  }
  location ~/(?<sub>[^/]+)(?<remaining_uri>.*)$ {
    rewrite ^ https://$sub.$domain$remaining_uri;
  }
}

server {
  listen *:8080;
  server_name ~^(.+).internal.*;
  location = /oauth2/start {
    proxy_pass https://google-auth/oauth2/start?rd=%2F$1;
  }
  location / {
    proxy_pass https://google-auth;
    proxy_set_header Host $host;
  }
}

Sites-enabled: Upstreams Configuration

upstream app2 {
  least_conn;
  server <app2 private IP>;
}

upstream app1 {
  least_conn;
  server <app1 private IP>;
}

server {
  access_log /var/log/nginx/default.log oauth;
  listen 8081;
  server_name "";
  add_header Content-Type text/plain;
}

 server {
  access_log /var/log/nginx/app1.log oauth;
  listen 8081;
  server_name ftp.internal.*;
  location / {
    proxy_pass  http://ftp;
    proxy_redirect     off;
  }
}

server {
  access_log /var/log/nginx/app2.log oauth;
  listen 8081;
  server_name app2.internal.*;
  location / {
    proxy_pass  http://app2;
    proxy_redirect     off;

  }
}

Nginx.conf Settings for Logging

rewrite_log on;
    log_format oauth '"[$time_local]" host: "$host" host_header: "$http_HOST" server_name:  "$server_name"  "$request" remote address: "$remote_addr" to upstream: "$upstream_addr:"

1. Assumes

2. External traffic is coming into port 8080
3. AWS ELB (TCP passthrough for 443)