diff --git a/deploy/README.md b/deploy/README.md index 4eaa0588..8ccb9977 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -2,7 +2,7 @@ ### Set up SSH -Ask Mike Bland or Eric Mill to help add your public key (typically `$HOME/.ssh/id_rsa.pub`) to `$HOME/.ssh/authorized_keys` on `hub.18f.us` and `18f.gsa.gov`. Then make sure your `$HOME/.ssh/config` file on your machine contains the following entries: +Ask Mike Bland or Eric Mill to help add your public key (typically `$HOME/.ssh/id_rsa.pub`) to `$HOME/.ssh/authorized_keys` on `hub.18f.gov` and `18f.gsa.gov`. Then make sure your `$HOME/.ssh/config` file on your machine contains the following entries: ``` Host 18f-site @@ -12,7 +12,7 @@ Host 18f-site IdentitiesOnly yes Host 18f-hub - Hostname hub.18f.us + Hostname hub.18f.gov User ubuntu IdentityFile [$HOME]/.ssh/id_rsa IdentitiesOnly yes @@ -22,19 +22,19 @@ This configuration allows you to log into each machine as the appropriate user b ### DEPRECATED: `publish.sh` and `publish-prod.sh` -Not much to see here; after regenerating the site with Jekyll, [publish.sh](publish.sh) just `rsync`s it to `hub.18f.us`. [publish-prod.sh](publish-prod.sh) does the same thing for the Public Hub. +Not much to see here; after regenerating the site with Jekyll, [publish.sh](publish.sh) just `rsync`s it to `hub.18f.gov`. [publish-prod.sh](publish-prod.sh) does the same thing for the Public Hub. Now that automated deployments are underway (see below), these scripts are deprecated. They remain as examples for those wishing to deploy their own Hub instances quickly. ### AWS -https://hub.18f.us/ is running as an AWS EC2 instance named `18f-hub` based on 18F's `m3.medium` image. The AWS Elastic IP of the instance is assigned to the `hub.18f.us` subdomain via the AWS Route 53 panel. The associated AWS Security Group restricts access to the SSH, HTTP, and HTTPS ports (22, 80, and 443), but those ports are reachable from any source IP address. +https://hub.18f.gov/ is running as an AWS EC2 instance named `18f-hub` based on 18F's `m3.medium` image. The AWS Elastic IP of the instance is assigned to the `hub.18f.gov` subdomain via the AWS Route 53 panel. The associated AWS Security Group restricts access to the SSH, HTTP, and HTTPS ports (22, 80, and 443), but those ports are reachable from any source IP address. The Public Hub is served directly from https://18f.gsa.gov/. ### Nginx -For the internal Hub, [/etc/nginx/nginx.conf](etc/nginx/nginx.conf) is the stock 18F nginx config that comes with the image, modified to include the Hub-specific config, [/etc/nginx/vhosts/hub.conf](etc/nginx/vhosts/hub.conf), further described in the Google Auth Proxy section below: +For the internal Hub, [/etc/nginx/nginx.conf](etc/nginx/nginx.conf) is the stock 18F nginx config that comes with the image, modified to include the Hub-specific config, [/etc/nginx/vhosts/hub.conf](etc/nginx/vhosts/hub.conf), further described in the OAuth2 Proxy section below: ``` ## @@ -53,36 +53,40 @@ For the internal hub, [/etc/nginx/vhosts/hub.conf](etc/nginx/vhosts/hub.conf) is The public Hub is served by the same web server as https://18f.gsa.gov and doesn't require its own configuration. -### Google Auth Proxy +### OAuth2 Proxy _This pertains to the internal Hub only._ -The [etc](etc) and [usr](usr) subdirectory trees contain the files needed to configure Nginx and the [Google Auth Proxy](https://github.com/bitly/google_auth_proxy) in concert to ensure only 18F team members can access the Hub. The current version of `google_auth_proxy` running on `hub.18f.us` is 1.0; the latest version can be downloaded from: https://github.com/bitly/google_auth_proxy/releases and unpacked on the Hub machine as `/usr/local/18f/bin/google_auth_proxy`. +The [etc](etc) and [usr](usr) subdirectory trees contain the files needed to configure Nginx and the [OAuth2 Proxy](https://github.com/bitly/oauth2_proxy) in concert to ensure only 18F team members can access the Hub. The current version of `oauth2_proxy` running on `hub.18f.gov` was built at commit a80aad04f7bbe821bca9ea9659fef04c869ac970. The latest packaged version can be downloaded from: https://github.com/bitly/oauth2_proxy/releases and unpacked on the Hub machine as `/usr/local/18f/bin/oauth2_proxy`. -* [/etc/init.d/google_auth_proxy](etc/init.d/google_auth_proxy): Enables the `google_auth_proxy` service to be started and stopped like any other standard service, via `sudo service google_auth_proxy [start|stop|restart]`. +* [/etc/init.d/oauth2_proxy](etc/init.d/oauth2_proxy): Enables the `oauth2_proxy` service to be started and stopped like any other standard service, via `sudo service oauth2_proxy [start|stop|restart]`. -* [/etc/nginx/vhosts/hub.conf](etc/nginx/vhosts/hub.conf): All `http://` requests are permanently redirected (301) to the `https://` equivalent by the first `server` block. The second `server` block, listening for `https://` requests, is configured to forward all requests (except for the logo used for the Google OAuth consent screen) to the `google_auth_proxy` service. The content of the site is ultimately served by the final `server` block, accessible only by the running `google_auth_proxy` instance on the localhost, given the AWS Security Group port restrictions. +* [/etc/nginx/vhosts/hub.conf](etc/nginx/vhosts/hub.conf): All `http://` requests are permanently redirected (301) to the `https://` equivalent by the first `server` block. The second `server` block, listening for `https://` requests, is configured to forward all requests (except for the logo used for the consent screen) to the `oauth2_proxy` service. The content of the site is ultimately served by the final `server` block, accessible only by the running `oauth2_proxy` instance on the localhost, given the AWS Security Group port restrictions. *Notice the `port_in_redirect off;` line in the third `server` block:* Without this line, permanent redirects from directory URLs _without_ a trailing slash to directory URLs _with_ a trailing slash will include the local server's port, which will cause the redirect to fail. -* [/usr/local/18f/bin/google_auth_proxy.sh](usr/local/18f/bin/google_auth_proxy.sh): A shim that allows the `google_auth_proxy` logs to be captured in `/var/log/google_auth_proxy/access.log`. +* [/usr/local/18f/bin/oauth2_proxy.sh](usr/local/18f/bin/oauth2_proxy.sh): A shim that allows the `oauth2_proxy` logs to be captured in `/var/log/oauth2_proxy/access.log`. -* [/usr/local/18f/etc/google_auth_proxy.cfg](usr/local/18f/etc/google_auth_proxy.cfg): The configuration file for the `google_auth_proxy`, specified as a command line flag by [/etc/init.d/google_auth_proxy](etc/init.d/google_auth_proxy). The `client_id` and `client_secret` fields have been redacted from the repository. Currently they come from Mike Bland's personal account, since https://console.developers.google.com/ is currently disabled for the gsa.gov domain. A dedicated, shared `18f-hub-admin@gsa.gov` account would be ideal. For progress on this front, see 18F DevOps issues [60](https://github.com/18F/DevOps/issues/60) and [79](https://github.com/18F/DevOps/issues/79). +* [/usr/local/18f/etc/oauth2_proxy.cfg](usr/local/18f/etc/oauth2_proxy.cfg): The configuration file for the `oauth2_proxy`, specified as a command line flag by [/etc/init.d/oauth2_proxy](etc/init.d/oauth2_proxy). The `client_id` and `client_secret` fields have been redacted from the repository. Currently they contain Mike Bland's MyUSA app credentials for the 18F Hub. + + Previously they came from Mike Bland's personal account, since https://console.developers.google.com/ is currently disabled for the gsa.gov domain. A dedicated, shared `18f-hub-admin@gsa.gov` account would be ideal. For progress on this front, see 18F DevOps issues [60](https://github.com/18F/DevOps/issues/60) and [79](https://github.com/18F/DevOps/issues/79). *Notice the `authenticated_emails_file` setting, and that `google_apps_domains` has been commented out.* Access is granted to the union of these two sets, i.e. to everyone in the `authenticated_emails_file` _or_ in the `google_apps_domains`. #### Single Sign-On -For the details on how the Hub's `google_auth_proxy` instance is configured as a single sign-on service for multiple 18f.gov domains, see the [Single Sign-On instructions](SSO.md). +For the details on how the Hub's `oauth2_proxy` instance is configured as a single sign-on service for multiple 18f.gov domains, see the [Single Sign-On instructions](SSO.md). #### Adding/removing users -The `authenticated_emails_file` is the list of Google Apps-authenticated users authorized to access the Hub. It is currently generated by the [auth.rb](../_plugins/auth.rb) plugin. Whenever team member email address information is updated in [_data/private/team.yml](../_data/private/team.yml) or [_data/private/hub/guest_users.yml](../_data/private/hub/guest_users.yml), a new version of this file will be generated, and the `google_auth_proxy` will need to be restarted. +The `authenticated_emails_file` is the list of authenticated users authorized to access the Hub. It is currently generated by the [auth.rb](../_plugins/auth.rb) plugin. Whenever team member email address information is updated in [_data/private/team.yml](../_data/private/team.yml) or [_data/private/hub/guest_users.yml](../_data/private/hub/guest_users.yml), a new version of this file will be generated, and the `oauth2_proxy` will reload the file. + +#### Configuration changes -One you have your `$HOME/.ssh/config` configured as described at the beginning of this document, you can restart the `google_auth_proxy` like so: +The `oauth2_proxy` must be restarted if you update `/usr/local/18f/etc/oauth2_proxy.cfg`. One you have your `$HOME/.ssh/config` configured as described at the beginning of this document, you can restart the `oauth2_proxy` like so: ``` -$ ssh 18f-hub sudo service google_auth_proxy restart +$ ssh 18f-hub sudo service oauth2_proxy restart ``` ### Preparing for automated deployment diff --git a/deploy/SSO.md b/deploy/SSO.md index 4f410918..4ca07171 100644 --- a/deploy/SSO.md +++ b/deploy/SSO.md @@ -1,10 +1,10 @@ ## Single Sign-On -The Hub's `google_auth_proxy` instance is configured as a single sign-on -service for multiple 18f.gov sub-domains using a single set of MyUSA OAuth +The Hub's `oauth2_proxy` instance is configured as a single sign-on service +for multiple 18f.gov sub-domains using a single set of MyUSA OAuth credentials. The configuration, inspired by the instructions in -[bitly/google_auth_proxy#12](https://github.com/bitly/google_auth_proxy/issues/12), -is subtle, but effective. +[bitly/oauth2_proxy#12](https://github.com/bitly/oauth2_proxy/issues/12), is +subtle, but effective. ### MyUSA @@ -12,7 +12,7 @@ The **Redirect uri** field in the Hub's MyUSA application settings is set to `https://auth.18f.gov/oauth2/callback`. auth.18f.gov is defined in `auth.conf`, described below. -### /usr/local/18f/etc/google_auth_proxy.cfg +### /usr/local/18f/etc/oauth2_proxy.cfg These are the critical two lines: @@ -23,14 +23,14 @@ cookie_domain = ".18f.gov" The `cookie_domain` ensures that a single `.18f.gov` auth cookie will provide access to any application hosted on an 18f.gov sub-domain (for which the Hub's -`google_auth_proxy` is configured to act as a reverse proxy, of course). +`oauth2_proxy` is configured to act as a reverse proxy, of course). ### /etc/nginx/vhosts/auth-locations.conf -Include this file at the end of every Nginx `server` block for which you want the -`google_auth_proxy` to provide authentication. The `location /` block is the +Include this file at the end of every Nginx `server` block for which you want +the `oauth2_proxy` to provide authentication. The `location /` block is the normal case; after the user is authenticated, all requests will be routed -through this block and passed to the `google_auth_proxy`. +through this block and passed to the `oauth2_proxy`. ``` location / { @@ -49,9 +49,9 @@ the magic hack that makes single sign-on work. It sets the prefix of the `rd` (short for "redirect") parameter of the `/oauth2/start` query string to the name of the server in which this block is included. -For example, for hub.18f.gov/foo, it will be `rd=%2Fhub.18f.gov/foo`. `%2F` -is the hex-encoded version of the `/` character, so this is saying that once -authentication is complete, the `google_auth_proxy` should redirect back to +For example, for hub.18f.gov/foo, it will be `rd=%2Fhub.18f.gov/foo`. `%2F` is +the hex-encoded version of the `/` character, so this is saying that once +authentication is complete, the `oauth2_proxy` should redirect back to `/hub.18f.gov/foo`. As we will see, the full URL of the redirect will be `https://auth.18f.gov/hub.18f.gov/foo`. @@ -70,8 +70,8 @@ location = /oauth2/start { ### /etc/nginx/vhosts/auth.conf This file defines the `auth.18f.gov` server. The `/oauth2/callback` block -forwards the request from the OAuth provider to the `google_auth_proxy` -running locally on port 4180: +forwards the request from the OAuth provider to the `oauth2_proxy` running +locally on port 4180: ``` location = /oauth2/callback { @@ -82,11 +82,11 @@ running locally on port 4180: } ``` -After the OAuth handshake succeeds, the `google_auth_proxy` will redirect back -to the `rd` URL generated by the `/oauth/start` block from above. The -following block parses the target host (e.g. hub.18f.gov) from the redirected -request and forwards the request to the target host. Using the earlier -example, `https://auth.18f.gov/hub.18f.gov/foo` will be rewritten as +After the OAuth handshake succeeds, the `oauth2_proxy` will redirect back to +the `rd` URL generated by the `/oauth/start` block from above. The following +block parses the target host (e.g. hub.18f.gov) from the redirected request +and forwards the request to the target host. Using the earlier example, +`https://auth.18f.gov/hub.18f.gov/foo` will be rewritten as `https://hub.18f.gov/foo`. ``` diff --git a/deploy/ansible/roles/dev/tasks/google_auth_proxy.yml b/deploy/ansible/roles/dev/tasks/google_auth_proxy.yml deleted file mode 100644 index 6a230075..00000000 --- a/deploy/ansible/roles/dev/tasks/google_auth_proxy.yml +++ /dev/null @@ -1,27 +0,0 @@ ---- -- name: Download Google Auth Proxy - get_url: - url=https://github.com/bitly/google_auth_proxy/releases/download/v1.0/google_auth_proxy-1.0.linux-amd64.go1.3.tar.gz - dest=/home/vagrant/google_auth_proxy-1.0.linux-amd64.go1.3.tar.gz -- name: Unpack Google Auth Proxy - unarchive: - src=/home/vagrant/google_auth_proxy-1.0.linux-amd64.go1.3.tar.gz - dest=/home/vagrant - copy=no -- name: Symlink Google Auth Proxy - file: - src=/home/vagrant/google_auth_proxy-1.0.linux-amd64.go1.3/google_auth_proxy - dest=/usr/local/18f/bin/google_auth_proxy - state=link -- name: Copy Google Auth Proxy wrapper script - copy: - src=../../../../usr/local/18f/bin/google_auth_proxy.sh - dest=/usr/local/18f/bin/google_auth_proxy.sh -- name: Copy Google Auth Proxy config - copy: - src=../../../../usr/local/18f/etc/google_auth_proxy.cfg - dest=/usr/local/18f/etc/google_auth_proxy.cfg -- name: Copy Google Auth Proxy init script - copy: - src=../../../../etc/init.d/google_auth_proxy - dest=/etc/init.d/google_auth_proxy diff --git a/deploy/ansible/roles/dev/tasks/main.yml b/deploy/ansible/roles/dev/tasks/main.yml index 83469baf..c30a5f84 100644 --- a/deploy/ansible/roles/dev/tasks/main.yml +++ b/deploy/ansible/roles/dev/tasks/main.yml @@ -1,3 +1,3 @@ --- - include: nginx.yml -#- include: google_auth_proxy.yml +#- include: oauth2_proxy.yml diff --git a/deploy/ansible/roles/dev/tasks/oauth2_proxy.yml b/deploy/ansible/roles/dev/tasks/oauth2_proxy.yml new file mode 100644 index 00000000..aea24380 --- /dev/null +++ b/deploy/ansible/roles/dev/tasks/oauth2_proxy.yml @@ -0,0 +1,27 @@ +--- +- name: Download OAuth2 Proxy + get_url: + url=https://github.com/bitly/oauth2_proxy/releases/download/v1.0/google_auth_proxy-1.0.linux-amd64.go1.3.tar.gz + dest=/home/vagrant/oauth2_proxy-1.0.linux-amd64.go1.3.tar.gz +- name: Unpack OAuth2 Proxy + unarchive: + src=/home/vagrant/oauth2_proxy-1.0.linux-amd64.go1.3.tar.gz + dest=/home/vagrant + copy=no +- name: Symlink OAuth2 Proxy + file: + src=/home/vagrant/oauth2_proxy-1.0.linux-amd64.go1.3/google_auth_proxy + dest=/usr/local/18f/bin/oauth2_proxy + state=link +- name: Copy OAuth2 Proxy wrapper script + copy: + src=../../../../usr/local/18f/bin/oauth2_proxy.sh + dest=/usr/local/18f/bin/oauth2_proxy.sh +- name: Copy OAuth2 Proxy config + copy: + src=../../../../usr/local/18f/etc/oauth2_proxy.cfg + dest=/usr/local/18f/etc/oauth2_proxy.cfg +- name: Copy OAuth2 Proxy init script + copy: + src=../../../../etc/init.d/oauth2_proxy + dest=/etc/init.d/oauth2_proxy diff --git a/deploy/etc/init.d/google_auth_proxy b/deploy/etc/init.d/oauth2_proxy similarity index 76% rename from deploy/etc/init.d/google_auth_proxy rename to deploy/etc/init.d/oauth2_proxy index b4167c81..beb8a33b 100755 --- a/deploy/etc/init.d/google_auth_proxy +++ b/deploy/etc/init.d/oauth2_proxy @@ -1,40 +1,40 @@ #! /bin/sh ### BEGIN INIT INFO -# Provides: google_auth_proxy +# Provides: oauth2_proxy # Required-Start: $all # Required-Stop: $all # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 -# Short-Description: starts the google_auth_proxy for the 18F Hub -# Description: starts the google_auth_proxy for the 18F Hub +# Short-Description: starts the oauth2_proxy for the 18F Hub +# Description: starts the oauth2_proxy for the 18F Hub ### END INIT INFO ### # Originally sourced from Jamie Curle: # http://jamie.curle.io/blog/compiling-nginx-ubuntu/ # https://s3-eu-west-1.amazonaws.com/jcurle/code/nginx.sh -# +# # Modified by Eric Mill, for 18F, 2014-07-15. # https://github.com/18f/DevOps # # Modified by Mike Bland, for 18F, 2014-11-24. # https://github.com/18F/hub -### +### PATH=/sbin:/bin:/usr/sbin:/usr/bin -DAEMON=/usr/local/18f/bin/google_auth_proxy.sh -DAEMON_OPTS="--config=/usr/local/18f/etc/google_auth_proxy.cfg" -NAME=google_auth_proxy -DESC=google_auth_proxy -LOGS=/var/log/google_auth_proxy -PIDFILE=/var/run/google_auth_proxy.run +DAEMON=/usr/local/18f/bin/oauth2_proxy.sh +DAEMON_OPTS="--config=/usr/local/18f/etc/oauth2_proxy.cfg" +NAME=oauth2_proxy +DESC=oauth2_proxy +LOGS=/var/log/oauth2_proxy +PIDFILE=/var/run/oauth2_proxy.run test -x $DAEMON || exit 0 # Include nginx defaults if available -if [ -f /etc/default/google_auth_proxy ] ; then - . /etc/default/google_auth_proxy +if [ -f /etc/default/oauth2_proxy ] ; then + . /etc/default/oauth2_proxy fi set -e diff --git a/deploy/publish.sh b/deploy/publish.sh index dc77456f..7b3ee93f 100755 --- a/deploy/publish.sh +++ b/deploy/publish.sh @@ -27,5 +27,5 @@ pushd $(dirname $0) >/dev/null HUB_ROOT=$(dirname $(pwd -P)) popd >/dev/null -REMOTE="ubuntu@hub.18f.us:18f-hub" +REMOTE="ubuntu@hub.18f.gov:18f-hub" rsync -e ssh -vaxp --delete --ignore-errors $HUB_ROOT/_site{,_public} $REMOTE diff --git a/deploy/usr/local/18f/bin/google_auth_proxy.sh b/deploy/usr/local/18f/bin/google_auth_proxy.sh deleted file mode 100755 index f89b1e69..00000000 --- a/deploy/usr/local/18f/bin/google_auth_proxy.sh +++ /dev/null @@ -1,25 +0,0 @@ -#! /bin/sh -# -# 18F Hub - Docs & connections between team members, projects, and skill sets -# -# Written in 2014 by Mike Bland (michael.bland@gsa.gov) -# on behalf of the 18F team, part of the US General Services Administration: -# https://18f.gsa.gov/ -# -# To the extent possible under law, the author(s) have dedicated all copyright -# and related and neighboring rights to this software to the public domain -# worldwide. This software is distributed without any warranty. -# -# You should have received a copy of the CC0 Public Domain Dedication along -# with this software. If not, see -# . -# -# @author Mike Bland (michael.bland@gsa.gov) - -LOGS=/var/log/google_auth_proxy - -if [ ! -d $LOGS ]; then - mkdir -p $LOGS -fi - -exec /usr/local/18f/bin/google_auth_proxy "$@" >>$LOGS/access.log 2>&1 diff --git a/deploy/usr/local/18f/bin/oauth2_proxy.sh b/deploy/usr/local/18f/bin/oauth2_proxy.sh new file mode 100755 index 00000000..fb6ca2a8 --- /dev/null +++ b/deploy/usr/local/18f/bin/oauth2_proxy.sh @@ -0,0 +1,8 @@ +#! /bin/sh +LOGS=/var/log/oauth2_proxy + +if [ ! -d $LOGS ]; then + mkdir -p $LOGS +fi + +exec /usr/local/18f/bin/oauth2_proxy "$@" >>$LOGS/access.log 2>&1 diff --git a/deploy/usr/local/18f/etc/google_auth_proxy.cfg b/deploy/usr/local/18f/etc/oauth2_proxy.cfg similarity index 91% rename from deploy/usr/local/18f/etc/google_auth_proxy.cfg rename to deploy/usr/local/18f/etc/oauth2_proxy.cfg index c7f78f34..2315bd19 100644 --- a/deploy/usr/local/18f/etc/google_auth_proxy.cfg +++ b/deploy/usr/local/18f/etc/oauth2_proxy.cfg @@ -1,5 +1,5 @@ -## Google Auth Proxy Config File -## https://github.com/bitly/google_auth_proxy +## OAuth2 Proxy Config File +## https://github.com/bitly/oauth2_proxy ## : to listen on for HTTP clients http_address = "127.0.0.1:4180" @@ -21,7 +21,7 @@ pass_basic_auth = true #] -## The Google OAuth Client ID, Secret +## The OAuth2 Client ID, Secret # mbland: I've redacted these values from the repository. On the Hub, these # are set to the app keys from my MyUSA account. client_id = "" @@ -44,6 +44,8 @@ authenticated_emails_file = "/home/ubuntu/hub/_site/auth/hub-authenticated-email cookie_secret = "" cookie_domain = ".18f.gov" cookie_expire = "168h" +cookie_refresh = "144h" cookie_https_only = true + provider = "myusa" pass_access_token = true