General ideas for improving GitHub auth

[technosophos]

We'd like to figure out a way to improve the GitHub auth model so that we could:

• Use GitHub "Applications"
• Issue per-project keys based on a "master" key for a gateway
• In general, reduce the number of places where users/projects have to supply secrets for GitHub.

A few references:

https://developer.github.com/v3/users/keys/#create-a-public-key
https://developer.github.com/apps/building-oauth-apps/authorization-options-for-oauth-apps/#non-web-application-flow

Please leave comments with your ideas.

[jdotjdot]

No matter what the other security model options are, I do think that letting Github apps have access is sensible. Beyond that, I'd suggest actually making Brigade itself be an installable Github App.

I don't love per-project keys--that's basically what it is now and we've found it very unwieldy and fragile.

[squillace]

It's not even the AUTH, although that's a thing. I look at the hub model of interaction: https://github.com/github/hub does automatic log-in (supports 2FA) when you use it against a secured repo and THEN caches that value.

The UX should be the same for brigade. As an example, brig might well support a call to https://developer.github.com/v3/repos/hooks/#create-a-hook, enabling you to use brig to set up a webhook directly into a specific project.yaml file. Whilst that is NOT the requested design, I put it here to demonstrate the kind of interaction we should support somewhere. :-)

[technosophos]

Okay, now that GitHub has a Checks API, I have begun work on a new GitHub gateway. https://github.com/Azure/brigade-github-app

If anyone has a few spare cycles to test out this prototype, please let me know.

[squillace]

I'll have a look!

[technosophos]

We've got a prototype of the GitHub App method that should be landing soon.

[jdotjdot]

Nice!

I actually have a bash script for this that I should have shared with you long ago--if you'd like to see it, happy to share. If you are all set with current approach, that's great, too.

[technosophos]

That would be good. I'd like to see it if possible. With the new gateway, I have it running as a GitHub App, but right now it takes 12(!) manual steps to get it installed.

[jdotjdot]

We created our own Github app for this because we had to, but I actually think it would be possible to create an "official" Brigade Github App that users of Brigade can install into their repo and then hook up their local installation of Brigade to via the private key. That way, adding your Brigade to Github could be one-click (or pretty close to it) rather than a number of steps to create your own Github app, etc.

---

Here are the steps we followed.

First, you create a Github app, and then you install it to your organization/repo, and write down all the necessary tokens, etc. You should end up with three credentials that I map to three environmental variables that need to be set in the sidecar:

• BRIGADE_REPO_GITHUB_APP_IDENTIFIER
• BRIGADE_REPO_GITHUB_APP_INSTALLATION_ID
• BRIGADE_REPO_GITHUB_APP_PRIVATE_KEY

We currently set these via Vault, which is what the referenced entrypoint.sh below does—it authenticates to Vault and sets the environmental vars. But that could be done using Kubernetes secrets or anything else, without an entrypoint.

Then, we use the files below to have the sidecar authenticate as the github app, grab a short-lived OAuth token, and download the repo. Alternatively, the sidecar could be used just with jwt-auth.sh to grab a short-lived github token via the app for any type of purpose, so we use this also to grab OAuth tokens have our other Brigade jobs take various github actions using the API. Here is our brigade.js function:

/*
 * getGithubToken()
 * Runs a job that returns a short-lived Github OAuth2 token for the 
 * internal Github App
 * 
 * Usage: getGithubToken().then(token => {});
 */
function getGithubToken() {
    let getGithubToken = new Job("get-github-token", '$WAYUP_DOCKER_REGISTRY/brigade/git-sidecar:v0.2.1');
    getGithubToken.useSource = false;
    getGithubToken.storage.enabled = false;
    getGithubToken.imageForcePull = true;
    getGithubToken.tasks = [
        '/entrypoint.sh /askpass.sh'
    ];
    return getGithubToken.run().then((result) => {
        return result.data.trim();
    });
}

---

Here are our git sidecar files:

Dockerfile:

FROM alpine:3.7

RUN apk update && apk add --no-cache \
    bash \
    ca-certificates \
    curl \
    git \
    jq \
    openssh-client \
    openssl \
    && update-ca-certificates

COPY rootfs /

ENV GIT_SSH=/gitssh.sh
ENV GIT_ASKPASS=/askpass.sh

RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/bin/bash", "/entrypoint.sh"]
CMD ["/clone.sh"]

/gitssh.sh:

#!/bin/sh
extra=""

if [ "" != "${BRIGADE_REPO_KEY}" ]; then
  KEY="./id_dsa"
  echo ${BRIGADE_REPO_KEY} | sed 's/\$/\n/g' > $KEY
  chmod 600 $KEY
  extra="-i $KEY -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
fi

ssh $extra $@

/askpass.sh:

#!/bin/sh

if [ ! -z "${BRIGADE_REPO_AUTH_TOKEN+x}" ] ; then
  echo $BRIGADE_REPO_AUTH_TOKEN
elif [ ! -z "${BRIGADE_REPO_GITHUB_APP_IDENTIFIER+x}" ] && \
	[ ! -z "${BRIGADE_REPO_GITHUB_APP_INSTALLATION_ID+x}" ] && \
	[ ! -z "${BRIGADE_REPO_GITHUB_APP_PRIVATE_KEY+x}" ] ; then
  curl -s -XPOST "https://api.github.com/installations/$BRIGADE_REPO_GITHUB_APP_INSTALLATION_ID/access_tokens" \
	  -H 'Accept: application/vnd.github.machine-man-preview+json' \
	  -H "Authorization: Bearer $(bash /jwt-auth.sh RS256 '{"iss": "'$BRIGADE_REPO_GITHUB_APP_IDENTIFIER'"}' "$BRIGADE_REPO_GITHUB_APP_PRIVATE_KEY")" \
		| jq -r .token
fi  

/jws-auth.sh:

#!/usr/bin/env bash

# Inspired by implementation by Will Haley at:
#   http://willhaley.com/blog/generate-jwt-with-bash/
# Borrowed from https://stackoverflow.com/questions/46657001/how-do-you-create-an-rs256-jwt-assertion-with-bash-shell-scripting

set -o pipefail

build_header() {
  local header_template='{
    "typ": "JWT"
  }'

  jq -c \
        --arg iat_str "$(date +%s)" \
        --arg alg "${1:-HS256}" \
  '
  ($iat_str | tonumber) as $iat
  | .alg = $alg
  | .iat = $iat
  | .exp = ($iat + 1)
  ' <<<"$header_template" | tr -d '\n'
}

build_payload() {
  jq -c --arg iat_str "$(date +%s)" \
  '
  ($iat_str | tonumber) as $iat
  | .iat = $iat
  | .exp = ($iat + 600)
  ' <<<"$1" | tr -d '\n'
}

b64enc() { openssl enc -base64 -A | tr '+/' '-_' | tr -d '='; }
json() { jq -c . | LC_CTYPE=C tr -d '\n'; }
hs_sign() { openssl dgst -binary -sha"${1}" -hmac "$2"; }
rs_sign() { openssl dgst -binary -sha"${1}" -sign <(printf '%s\n' "$2"); }

sign() {
        local algo payload header sig secret=$3
        algo=${1:-RS256}
        header=$(build_header "$algo") || return
      	payload=$(build_payload "$2")
        signed_content="$(json <<<"$header" | b64enc).$(json <<<"$payload" | b64enc)"
        case $algo in
                HS*) sig=$(printf %s "$signed_content" | hs_sign "${algo#HS}" "$secret" | b64enc) ;;
                RS*) sig=$(printf %s "$signed_content" | rs_sign "${algo#RS}" "$secret" | b64enc) ;;
                *) echo "Unknown algorithm" >&2; return 1 ;;
        esac
        printf '%s.%s\n' "${signed_content}" "${sig}"
}

(( $# )) && sign "$@"

[krancour]

This probably should have been closed when the newer GitHub gateway was released. Closing. @technosophos feel free to re-open if you think I am wrong.