adds support for JWT access tokens (#923)

* adds implementation of jwt authenticator * adds jwt authenticator settings * adds integration test * addresses PR feedback - add timer to JWKS refresh - add retries to JWKS refresh - updates comments * addresses PR feedback - adds support for disabling the JWT auth - JWT auth doesn't implement the authenticator interface * triggers JWT authentication if auth header is deemed to be an access token * adds support for access tokens generated using rs256 * makes the supported algorithms configurable * passes control downstream even if JWT auth fails * allows spnego auth to choose correct authorization header * resolves the cheshire dependency used by buddy and jet * formatting changes * makes jet authentication disabled by default * addresses PR feedback - changes info to debug - moves request->host from utils to jwt - renames request->host to request->realm * introduces jwks server to enable jwt testing * removes access-token creation from integration tests * updates ci scripts for jwt tests * addresses PR feedback - uses ring/ring-core to parse query params - renames integration tests - moves generate-jwt-access-token to jwt-test * avoids hacky use of private buddy.sign.jwt/validate-claims * adds integration tests to check the 401 unauthorized and 403 forbidden paths * corrects test - 403 responses do not content www-authenticate header * adds sources of the provided keys
twosigma · Sep 7, 2019 · 82dd01f · 82dd01f
1 parent acf5b60
commit 82dd01f
Show file tree

Hide file tree

Showing 36 changed files with 1,929 additions and 73 deletions.
diff --git a/containers/test-apps/jwks-server/.gitignore b/containers/test-apps/jwks-server/.gitignore
@@ -0,0 +1,19 @@
+/target
+/classes
+/checkouts
+pom.xml
+pom.xml.asc
+lib/
+*.class
+/.lein-*
+/.nrepl-port
+log/
+
+# Intellij specific files
+.idea
+*.iml
+*.ipr
+
+# test2junit output files
+build.xml
+test2junit/
diff --git a/containers/test-apps/jwks-server/README.md b/containers/test-apps/jwks-server/README.md
@@ -0,0 +1,38 @@
+The jwks-server is a server to help with JWT access token authentication testing.
+It supports two endpoints: `get-token` and `keys`.
+
+# Usage
+
+To run the server:
+```bash
+$ lein run <port> <key-file> <settings-file>
+```
+
+## Example:
+
+```bash
+$ lein run 8080 resources/jwks.json resources/settings.edn
+...
+jwks-server.main - command-line arguments: [8080 resources/jwks.json resources/settings.edn]
+jwks-server.main - port: 8080
+jwks-server.main - jwks file: resources/jwks.json
+jwks-server.main - settings file: resources/settings.edn
+jwks-server.main - starting server on port 8080
+...
+eclipse.jetty.server.Server - Started @10846ms
+```
+
+# Build Uberjar
+
+```bash
+$ lein uberjar
+...
+Created /path-to-waiter-jwks-server/target/uberjar/jwks-server-0.1.0-SNAPSHOT.jar
+Created /path-to-waiter-jwks-server/target/uberjar/jwks-server-0.1.0-SNAPSHOT-standalone.jar
+```
+
+# JWKS keys
+
+The keys stored in [jwks.json](resources/jwks.json) were obtained as follows:
+- EdDSA keys were generated [uisng nimbus library's OctetKeyPair](https://connect2id.com/blog/nimbus-jose-jwt-6).
+- The RS256 key was obtained from [buddy tests](https://github.com/funcool/buddy-sign/blob/master/test/buddy/sign/jwk_tests.clj).
diff --git a/containers/test-apps/jwks-server/project.clj b/containers/test-apps/jwks-server/project.clj
@@ -0,0 +1,44 @@
+;;
+;; Copyright (c) Two Sigma Open Source, LLC
+;;
+;; Licensed under the Apache License, Version 2.0 (the "License");
+;; you may not use this file except in compliance with the License.
+;; You may obtain a copy of the License at
+;;
+;;  http://www.apache.org/licenses/LICENSE-2.0
+;;
+;; Unless required by applicable law or agreed to in writing, software
+;; distributed under the License is distributed on an "AS IS" BASIS,
+;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+;; See the License for the specific language governing permissions and
+;; limitations under the License.
+;;
+(defproject jwks-server "0.1.0-SNAPSHOT"
+  :dependencies [[buddy/buddy-sign "3.1.0"]
+                 [cheshire "5.9.0"]
+                 [org.clojure/clojure "1.10.1"]
+                 [org.clojure/core.async "0.4.500"]
+                 [org.clojure/data.json "0.2.6"]
+                 [org.clojure/tools.logging "0.5.0"]
+                 [org.slf4j/slf4j-log4j12 "1.7.28"]
+                 [prismatic/plumbing "0.5.5"]
+                 [ring/ring-core "1.7.1"
+                  :exclusions [org.clojure/tools.reader]]
+                 [twosigma/jet "0.7.10-20190831_055713-gf193d34"
+                  :exclusions [org.eclipse.jetty/jetty-client
+                               org.eclipse.jetty.alpn/alpn-api
+                               org.eclipse.jetty.http2/http2-client
+                               org.eclipse.jetty.websocket/websocket-client
+                               org.eclipse.jetty/jetty-alpn-openjdk8-client
+                               org.mortbay.jetty.alpn/alpn-boot]]]
+  :jvm-opts ["-server"
+             "-XX:+UseG1GC"
+             "-XX:MaxGCPauseMillis=50"]
+  :main ^:skip-aot jwks-server.main
+  :profiles {:uberjar {:aot :all}}
+  :resource-paths ["resources"]
+  :target-path "target/%s"
+  :test-selectors {:default (every-pred (complement :dev) (complement :integration))
+                   :dev :dev
+                   :integration (every-pred :integration (complement :explicit))}
+  :uberjar-name ~(System/getenv "UBERJAR_NAME"))
diff --git a/containers/test-apps/jwks-server/resources/jwks.json b/containers/test-apps/jwks-server/resources/jwks.json
@@ -0,0 +1,90 @@
+{
+  "keys": [
+    {
+      "alg": "RS256",
+      "e": "AQAB",
+      "kid": "7c368fc914ce6cb181fa0d670f63bd5df6db7b25",
+      "kty": "RSA",
+      "n": "vWmir2ZdXeMZkfsg0GTPfQw7CKmDNu50Sc76pndZPNyLf5JeR39JueHIPVXJ",
+      "use": "enc"
+    },
+    {
+      "alg": "RS256",
+      "d": "Eq5xpGnNCivDflJsRQBXHx1hdR1k6Ulwe2JZD50LpXyWPEAeP88vLNO97IjlA7_GQ5sLKMgvfTeXZx9SE-7YwVol2NXOoAJe46sui395IW_GO-pWJ1O0BkTGoVEn2bKVRUCgu-GjBVaYLU6f3l9kJfFNS3E0QbVdxzubSu3Mkqzjkn439X0M_V51gfpRLI9JYanrC4D4qAdGcopV_0ZHHzQlBjudU2QvXt4ehNYTCBr6XCLQUShb1juUO1ZdiYoFaFQT5Tw8bGUl_x_jTj3ccPDVZFD9pIuhLhBOneufuBiB4cS98l2SR_RQyGWSeWjnczT0QU91p1DhOVRuOopznQ",
+      "e": "AQAB",
+      "kid": "wwjcyqculjybrlzo0tzwjjniusfb4p4fakdotbf6",
+      "kty": "RSA",
+      "n": "ofgWCuLjybRlzo0tZWJjNiuSfb4p4fAkd_wWJcyQoTbji9k0l8W26mPddxHmfHQp-Vaw-4qPCJrcS2mJPMEzP1Pt0Bm4d4QlL-yRT-SFd2lZS-pCgNMsD1W_YpRPEwOWvG6b32690r2jZ47soMZo9wGzjb_7OMg0LOL-bSf63kpaSHSXndS5z5rexMdbBYUsLA9e-KXBdQOS-UTo7WTBEMa2R2CapHg665xsmtdVMTBQY4uDZlxvb3qCo5ZwKh9kG4LT6_I5IhlJH7aGhyxXFvUK-DWNmoudF8NAco9_h9iaGNj8q2ethFkMLs91kzk2PAcDTW9gb54h4FRWyuXpoQ",
+      "use": "sig"
+    },
+    {
+      "crv": "Ed25519",
+      "d": "8OSgWqVALOqZyLTLBcVJSgRZ6yxo0Wk_FkNG7uRCV28",
+      "kid": "f8f78b86-af1b-46c5-8849-006020f34d83",
+      "kty": "OKP",
+      "use": "sig",
+      "x": "H5UiUvUNr0OcxjEXpGCfFkYig-63Cs2SV_q6Y_5uDck"
+    },
+    {
+      "crv": "Ed25519",
+      "d": "NIX7-o9XrTLAYNanapB9_fIeOnZUHug2jhwbZ6zs6dE",
+      "kid": "b66fd551-5e49-46e3-8c3b-06b953a16f6b",
+      "kty": "OKP",
+      "use": "sig",
+      "x": "gLYQSUzOu4T0xq12F_RCc51YgSc8w9GZK3WIyYRlH44"
+    },
+    {
+      "crv": "Ed25519",
+      "d": "SXbPPwVHzPYYTwwRG_27RuoTN15Xn8zctwND727B2no",
+      "kid": "3106d31a-87a2-482f-a8dd-b727a043de5a",
+      "kty": "OKP",
+      "use": "sig",
+      "x": "qE4PwqfWP9pQ8iVpPhvQY433hiAn_rzjH-CPTnY_8yQ"
+    },
+    {
+      "crv": "Ed25519",
+      "d": "dx_kwm1Kg47VwP-4DZHBO9bLdGgSu-w20VYwKyqHOw0",
+      "kid": "73711a87-eeb6-42be-b537-3fdeaa4872bb",
+      "kty": "OKP",
+      "use": "sig",
+      "x": "SycUT4WVy3AKTR_FxBjkAa468LocX8QrM480EK2EOoc"
+    },
+    {
+      "crv": "Ed25519",
+      "d": "kdE8RVHsC87AdRSOA9jrJct75r-oZtggCgPQmpZHQzg",
+      "kid": "bfc58a16-8e07-42af-a632-2d3165dbf859",
+      "kty": "OKP",
+      "use": "sig",
+      "x": "nkKQj8rkxfvGFRcXNCeVz236ePxBZMtuxAp_F2PeIvk"
+    },
+    {
+      "crv": "Ed25519",
+      "d": "oyi29vqepPO-ZCQLTnthB6-IWGME_88E3beFPwXK4Xs",
+      "kid": "e787c763-ffe1-405d-bdee-bc50ea0e3aa5",
+      "kty": "OKP",
+      "use": "sig",
+      "x": "nv4W45wfSuWBAKeY9hEJ_SLp1o3d0kdViCs9TpbsEWk"
+    },
+    {
+      "crv": "Ed25519",
+      "d": "sH8QTNekLsfdXA29vipsLHiC8xQHqwteWcFtjM5gexs",
+      "kid": "9bf169b7-d8cc-4f60-82bd-2d060b11653b",
+      "kty": "OKP",
+      "use": "sig",
+      "x": "Tw32CCAlFNn2jz6wwVAQOgABuuA4pyLfaSnwzTEVhoY"
+    },
+    {
+      "crv": "P-256",
+      "kid": "e787c763-ffe1-d8cc-b537-bc50ea0e3aa5",
+      "kty": "EC",
+      "x": "f83OJ3D2xF1Bg8vub9tLe1gHMzV76e8Tus9uPHvRVEU",
+      "y": "x_FEzRu9m36HLN_tue659LNpXW6pCyStikYjKIWI5a0"
+    },
+    {
+      "e": "AQAB",
+      "kid": "73711a87-eeb6-46e3-b537-2d060b11653b",
+      "kty": "RSA",
+      "n": "kWp2zRA23Z3vTL4uoe8kTFptxBVFunIoP4t_8TDYJrOb7D1iZNDXVeEsYKp6ppmrTZDAgd"
+    }
+  ]
+}
diff --git a/containers/test-apps/jwks-server/resources/log4j.properties b/containers/test-apps/jwks-server/resources/log4j.properties
@@ -0,0 +1,10 @@
+log4j.rootLogger=DEBUG, stdout
+
+log4j.appender.stdout=org.apache.log4j.ConsoleAppender
+log4j.appender.stdout.Target=System.out
+log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
+log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{4} - %m%n
+log4j.appender.stdout.Threshold=DEBUG
+
+# DEBUG is way too noisy for some categories
+log4j.logger.org.eclipse.jetty=INFO
diff --git a/containers/test-apps/jwks-server/resources/settings.edn b/containers/test-apps/jwks-server/resources/settings.edn
@@ -0,0 +1,3 @@
+{:issuer "test.com"
+ :subject-key :sub
+ :token-type "JWT"}
diff --git a/containers/test-apps/jwks-server/src/jwks_server/config.clj b/containers/test-apps/jwks-server/src/jwks_server/config.clj
@@ -0,0 +1,37 @@
+;;
+;; Copyright (c) Two Sigma Open Source, LLC
+;;
+;; Licensed under the Apache License, Version 2.0 (the "License");
+;; you may not use this file except in compliance with the License.
+;; You may obtain a copy of the License at
+;;
+;;  http://www.apache.org/licenses/LICENSE-2.0
+;;
+;; Unless required by applicable law or agreed to in writing, software
+;; distributed under the License is distributed on an "AS IS" BASIS,
+;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+;; See the License for the specific language governing permissions and
+;; limitations under the License.
+;;
+(ns jwks-server.config
+  (:require [clojure.tools.logging :as log]))
+
+(let [settings-promise (promise)]
+  (defn initialize-settings
+    [settings]
+    (log/info "settings:" settings)
+    (deliver settings-promise settings))
+
+  (defn retrieve-settings
+    []
+    (deref settings-promise 0 {})))
+
+(let [jwks-promise (promise)]
+  (defn initialize-jwks
+    [jwks]
+    (log/info "num jwks entries:" (-> jwks :keys count))
+    (deliver jwks-promise jwks))
+
+  (defn retrieve-jwks
+    []
+    (deref jwks-promise 0 {})))
diff --git a/containers/test-apps/jwks-server/src/jwks_server/handler.clj b/containers/test-apps/jwks-server/src/jwks_server/handler.clj
@@ -0,0 +1,115 @@
+;;
+;; Copyright (c) Two Sigma Open Source, LLC
+;;
+;; Licensed under the Apache License, Version 2.0 (the "License");
+;; you may not use this file except in compliance with the License.
+;; You may obtain a copy of the License at
+;;
+;;  http://www.apache.org/licenses/LICENSE-2.0
+;;
+;; Unless required by applicable law or agreed to in writing, software
+;; distributed under the License is distributed on an "AS IS" BASIS,
+;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+;; See the License for the specific language governing permissions and
+;; limitations under the License.
+;;
+(ns jwks-server.handler
+  (:require [buddy.core.keys :as buddy-keys]
+            [buddy.sign.jwt :as jwt]
+            [clojure.data.json :as json]
+            [clojure.tools.logging :as log]
+            [clojure.string :as str]
+            [jwks-server.config :as config]
+            [plumbing.core :as pc]
+            [ring.middleware.params :as ring-params]
+            [ring.util.request :as ring-request]))
+
+(defn prepare-response
+  "Prepares and returns a standard response"
+  ([{:keys [query-string request-method uri]} message status]
+   (prepare-response (cond-> {"message" message
+                              "request-method" request-method
+                              "uri" uri}
+                       (not (str/blank? query-string))
+                       (assoc "query-string" query-string))
+                     status))
+  ([data-map status]
+   {:body (json/write-str data-map)
+    :headers {"content-type" "application/json"
+              "server" "jwks-server"}
+    :status status}))
+
+(defn generate-jwt-access-token
+  "Generates the JWT access token using the provided private key."
+  [alg jwk-entry payload header]
+  (let [private-key (-> jwk-entry pc/keywordize-map buddy-keys/jwk->private-key)
+        options {:alg alg :header header}]
+    (jwt/sign payload private-key options)))
+
+(defn- request->query-params
+  "Like Ring's params-request, but doesn't try to pull params from the body."
+  [request]
+  (->> (or (ring-request/character-encoding request) "UTF-8")
+    (ring-params/assoc-query-params request)
+    :query-params))
+
+(defn process-get-token-request
+  "Retrieves an JWT access token generated using a random EdDSA key."
+  [{:keys [query-string] :as request}]
+  (log/info "query string:" query-string)
+  (let [{:strs [host]} (request->query-params request)
+        _ (when (str/blank? host)
+            (throw (ex-info "host query parameter not provided" {:status 400})))
+        {:keys [keys]} (config/retrieve-jwks)
+        eddsa-keys (filter (fn [{:keys [crv]}] (= "Ed25519" crv)) keys)
+        {:keys [kid] :as entry} (rand-nth eddsa-keys)
+        _ (log/info "selected kid:" kid)
+        principal (System/getProperty "user.name")
+        _ (log/info "principal:" principal)
+        {:keys [issuer subject-key token-type]} (config/retrieve-settings)
+        subject-key (keyword subject-key)
+        expiry-time-secs (+ (long (/ (System/currentTimeMillis) 1000)) 600)
+        payload (cond-> {:aud (str host) :exp expiry-time-secs :iss issuer :sub principal}
+                  (not= :sub subject-key) (assoc subject-key principal))
+        header {:kid kid :typ token-type}
+        access-token (generate-jwt-access-token :eddsa entry payload header)]
+    (prepare-response
+      {"access_token" access-token
+       "kid" kid
+       "issuer" issuer
+       "principal" principal
+       "realm" host
+       "subject-key" subject-key
+       "token-type" token-type}
+      200)))
+
+(defn process-get-keys-request
+  "Returns the JWKS managed bu this server."
+  [_]
+  (prepare-response
+    (update (config/retrieve-jwks)
+            :keys
+            (fn [keys]
+              (map (fn [entry] (dissoc entry :d)) keys)))
+    200))
+
+(defn request-handler
+  "Factory for the ring request handler."
+  [{:keys [headers request-method uri] :as request}]
+  (log/info "received" request-method "request as path" uri)
+  (log/info "request headers:" headers)
+  (try
+    (cond
+      (= uri "/get-token")
+      (if (= request-method :get)
+        (process-get-token-request request)
+        (prepare-response request "method not allowed" 405))
+      (= uri "/keys")
+      (if (= request-method :get)
+        (process-get-keys-request request)
+        (prepare-response request "method not allowed" 405))
+      :else
+      (prepare-response request "unsupported endpoint" 404))
+    (catch Throwable th
+      (log/error th "error in processing request")
+      (prepare-response request (.getMessage th) (:status (ex-data th) 500)))))