Skip to content
Permalink
Browse files

adds support for JWT access tokens (#923)

* adds implementation of jwt authenticator

* adds jwt authenticator settings

* adds integration test

* addresses PR feedback
- add timer to JWKS refresh
- add retries to JWKS refresh
- updates comments

* addresses PR feedback
- adds support for disabling the JWT auth
- JWT auth doesn't implement the authenticator interface

* triggers JWT authentication if auth header is deemed to be an access token

* adds support for access tokens generated using rs256

* makes the supported algorithms configurable

* passes control downstream even if JWT auth fails

* allows spnego auth to choose correct authorization header

* resolves the cheshire dependency used by buddy and jet

* formatting changes

* makes jet authentication disabled by default

* addresses PR feedback
- changes info to debug
- moves request->host from utils to jwt
- renames request->host to request->realm

* introduces jwks server to enable jwt testing

* removes access-token creation from integration tests

* updates ci scripts for jwt tests

* addresses PR feedback
- uses ring/ring-core to parse query params
- renames integration tests
- moves generate-jwt-access-token to jwt-test

* avoids hacky use of private buddy.sign.jwt/validate-claims

* adds integration tests to check the 401 unauthorized and 403 forbidden paths

* corrects test - 403 responses do not content www-authenticate header

* adds sources of the provided keys
  • Loading branch information...
shamsimam authored and sradack committed Sep 7, 2019
1 parent acf5b60 commit 82dd01f62e50e97c5da197414bb28f845616a4ff
Showing with 1,929 additions and 73 deletions.
  1. +19 −0 containers/test-apps/jwks-server/.gitignore
  2. +38 −0 containers/test-apps/jwks-server/README.md
  3. +44 −0 containers/test-apps/jwks-server/project.clj
  4. +90 −0 containers/test-apps/jwks-server/resources/jwks.json
  5. +10 −0 containers/test-apps/jwks-server/resources/log4j.properties
  6. +3 −0 containers/test-apps/jwks-server/resources/settings.edn
  7. +37 −0 containers/test-apps/jwks-server/src/jwks_server/config.clj
  8. +115 −0 containers/test-apps/jwks-server/src/jwks_server/handler.clj
  9. +80 −0 containers/test-apps/jwks-server/src/jwks_server/main.clj
  10. +28 −0 waiter/bin/ci/jwks-server-setup.sh
  11. +6 −0 waiter/bin/ci/run-integration-tests-composite-scheduler.sh
  12. +6 −0 waiter/bin/ci/run-integration-tests-k8s-scheduler.sh
  13. +6 −0 waiter/bin/ci/run-integration-tests-marathon-scheduler.sh
  14. +10 −1 waiter/config-composite.edn
  15. +21 −0 waiter/config-full.edn
  16. +10 −1 waiter/config-k8s.edn
  17. +1 −2 waiter/config-minimal.edn
  18. +10 −1 waiter/config-minimesos.edn
  19. +2 −1 waiter/config-shell.edn
  20. +219 −2 waiter/integration/waiter/authentication_test.clj
  21. +4 −0 waiter/project.clj
  22. +30 −4 waiter/src/waiter/auth/authentication.clj
  23. +363 −0 waiter/src/waiter/auth/jwt.clj
  24. +18 −15 waiter/src/waiter/auth/spnego.clj
  25. +33 −11 waiter/src/waiter/core.clj
  26. +2 −2 waiter/src/waiter/handler.clj
  27. +13 −0 waiter/src/waiter/schema.clj
  28. +4 −2 waiter/src/waiter/settings.clj
  29. +10 −0 waiter/src/waiter/util/client_tools.clj
  30. +12 −0 waiter/src/waiter/util/ring_utils.clj
  31. +90 −0 waiter/test-files/jwt/jwks.json
  32. +2 −1 waiter/test/waiter/auth/authenticator_test.clj
  33. +468 −0 waiter/test/waiter/auth/jwt_test.clj
  34. +40 −1 waiter/test/waiter/auth/spnego_test.clj
  35. +60 −29 waiter/test/waiter/core_test.clj
  36. +25 −0 waiter/test/waiter/settings_test.clj
@@ -0,0 +1,19 @@
/target
/classes
/checkouts
pom.xml
pom.xml.asc
lib/
*.class
/.lein-*
/.nrepl-port
log/

# Intellij specific files
.idea
*.iml
*.ipr

# test2junit output files
build.xml
test2junit/
@@ -0,0 +1,38 @@
The jwks-server is a server to help with JWT access token authentication testing.
It supports two endpoints: `get-token` and `keys`.

# Usage

To run the server:
```bash
$ lein run <port> <key-file> <settings-file>
```

## Example:

```bash
$ lein run 8080 resources/jwks.json resources/settings.edn
...
jwks-server.main - command-line arguments: [8080 resources/jwks.json resources/settings.edn]
jwks-server.main - port: 8080
jwks-server.main - jwks file: resources/jwks.json
jwks-server.main - settings file: resources/settings.edn
jwks-server.main - starting server on port 8080
...
eclipse.jetty.server.Server - Started @10846ms
```

# Build Uberjar

```bash
$ lein uberjar
...
Created /path-to-waiter-jwks-server/target/uberjar/jwks-server-0.1.0-SNAPSHOT.jar
Created /path-to-waiter-jwks-server/target/uberjar/jwks-server-0.1.0-SNAPSHOT-standalone.jar
```

# JWKS keys

The keys stored in [jwks.json](resources/jwks.json) were obtained as follows:
- EdDSA keys were generated [uisng nimbus library's OctetKeyPair](https://connect2id.com/blog/nimbus-jose-jwt-6).
- The RS256 key was obtained from [buddy tests](https://github.com/funcool/buddy-sign/blob/master/test/buddy/sign/jwk_tests.clj).
@@ -0,0 +1,44 @@
;;
;; Copyright (c) Two Sigma Open Source, LLC
;;
;; Licensed under the Apache License, Version 2.0 (the "License");
;; you may not use this file except in compliance with the License.
;; You may obtain a copy of the License at
;;
;; http://www.apache.org/licenses/LICENSE-2.0
;;
;; Unless required by applicable law or agreed to in writing, software
;; distributed under the License is distributed on an "AS IS" BASIS,
;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
;; See the License for the specific language governing permissions and
;; limitations under the License.
;;
(defproject jwks-server "0.1.0-SNAPSHOT"
:dependencies [[buddy/buddy-sign "3.1.0"]
[cheshire "5.9.0"]
[org.clojure/clojure "1.10.1"]
[org.clojure/core.async "0.4.500"]
[org.clojure/data.json "0.2.6"]
[org.clojure/tools.logging "0.5.0"]
[org.slf4j/slf4j-log4j12 "1.7.28"]
[prismatic/plumbing "0.5.5"]
[ring/ring-core "1.7.1"
:exclusions [org.clojure/tools.reader]]
[twosigma/jet "0.7.10-20190831_055713-gf193d34"
:exclusions [org.eclipse.jetty/jetty-client
org.eclipse.jetty.alpn/alpn-api
org.eclipse.jetty.http2/http2-client
org.eclipse.jetty.websocket/websocket-client
org.eclipse.jetty/jetty-alpn-openjdk8-client
org.mortbay.jetty.alpn/alpn-boot]]]
:jvm-opts ["-server"
"-XX:+UseG1GC"
"-XX:MaxGCPauseMillis=50"]
:main ^:skip-aot jwks-server.main
:profiles {:uberjar {:aot :all}}
:resource-paths ["resources"]
:target-path "target/%s"
:test-selectors {:default (every-pred (complement :dev) (complement :integration))
:dev :dev
:integration (every-pred :integration (complement :explicit))}
:uberjar-name ~(System/getenv "UBERJAR_NAME"))
@@ -0,0 +1,90 @@
{
"keys": [
{
"alg": "RS256",
"e": "AQAB",
"kid": "7c368fc914ce6cb181fa0d670f63bd5df6db7b25",
"kty": "RSA",
"n": "vWmir2ZdXeMZkfsg0GTPfQw7CKmDNu50Sc76pndZPNyLf5JeR39JueHIPVXJ",
"use": "enc"
},
{
"alg": "RS256",
"d": "Eq5xpGnNCivDflJsRQBXHx1hdR1k6Ulwe2JZD50LpXyWPEAeP88vLNO97IjlA7_GQ5sLKMgvfTeXZx9SE-7YwVol2NXOoAJe46sui395IW_GO-pWJ1O0BkTGoVEn2bKVRUCgu-GjBVaYLU6f3l9kJfFNS3E0QbVdxzubSu3Mkqzjkn439X0M_V51gfpRLI9JYanrC4D4qAdGcopV_0ZHHzQlBjudU2QvXt4ehNYTCBr6XCLQUShb1juUO1ZdiYoFaFQT5Tw8bGUl_x_jTj3ccPDVZFD9pIuhLhBOneufuBiB4cS98l2SR_RQyGWSeWjnczT0QU91p1DhOVRuOopznQ",
"e": "AQAB",
"kid": "wwjcyqculjybrlzo0tzwjjniusfb4p4fakdotbf6",
"kty": "RSA",
"n": "ofgWCuLjybRlzo0tZWJjNiuSfb4p4fAkd_wWJcyQoTbji9k0l8W26mPddxHmfHQp-Vaw-4qPCJrcS2mJPMEzP1Pt0Bm4d4QlL-yRT-SFd2lZS-pCgNMsD1W_YpRPEwOWvG6b32690r2jZ47soMZo9wGzjb_7OMg0LOL-bSf63kpaSHSXndS5z5rexMdbBYUsLA9e-KXBdQOS-UTo7WTBEMa2R2CapHg665xsmtdVMTBQY4uDZlxvb3qCo5ZwKh9kG4LT6_I5IhlJH7aGhyxXFvUK-DWNmoudF8NAco9_h9iaGNj8q2ethFkMLs91kzk2PAcDTW9gb54h4FRWyuXpoQ",
"use": "sig"
},
{
"crv": "Ed25519",
"d": "8OSgWqVALOqZyLTLBcVJSgRZ6yxo0Wk_FkNG7uRCV28",
"kid": "f8f78b86-af1b-46c5-8849-006020f34d83",
"kty": "OKP",
"use": "sig",
"x": "H5UiUvUNr0OcxjEXpGCfFkYig-63Cs2SV_q6Y_5uDck"
},
{
"crv": "Ed25519",
"d": "NIX7-o9XrTLAYNanapB9_fIeOnZUHug2jhwbZ6zs6dE",
"kid": "b66fd551-5e49-46e3-8c3b-06b953a16f6b",
"kty": "OKP",
"use": "sig",
"x": "gLYQSUzOu4T0xq12F_RCc51YgSc8w9GZK3WIyYRlH44"
},
{
"crv": "Ed25519",
"d": "SXbPPwVHzPYYTwwRG_27RuoTN15Xn8zctwND727B2no",
"kid": "3106d31a-87a2-482f-a8dd-b727a043de5a",
"kty": "OKP",
"use": "sig",
"x": "qE4PwqfWP9pQ8iVpPhvQY433hiAn_rzjH-CPTnY_8yQ"
},
{
"crv": "Ed25519",
"d": "dx_kwm1Kg47VwP-4DZHBO9bLdGgSu-w20VYwKyqHOw0",
"kid": "73711a87-eeb6-42be-b537-3fdeaa4872bb",
"kty": "OKP",
"use": "sig",
"x": "SycUT4WVy3AKTR_FxBjkAa468LocX8QrM480EK2EOoc"
},
{
"crv": "Ed25519",
"d": "kdE8RVHsC87AdRSOA9jrJct75r-oZtggCgPQmpZHQzg",
"kid": "bfc58a16-8e07-42af-a632-2d3165dbf859",
"kty": "OKP",
"use": "sig",
"x": "nkKQj8rkxfvGFRcXNCeVz236ePxBZMtuxAp_F2PeIvk"
},
{
"crv": "Ed25519",
"d": "oyi29vqepPO-ZCQLTnthB6-IWGME_88E3beFPwXK4Xs",
"kid": "e787c763-ffe1-405d-bdee-bc50ea0e3aa5",
"kty": "OKP",
"use": "sig",
"x": "nv4W45wfSuWBAKeY9hEJ_SLp1o3d0kdViCs9TpbsEWk"
},
{
"crv": "Ed25519",
"d": "sH8QTNekLsfdXA29vipsLHiC8xQHqwteWcFtjM5gexs",
"kid": "9bf169b7-d8cc-4f60-82bd-2d060b11653b",
"kty": "OKP",
"use": "sig",
"x": "Tw32CCAlFNn2jz6wwVAQOgABuuA4pyLfaSnwzTEVhoY"
},
{
"crv": "P-256",
"kid": "e787c763-ffe1-d8cc-b537-bc50ea0e3aa5",
"kty": "EC",
"x": "f83OJ3D2xF1Bg8vub9tLe1gHMzV76e8Tus9uPHvRVEU",
"y": "x_FEzRu9m36HLN_tue659LNpXW6pCyStikYjKIWI5a0"
},
{
"e": "AQAB",
"kid": "73711a87-eeb6-46e3-b537-2d060b11653b",
"kty": "RSA",
"n": "kWp2zRA23Z3vTL4uoe8kTFptxBVFunIoP4t_8TDYJrOb7D1iZNDXVeEsYKp6ppmrTZDAgd"
}
]
}
@@ -0,0 +1,10 @@
log4j.rootLogger=DEBUG, stdout

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{4} - %m%n
log4j.appender.stdout.Threshold=DEBUG

# DEBUG is way too noisy for some categories
log4j.logger.org.eclipse.jetty=INFO
@@ -0,0 +1,3 @@
{:issuer "test.com"
:subject-key :sub
:token-type "JWT"}
@@ -0,0 +1,37 @@
;;
;; Copyright (c) Two Sigma Open Source, LLC
;;
;; Licensed under the Apache License, Version 2.0 (the "License");
;; you may not use this file except in compliance with the License.
;; You may obtain a copy of the License at
;;
;; http://www.apache.org/licenses/LICENSE-2.0
;;
;; Unless required by applicable law or agreed to in writing, software
;; distributed under the License is distributed on an "AS IS" BASIS,
;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
;; See the License for the specific language governing permissions and
;; limitations under the License.
;;
(ns jwks-server.config
(:require [clojure.tools.logging :as log]))

(let [settings-promise (promise)]
(defn initialize-settings
[settings]
(log/info "settings:" settings)
(deliver settings-promise settings))

(defn retrieve-settings
[]
(deref settings-promise 0 {})))

(let [jwks-promise (promise)]
(defn initialize-jwks
[jwks]
(log/info "num jwks entries:" (-> jwks :keys count))
(deliver jwks-promise jwks))

(defn retrieve-jwks
[]
(deref jwks-promise 0 {})))
@@ -0,0 +1,115 @@
;;
;; Copyright (c) Two Sigma Open Source, LLC
;;
;; Licensed under the Apache License, Version 2.0 (the "License");
;; you may not use this file except in compliance with the License.
;; You may obtain a copy of the License at
;;
;; http://www.apache.org/licenses/LICENSE-2.0
;;
;; Unless required by applicable law or agreed to in writing, software
;; distributed under the License is distributed on an "AS IS" BASIS,
;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
;; See the License for the specific language governing permissions and
;; limitations under the License.
;;
(ns jwks-server.handler
(:require [buddy.core.keys :as buddy-keys]
[buddy.sign.jwt :as jwt]
[clojure.data.json :as json]
[clojure.tools.logging :as log]
[clojure.string :as str]
[jwks-server.config :as config]
[plumbing.core :as pc]
[ring.middleware.params :as ring-params]
[ring.util.request :as ring-request]))

(defn prepare-response
"Prepares and returns a standard response"
([{:keys [query-string request-method uri]} message status]
(prepare-response (cond-> {"message" message
"request-method" request-method
"uri" uri}
(not (str/blank? query-string))
(assoc "query-string" query-string))
status))
([data-map status]
{:body (json/write-str data-map)
:headers {"content-type" "application/json"
"server" "jwks-server"}
:status status}))

(defn generate-jwt-access-token
"Generates the JWT access token using the provided private key."
[alg jwk-entry payload header]
(let [private-key (-> jwk-entry pc/keywordize-map buddy-keys/jwk->private-key)
options {:alg alg :header header}]
(jwt/sign payload private-key options)))

(defn- request->query-params
"Like Ring's params-request, but doesn't try to pull params from the body."
[request]
(->> (or (ring-request/character-encoding request) "UTF-8")
(ring-params/assoc-query-params request)
:query-params))

(defn process-get-token-request
"Retrieves an JWT access token generated using a random EdDSA key."
[{:keys [query-string] :as request}]
(log/info "query string:" query-string)
(let [{:strs [host]} (request->query-params request)
_ (when (str/blank? host)
(throw (ex-info "host query parameter not provided" {:status 400})))
{:keys [keys]} (config/retrieve-jwks)
eddsa-keys (filter (fn [{:keys [crv]}] (= "Ed25519" crv)) keys)
{:keys [kid] :as entry} (rand-nth eddsa-keys)
_ (log/info "selected kid:" kid)
principal (System/getProperty "user.name")
_ (log/info "principal:" principal)
{:keys [issuer subject-key token-type]} (config/retrieve-settings)
subject-key (keyword subject-key)
expiry-time-secs (+ (long (/ (System/currentTimeMillis) 1000)) 600)
payload (cond-> {:aud (str host) :exp expiry-time-secs :iss issuer :sub principal}
(not= :sub subject-key) (assoc subject-key principal))
header {:kid kid :typ token-type}
access-token (generate-jwt-access-token :eddsa entry payload header)]
(prepare-response
{"access_token" access-token
"kid" kid
"issuer" issuer
"principal" principal
"realm" host
"subject-key" subject-key
"token-type" token-type}
200)))

(defn process-get-keys-request
"Returns the JWKS managed bu this server."
[_]
(prepare-response
(update (config/retrieve-jwks)
:keys
(fn [keys]
(map (fn [entry] (dissoc entry :d)) keys)))
200))

(defn request-handler
"Factory for the ring request handler."
[{:keys [headers request-method uri] :as request}]
(log/info "received" request-method "request as path" uri)
(log/info "request headers:" headers)
(try
(cond
(= uri "/get-token")
(if (= request-method :get)
(process-get-token-request request)
(prepare-response request "method not allowed" 405))
(= uri "/keys")
(if (= request-method :get)
(process-get-keys-request request)
(prepare-response request "method not allowed" 405))
:else
(prepare-response request "unsupported endpoint" 404))
(catch Throwable th
(log/error th "error in processing request")
(prepare-response request (.getMessage th) (:status (ex-data th) 500)))))

0 comments on commit 82dd01f

Please sign in to comment.
You can’t perform that action at this time.