GovCloud and other partition issues in v4-sign-http-request

[lambda-9]

Dependencies

{:deps {com.cognitect.aws/api           {:mvn/version "0.8.391"}
        com.cognitect.aws/endpoints     {:mvn/version "1.1.11.670"}
        com.cognitect.aws/sts           {:mvn/version "770.2.568.0"}}

Description with failing test case

v4-sign-http-request does not work with alternate AWS partitions such as GovCloud (aws-us-gov). This is due to logic defaulting to using the us-east-1 region to sign requests on global endpoints.

user> (let [sts-client (aws/client {:api :sts :region "us-gov-west-1"})]
                 (aws/invoke sts-client {:op :GetCallerIdentity}))

{:ErrorResponse
 {:Error
  {:Type "Sender",
   :TypeAttrs {},
   :Code "SignatureDoesNotMatch",
   :CodeAttrs {},
   :Message
   "Credential should be scoped to a valid region, not 'us-east-1'. ",
   :MessageAttrs {}},
  :ErrorAttrs {},
  :RequestId "da8736e5-07de-11ea-a0de-5157849b0806",
  :RequestIdAttrs {}},
 :ErrorResponseAttrs
 {:xmlns "https://sts.amazonaws.com/doc/2011-06-15/"},
 :cognitect.anomalies/category :cognitect.anomalies/forbidden}

I would not expect to see credentials scoped to us-east-1 in GovCloud. Global endpoints are only global per partition.

The request references the correct endpoint, but the request is signed with the wrong credential scope as shown in the authorization headers:

user> (let [sts-client (aws/client {:api :sts :region "us-gov-west-1"})]
                 (get
                  (meta (aws/invoke sts-client {:op :GetCallerIdentity}))
                  :http-request))
{:request-method :post,
 :scheme :https,
 :server-port 443,
 :uri "/",
 :headers
 {"x-amz-date" "20191115T201319Z",
  "content-type" "application/x-www-form-urlencoded; charset=utf-8",
  "host" "sts.us-gov-west-1.amazonaws.com",
  "x-amz-security-token"
  "<REDACTED>",
  "authorization"
  "AWS4-HMAC-SHA256 Credential=<REDACTED>/20191115/us-east-1/sts/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=<REDACTED>"},
 :body
 #object[java.nio.HeapByteBuffer 0x70eaf46c "java.nio.HeapByteBuffer[pos=0 lim=43 cap=43]"],
 :server-name "sts.us-gov-west-1.amazonaws.com"}

Problem Source

In v4-sign-http-request:

 (let [{:keys [:aws/access-key-id :aws/secret-access-key :aws/session-token]} credentials
        auth-info      { ; ...
                        :region            (if (and (-> service :metadata :globalEndpoint)
                                                    (not= "s3" (service/service-name service)))
                                             "us-east-1"
                                             (name region))}

To make matters more complicated, global endpoints may only be global in certain regions. STS in GovCloud is not a global endpoint according to the docs, Javascript SDK (clearest example of partition variations), and the Java SDK.

Failing test

(t/testing "testing signing-key credential scope"
  (t/is
   (clojure.string/includes?
    (let [sts-client (aws/client {:api :sts :region "us-gov-west-1"})]
      (-> (filterv #(clojure.string/starts-with? % "Credential=")
                   (-> (aws/invoke sts-client {:op :GetCallerIdentity})
                       meta
                       (get-in [:http-request :headers "authorization"])
                       (clojure.string/split #"\s")))
            first
            (clojure.string/split #"/")
            (nth 2))) ; credentialScope
    "us-gov")))

[dchelimsky]

Thanks for the repro and pointer to the problem spot. It's very helpful.

[dchelimsky]

@bennyhahmeen I have a prototype solution. I was able to test it manually with every region except private regions (iso), china, and gov't regions.

When attempting to test with a gov region, I get InvalidToken instead of SignatureDoesNotMatch, which I'd expect since I don't have credentials for gov regions.

Would you be willing to test it? I'm assuming you have access to an account in a gov region. If that is correct, and you're willing to help out, please set your deps as listed below and try your example above.

{:deps {com.cognitect.aws/api           {:git/url "https://github.com/cognitect-labs/aws-api.git"
                                         :sha "81bf067dd032d70e435a10687ce99b91cdb58999"}
        com.cognitect.aws/endpoints     {:mvn/version "1.1.11.670"}
        com.cognitect.aws/sts           {:mvn/version "770.2.568.0"}}}

[lambda-9]

I do have valid credentials in the GovCloud partition and I was able to test using the the deps provided.

I can confirm that the fix does allow me to call STS GetCallerIdentity successfully. I also tested another API in GovCloud partition (Organizations) and it works successfully as well. Thank you! That was quick.

[dchelimsky]

You're welcome! Your report pointed right at the problem and I just had to a little bit of research to find a good path.

[dchelimsky]

Addressed in ebeba57 (on master).

[dchelimsky]

Fix released in 0.8.408