Skip to content

Commit

Permalink
Introduce shared http-client
Browse files Browse the repository at this point in the history 
* default aws-client constructor uses it
* client.api/stop won't stop it
  * but it will stop any other instance

Signed-off-by: Ghadi Shayban <ghadi@cognitect.com>
  • Loading branch information
@dchelimsky
dchelimsky committed Feb 10, 2020
1 parent 8506d17 commit f5acd9b
Show file tree
Hide file tree
Showing 16 changed files with 433 additions and 221 deletions.
7 changes: 7 additions & 0 deletions CHANGES.md
View file
Expand Up @@ -3,6 +3,13 @@
## DEV

* upgrade to cognitect.http-client 0.1.104 [#115](https://github.com/cognitect-labs/aws-api/issues/115)
* all aws clients use shared http-client, credentials-provider, and region-provider by default
* addresses [#109](https://github.com/cognitect-labs/aws-api/issues/109)
* first call to invoke takes hit of fetching region and credentials
* `com.cognitect.aws.api/stop` will not stop the shared http-client, but stop any other instance

See [Upgrade Notes](https://github.com/cognitect-labs/aws-api/blob/master/UPGRADE.md) for more
information about upgrading to this version.

## 0.8.423 / 2020-01-17

Expand Down
36 changes: 18 additions & 18 deletions README.md
View file
Expand Up @@ -216,27 +216,16 @@ the `:path` in the `:endpoint-override` map.

## http-client

NOTE: the behavior of `com.cognitect.aws.api/client` and `com.cognitect.aws.api/stop`
changed as of release 0.8.430. See [Upgrade
Notes](https://github.com/cognitect-labs/aws-api/blob/master/UPGRADE.md)
for more information.

The aws-api client uses an http-client to send requests to AWS,
including any operations you invoke _and_ fetching the region and
credentials when you're running in EC2 or ECS. By default, each
aws-api client creates its own http-client, which, in turn, manages
its own resources. Invoke `cognitect.aws.client.api/stop` on the
client if you want it to shut down any resources it and its
http-client are using.

If you're creating multiple aws-api clients, you can, optionally,
create a single http-client and share it across aws-api clients e.g.

``` clojure
(require '[cognitect.aws.client.api :as aws])
(def http-client (aws/default-http-client))
(def s3-client (aws/client {:api :s3 :http-client http-client}))
(def ssm-client (aws/client {:api :ssm :http-client http-client}))
;; etc
```

If you call `stop` on `s3-client` or `ssm-client` in this example, the
single http-client gets shut down for both.
aws-api client uses a single, shared http-client, whose resources
are managed by aws-api.

## Contributing

Expand Down Expand Up @@ -276,6 +265,17 @@ access.
Remedy: check [AWS Regions and Endpoints](https://docs.aws.amazon.com/general/latest/gr/rande.html),
and supply the correct endpoint as described in [nodename nor servname provided, or not known](#nodename-nor-servname-provided-or-not-known), above.

#### Ops limit reached

The underlying http-client has a `:pending-ops-limit` configuration
which, when reached, results in an exception with the message "Ops
limit reached". As of this writing, aws-api does not provide access to
the http-client's configuration. Programs that encounter "Ops limit
reached" can avoid it by creating separate http-clients for each
aws-client. You may wish to explicitly stop
(`com.cognitect.aws.api/stop`) these aws-clients when the are not
longer in use to conserve resources.

### S3 Issues

#### "Invalid 'Location' header: null"
Expand Down
48 changes: 48 additions & 0 deletions UPGRADE.md
View file
@@ -0,0 +1,48 @@
# Upgrade Notes

## 0.8.430

This release changed the behavior of the following functions:

### com.cognitect.aws.api/client

As of 0.8.430, each aws-api client uses a single shared http-client by
default. Before this release, each aws-client got its own instance of
http-client by default, which caused the number of threads consumed to
increase linearly in relation to the number of aws-clients created.
To reduce resource consumption in the case of many aws-clients, we
recommended that you create a single instance of the http-client and
explicitly share it across all aws-clients. This is no longer
necessary.

### com.cognitect.aws.api/stop

With the introduction of a shared http-client, this function was
updated so that it has no effect when using the shared http-client,
but will continue to call `cognitect.aws.http/stop` on any other
http-client instance.

### effects

These changes have the following effects:

Programs that were creating multiple aws-clients without supplying
an http-client, and without ever calling stop, will see a reduction
in resource consumption.

Programs that were creating an instance of
`cognitect.aws.client.api/default-http-client` and sharing it across
aws-clients should see no change. You can, however, safely stop doing
that.

For programs that were using the default aws-client constructor and
calling stop on each aws-client, the shared http-client will not be
shut down. This should have no negative impact on resource consumption,
as there is only one http-client in this case, and its resources are
managed by aws-api.

For programs that were creating multiple aws-clients in order to get
around an ["Ops limit reached"
error](https://github.com/cognitect-labs/aws-api/issues/98), this is a
breaking change. For this case, we recommend, for now, that you supply
a new http-client for each aws-client.
27 changes: 14 additions & 13 deletions deps.edn
View file
Expand Up @@ -2,22 +2,23 @@
;; All rights reserved.

{:paths ["src" "resources"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/core.async {:mvn/version "0.5.527"}
org.clojure/tools.logging {:mvn/version "0.5.0"}
org.clojure/data.json {:mvn/version "0.2.7"}
org.clojure/data.xml {:mvn/version "0.2.0-alpha6"}
com.cognitect/http-client {:mvn/version "0.1.104"}}
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/core.async {:mvn/version "0.5.527"}
org.clojure/tools.logging {:mvn/version "0.5.0"}
org.clojure/data.json {:mvn/version "0.2.7"}
org.clojure/data.xml {:mvn/version "0.2.0-alpha6"}
com.cognitect/http-client {:mvn/version "0.1.104"}}
:aliases {:update-versions {:extra-paths ["build/src"]
:main-opts ["-m" "cognitect.aws.version-updater"]}
:dev {:extra-paths ["dev/src" "dev/resources" "test/src" "test/resources"]
:extra-deps {commons-io/commons-io {:mvn/version "2.6"}
org.clojure/test.check {:mvn/version "0.10.0"}
org.slf4j/slf4j-log4j12 {:mvn/version "1.7.28"}
http-kit {:mvn/version "2.3.0"}}}
:test {:extra-deps {com.cognitect/test-runner {:git/url "https://github.com/cognitect-labs/test-runner.git"
:sha "cb96e80f6f3d3b307c59cbeb49bb0dcb3a2a780b"}}
:main-opts ["-m" "cognitect.test-runner"]}
http-kit {:mvn/version "2.3.0"}
com.cognitect.aws/s3 {:mvn/version "747.2.533.0"}}}
:test {:extra-deps {com.cognitect/test-runner {:git/url "https://github.com/cognitect-labs/test-runner.git"
:sha "cb96e80f6f3d3b307c59cbeb49bb0dcb3a2a780b"}}
:main-opts ["-m" "cognitect.test-runner"]}
:examples {:extra-paths ["examples" "examples/resources" "dev/resources"]
:extra-deps {org.clojure/test.check {:mvn/version "0.10.0"}
org.slf4j/slf4j-log4j12 {:mvn/version "1.7.28"}
Expand All @@ -26,6 +27,6 @@
com.cognitect.aws/ec2 {:mvn/version "746.2.533.0"}
com.cognitect.aws/iam {:mvn/version "746.2.533.0"}
com.cognitect.aws/lambda {:mvn/version "746.2.533.0"}
com.cognitect.aws/s3 {:mvn/version "747.2.533.0"}
com.cognitect.aws/ssm {:mvn/version "747.2.533.0"}
com.cognitect.aws/sts {:mvn/version "747.2.533.0"}}}}}

com.cognitect.aws/ssm {:mvn/version "747.2.533.0"}
com.cognitect.aws/sts {:mvn/version "747.2.533.0"}}}}}
2 changes: 1 addition & 1 deletion examples/assume_role_example.clj
View file
Expand Up @@ -45,7 +45,7 @@
;; make a credentials provider that can assume a role
(defn assumed-role-credentials-provider [role-arn session-name refresh-every-n-seconds]
(let [sts (aws/client {:api :sts})]
(credentials/auto-refreshing-credentials
(credentials/cached-credentials-with-auto-refresh
(reify credentials/CredentialsProvider
(fetch [_]
(when-let [creds (:Credentials
Expand Down
71 changes: 31 additions & 40 deletions src/cognitect/aws/client.clj
View file
Expand Up @@ -7,6 +7,8 @@
[cognitect.aws.http :as http]
[cognitect.aws.util :as util]
[cognitect.aws.interceptors :as interceptors]
[cognitect.aws.endpoint :as endpoint]
[cognitect.aws.region :as region]
[cognitect.aws.credentials :as credentials])
(:import [java.util.concurrent Callable ExecutorService Executors ThreadFactory]))

Expand Down Expand Up @@ -61,50 +63,39 @@
port (assoc :server-port port)
path (assoc :uri path)))

(defn- send-request*
"*Blocking* helper, call only from send-request."
[ch client op-map]
(let [err-meta (atom {})]
(defn send-request
"Send the request to AWS and return a channel which delivers the response."
[client op-map]
(let [{:keys [service http-client region-provider credentials-provider endpoint-provider]}
(-get-info client)
ch (a/promise-chan)
err-meta (atom {})]
(try
(let [{:keys [service region credentials endpoint http-client]} (-get-info client)
http-request (sign-http-request service endpoint
(credentials/fetch credentials)
(-> (build-http-request service op-map)
(with-endpoint endpoint)
(update :body util/->bbuf)
((partial interceptors/modify-http-request service op-map))))]
(swap! err-meta assoc :http-request http-request)
(a/take!
(http/submit http-client http-request)
(fn [response]
(a/put! ch (with-meta
(handle-http-response service op-map response)
{:http-request http-request
:http-response (update response :body util/bbuf->input-stream)}))))
ch)
(a/take!
(region/fetch-async region-provider)
(fn [region]
(a/take!
(credentials/fetch-async credentials-provider)
(fn [creds]
(let [endpoint (endpoint/fetch endpoint-provider region)
http-request (sign-http-request service endpoint
creds
(-> (build-http-request service op-map)
(with-endpoint endpoint)
(update :body util/->bbuf)
((partial interceptors/modify-http-request service op-map))))]
(swap! err-meta assoc :http-request http-request)
(a/take!
(http/submit http-client http-request)
(fn [response]
(a/put! ch (with-meta
(handle-http-response service op-map response)
{:http-request http-request
:http-response (update response :body util/bbuf->input-stream)})))))))))
ch
(catch Throwable t
(a/put! ch (with-meta
{:cognitect.anomalies/category :cognitect.anomalies/fault
::throwable t}
(assoc @err-meta :op-map op-map)))
ch))))

(defonce send-pool
(delay
(let [idx (atom 0)]
(Executors/newFixedThreadPool
4
(reify ThreadFactory
(newThread
[_ runnable]
(doto (.newThread (Executors/defaultThreadFactory) runnable)
(.setName (str "cognitect-aws-send-" (swap! idx inc)))
(.setDaemon true))))))))

(defn send-request
"Send the request to AWS and return a channel which delivers the response."
[client op-map]
(let [ch (a/chan 1)]
(.submit ^ExecutorService @send-pool
^Callable #(send-request* ch client op-map))
ch))
85 changes: 46 additions & 39 deletions src/cognitect/aws/client/api.clj
View file
Expand Up @@ -9,6 +9,7 @@
[cognitect.aws.dynaload :as dynaload]
[cognitect.aws.client :as client]
[cognitect.aws.retry :as retry]
[cognitect.aws.client.shared :as shared]
[cognitect.aws.credentials :as credentials]
[cognitect.aws.endpoint :as endpoint]
[cognitect.aws.http :as http]
Expand All @@ -25,17 +26,18 @@
:api - required, this or api-descriptor required, the name of the api
you want to interact with e.g. :s3, :cloudformation, etc
:http-client - optional, to share http-clients across aws-clients.
See default-http-client.
:region-provider - optional, implementation of aws-clojure.region/RegionProvider
protocol, defaults to cognitect.aws.region/default-region-provider.
Ignored if :region is also provided
:region - optional, the aws region serving the API endpoints you
want to interact with, defaults to region provided by
by the default region provider (see cognitect.aws.region)
by the region-provider
:credentials-provider - optional, implementation of
cognitect.aws.credentials/CredentialsProvider
protocol, defaults to
cognitect.aws.credentials/default-credentials-provider
:region-provider - optional, implementation of aws-clojure.region/RegionProvider
protocol, defaults to cognitect.aws.region/default-region-provider
:http-client - optional, to share http-clients across aws-clients.
See default-http-client.
:endpoint-override - optional, map to override parts of the endpoint. Supported keys:
:protocol - :http or :https
:hostname - string
Expand All @@ -58,44 +60,48 @@
(if the request is retriable?), or nil if it should stop.
Defaults to cognitect.aws.retry/default-backoff.
By default, all clients use shared http-client, credentials-provider, and
region-provider instances which use a small collection of daemon threads.
Alpha. Subject to change."
[{:keys [api region region-provider retriable? backoff credentials-provider endpoint endpoint-override
http-client]
:or {endpoint-override {}}
:as config}]
:or {endpoint-override {}}}]
(when (string? endpoint-override)
(log/warn
(format
"DEPRECATION NOTICE: :endpoint-override string is deprecated.\nUse {:endpoint-override {:hostname \"%s\"}} instead."
endpoint-override)))
(let [service (service/service-description (name api))
http-client (http/resolve-http-client http-client)
region (keyword
(or region
(region/fetch
(or region-provider
(region/default-region-provider http-client)))))]
(let [service (service/service-description (name api))
http-client (if http-client
(http/resolve-http-client http-client)
(shared/http-client))
region-provider (cond region (reify region/RegionProvider (fetch [_] region))
region-provider region-provider
:else (shared/region-provider))
credentials-provider (or credentials-provider (shared/credentials-provider))
endpoint-provider (endpoint/default-endpoint-provider
api
(get-in service [:metadata :endpointPrefix])
endpoint-override)]
(dynaload/load-ns (symbol (str "cognitect.aws.protocols." (get-in service [:metadata :protocol]))))
(client/->Client
(atom {'clojure.core.protocols/datafy (fn [c]
(-> c
client/-get-info
(select-keys [:region :endpoint :service])
(update :endpoint select-keys [:hostname :protocols :signatureVersions])
(update :service select-keys [:metadata])
(assoc :ops (ops c))))})
{:service service
:region region
:endpoint (if-let [ep (endpoint/resolve (keyword (get-in service [:metadata :endpointPrefix]))
(keyword region))]
(merge ep (if (string? endpoint-override)
{:hostname endpoint-override}
endpoint-override))
(throw (ex-info "No known endpoint." {:service api :region region})))
:retriable? (or retriable? retry/default-retriable?)
:backoff (or backoff retry/default-backoff)
:http-client http-client
:credentials (or credentials-provider (credentials/default-credentials-provider http-client))})))
(let [i (client/-get-info c)]
(-> i
(select-keys [:service])
(assoc :region (-> i :region-provider region/fetch)
:endpoint (-> i :endpoint-provider endpoint/fetch))
(update :endpoint select-keys [:hostname :protocols :signatureVersions])
(update :service select-keys [:metadata])
(assoc :ops (ops c)))))})
{:service service
:retriable? (or retriable? retry/default-retriable?)
:backoff (or backoff retry/default-backoff)
:http-client http-client
:endpoint-provider endpoint-provider
:region-provider region-provider
:credentials-provider credentials-provider})))

(defn default-http-client
"Create an http-client to share across multiple aws-api clients."
Expand Down Expand Up @@ -207,13 +213,14 @@
(str "No docs for " (name operation)))))

(defn stop
"Shuts down the underlying http-client, releasing resources.
"Has no effect when the underlying http-client is the shared
instance.
NOTE: if you're sharing an http-client across aws-api clients,
this will shut down the shared client for all aws-api clients
that are using it.
If you explicitly provided any other instance of http-client, stops
it, releasing resources.
Alpha. Subject to change."
[client]
(let [{:keys [http-client credentials]} (client/-get-info client)]
(http/-stop http-client)))
[aws-client]
(let [{:keys [http-client]} (client/-get-info aws-client)]
(when-not (#'shared/shared-http-client? http-client)
(http/stop http-client))))

0 comments on commit f5acd9b

Please sign in to comment.