Unable to load an HTTP implementation from any provider in the chain. You must declare a dependency on an appropriate HTTP implementation or pass in an SdkHttpClient explicitly to the client builder

[jobetdelima]

Hi,

I'm trying to do a GetObject() call from S3 using an SBT project. When I try to run the project within my IDE, or do "sbt run", things work -- that is, I'm able to read a file from S3, and do what I need to do with it.

However, when I do "sbt assembly" to get an uber jar (which I plan on uploading to an AWS Lambda), I end up with the below exception:

software.amazon.awssdk.core.exception.SdkClientException: Unable to load an HTTP implementation from any provider in the chain. You must declare a dependency on an appropriate HTTP implementation or pass in an SdkHttpClient explicitly to the client builder.

I tried doing something like below, but to no avail. Exact same behaviour as before I tried to explicitly create an HttpClient.

val apacheClient: SdkHttpClient = ApacheSdkHttpClientFactory.builder()
                                                                .build()
                                                                .createHttpClient()

    val client = S3Client.builder()
                          .httpConfiguration(ClientHttpConfiguration.builder()
                                                                    .httpClient(apacheClient)
                                                                    .build())
                         .region(Region.US_EAST_1)
                         .build()
    val stream: ResponseInputStream[GetObjectResponse] = client.getObject(GetObjectRequest.builder()
                                                            .bucket(s3Bucket)
                                                            .key(s3EventKey)
                                                            .build())

the lib dependencies look like this on my build.sbt:

libraryDependencies ++=
  Seq(
    "org.apache.kafka" %% "kafka" % "1.0.0",
    "com.typesafe" % "config" % "1.3.1",
//    "com.fasterxml.jackson.core" % "jackson-core" % "2.9.1",
//    "com.fasterxml.jackson.core" % "jackson-databind" % "2.9.1",
    "org.apache.avro" % "avro" % "1.8.2",
    "io.confluent" % "kafka-avro-serializer" % "3.3.1",
//    "com.typesafe.play" %% "play-json" % "2.6.7",
//    "com.typesafe.akka" %% "akka-http" % "10.0.11",
    //"software.amazon.awssdk" % "dynamodb" % "2.0.0-preview-2",
    "software.amazon.awssdk" % "dynamodb" % "2.0.0-preview-9",
    "software.amazon.awssdk" % "core" % "2.0.0-preview-9",
   // "com.amazonaws" % "aws-java-sdk" % "1.11.297",
    "com.amazonaws" % "aws-lambda-java-core" % "1.2.0",
    "com.amazonaws" % "aws-lambda-java-events" % "2.1.0",
   // "com.amazonaws" % "aws-java-sdk-s3" % "1.11.301",
    "software.amazon.awssdk" % "sts" % "2.0.0-preview-9",
    "software.amazon.awssdk" % "s3" % "2.0.0-preview-9",
    "software.amazon.awssdk" % "aws-http-client-apache" % "2.0.0-preview-1"
  )

Your Environment

• AWS Java SDK version used: 2.0
• JDK version used: 1.8
• Operating System and version: Mac OS High Sierra (although I get the same exception after uploading and running the uber jar on an AWS lambda function).

[dagnir]

I'm not super familiar with sbt but I suspect it might have to do with the fact that the HTTP clients are declared as runtime dependencies of the service client modules:

aws-sdk-java-v2/services/pom.xml

Lines 163 to 174 in 521015e

	<dependency>
	<artifactId>apache-client</artifactId>
	<groupId>software.amazon.awssdk</groupId>
	<version>${awsjavasdk.version}</version>
	<scope>runtime</scope>
	</dependency>
	<dependency>
	<artifactId>netty-nio-client</artifactId>
	<groupId>software.amazon.awssdk</groupId>
	<version>${awsjavasdk.version}</version>
	<scope>runtime</scope>
	</dependency>

Can you check to ensure that the runtime classpath is being included in your assembly?

This looks related to sbt/sbt-assembly#120

[ethul]

I had a similar issue using Proguard.

In order to resolve the problem I had to ensure that the appropriate
META-INF/services/software.amazon.awssdk.http.SdkHttpService
file was being included in the resulting jar.

For example, to use the apache-client, the following file should be present in the jar:

https://github.com/aws/aws-sdk-java-v2/blob/f0b1a54bdcdd7168c676bee7e110becf7d724b31/http-clients/apache-client/src/main/resources/META-INF/services/software.amazon.awssdk.http.SdkHttpService

[oripwk]

@dagnir

I'm experiencing the exact same issue. Checked the final zip file, and it indeed does not include apache-client dependency.

What's the reason behind making it a runtime dependency? The developer cannot know it advance that the code will fail at runtime.

[warrenronsiek]

Solved the problem by including an http client and passing a client builder (as opposed to a HttpConfiguration). E.g.

import software.amazon.awssdk.http.nio.netty.NettyNioAsyncHttpClient

S3AsyncClient.builder
    .credentialsProvider(credentials)
    .region(Region.US_EAST_1)
    .httpClientBuilder(NettyNioAsyncHttpClient.builder()
      .maxConcurrency(10)
      .maxPendingConnectionAcquires(1000))
    .build()

I think this forces sbt-assembly to include the http library during compile time.

[khalil-jibran06]

Checkout below link as well where explicit httpClient setting solves the issue in normal STS role assumption scenario through AWS SDK 2.x.
https://github.com/khalil-jibran06/StsAssumeRoleCredentialsProvider

[yaswanth2216]

I am running into the same issue when invoking Transcribe Streaming Client
TranscribeStreamingAsyncClient client = TranscribeStreamingAsyncClient.builder().credentialsProvider(getCredentials()).region(Region.US_WEST_2).build(); CompletableFuture<Void> result = client.startStreamTranscription(getRequest(), new com.amazon.alexa.aws.TranscribeStreaming.AudioStreamPublisher(getStreamFromMic()), getResponseHandler()); result.get(); client.close();

software.amazon.awssdk.core.exception.SdkClientException: Unable to load an HTTP implementation from any provider in thechain. You must declare a dependency on an appropriate HTTPimplementation or pass in an SdkHttpClient explicitly to theclient builder.