Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Duplicate handler name: HttpStreamsClientHandler #1903

Closed
oripwk opened this issue Jun 17, 2020 · 6 comments
Closed

Duplicate handler name: HttpStreamsClientHandler #1903

oripwk opened this issue Jun 17, 2020 · 6 comments
Labels
bug This issue is a bug. closed-for-staleness response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days.

Comments

@oripwk
Copy link

oripwk commented Jun 17, 2020

Describe the bug

Getting Duplicate handler name: HttpStreamsClientHandler with 16 concurrent requests when using S3 client putObject

Expected Behavior

This exception should not be thrown

Current Behavior

Here's the stacktrace

2020-06-17 19:07:18
java.lang.Exception: Could not complete the stream element: Record @ 1592397283983 : (SessionMetadata(Some(e70b8d182a834550873c65359571059d),Some(0),f1fc416f-e0a8-4e03-b497-1c4f4724052b),SessionDto(Map(dom-event-count -> 0, url-event-count -> 0, event-count -> 1),[B@5bff10c3,[B@1efd85d8,[B@744daf57)).
	at org.apache.flink.streaming.api.operators.async.AsyncWaitOperator$ResultHandler.completeExceptionally(AsyncWaitOperator.java:362)
	at org.apache.flink.streaming.api.scala.async.JavaResultFutureWrapper.completeExceptionally(JavaResultFutureWrapper.scala:42)
	at walkme.flink.FilesSessionSink$$anonfun$asyncInvoke$1.apply(FilesSessionSink.scala:22)
	at walkme.flink.FilesSessionSink$$anonfun$asyncInvoke$1.apply(FilesSessionSink.scala:20)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
	at org.apache.flink.runtime.concurrent.DirectExecutorService.execute(DirectExecutorService.java:211)
	at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:109)
	at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
	at scala.concurrent.impl.Promise$DefaultPromise.link(Promise.scala:308)
	at scala.concurrent.impl.Promise$DefaultPromise.linkRootOf(Promise.scala:293)
	at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:255)
	at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:251)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
	at org.apache.flink.runtime.concurrent.DirectExecutorService.execute(DirectExecutorService.java:211)
	at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:109)
	at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
	at scala.concurrent.Promise$class.complete(Promise.scala:55)
	at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:157)
	at scala.concurrent.java8.FuturesConvertersImpl$P.accept(FutureConvertersImpl.scala:99)
	at scala.concurrent.java8.FuturesConvertersImpl$P.accept(FutureConvertersImpl.scala:97)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallTimeoutTrackingStage.lambda$execute$2(AsyncApiCallTimeoutTrackingStage.java:69)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:138)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$1(MakeAsyncHttpRequestStage.java:167)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.CompletionException: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Duplicate handler name: HttpStreamsClientHandler#0-body-subscriber
	at software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException(CompletableFutureUtils.java:60)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0(AsyncExecutionFailureExceptionReportingStage.java:51)
	at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)
	at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:74)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:85)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:144)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:125)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage$WrappedErrorForwardingResponseHandler.onError(MakeAsyncHttpRequestStage.java:122)
	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor.handleFailure(NettyRequestExecutor.java:269)
	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor.lambda$writeRequest$7(NettyRequestExecutor.java:223)
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
	at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
	at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:608)
	at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
	at io.netty.util.internal.PromiseNotificationUtil.tryFailure(PromiseNotificationUtil.java:64)
	at io.netty.channel.AbstractChannelHandlerContext.notifyOutboundHandlerException(AbstractChannelHandlerContext.java:812)
	at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
	at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:762)
	at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1089)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	... 1 more
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Duplicate handler name: HttpStreamsClientHandler#0-body-subscriber
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:199)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:143)
	... 25 more
Caused by: java.lang.IllegalArgumentException: Duplicate handler name: HttpStreamsClientHandler#0-body-subscriber
	at io.netty.channel.DefaultChannelPipeline.checkDuplicateName(DefaultChannelPipeline.java:1055)
	at io.netty.channel.DefaultChannelPipeline.filterName(DefaultChannelPipeline.java:284)
	at io.netty.channel.DefaultChannelPipeline.addAfter(DefaultChannelPipeline.java:301)
	at io.netty.channel.DefaultChannelPipeline.addAfter(DefaultChannelPipeline.java:290)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.unbufferedWrite(HttpStreamsHandler.java:321)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.flushNext(HttpStreamsHandler.java:368)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.write(HttpStreamsHandler.java:261)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsClientHandler.write(HttpStreamsClientHandler.java:59)
	at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:715)
	... 8 more

Steps to Reproduce

Create 16 threads where each thread does 3 concurrent putObject requests.

Possible Solution

No possible solution.

Context

It's an Apache Flink sink that's writing to S3 finished sessions

Your Environment

  • AWS Java SDK version used: 3.13.33
  • JDK version used: 1.8
  • Operating System and version: Amazon Linux
@oripwk oripwk added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jun 17, 2020
@debora-ito
Copy link
Member

@oripwk can you reproduce the issue consistently? Can you provide a repro case?

@debora-ito debora-ito added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. and removed needs-triage This issue or PR still needs to be triaged. labels Jun 30, 2020
@github-actions
Copy link

It looks like this issue hasn’t been active in longer than a week. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.

@github-actions github-actions bot added closing-soon This issue will close in 4 days unless further comments are made. closed-for-staleness and removed closing-soon This issue will close in 4 days unless further comments are made. labels Jun 30, 2020
@github-actions github-actions bot closed this as completed Jul 4, 2020
@nastra
Copy link

nastra commented Sep 29, 2020

I hit the same issue as can be seen in the below stack trace:

2020-09-28T14:06:06.5122180Z [writer] STDOUT: Caused by: java.util.concurrent.ExecutionException: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Duplicate handler name: HttpStreamsClientHandler#0-body-subscriber
2020-09-28T14:06:06.5161989Z [writer] STDOUT: 	at java.base/java.util.concurrent.CompletableFuture.reportGet(Unknown Source)
2020-09-28T14:06:06.5163529Z [writer] STDOUT: 	at java.base/java.util.concurrent.CompletableFuture.get(Unknown Source)
2020-09-28T14:06:06.5165353Z [writer] STDOUT: 	at org.apache.cassandra.io.storage.S3Commands.upload(S3Commands.java:160)
2020-09-28T14:06:06.5179096Z [writer] STDOUT: 	... 26 common frames omitted
2020-09-28T14:06:06.5191865Z [writer] STDOUT: Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Duplicate handler name: HttpStreamsClientHandler#0-body-subscriber
2020-09-28T14:06:06.5197683Z [writer] STDOUT: 	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
2020-09-28T14:06:06.5202300Z [writer] STDOUT: 	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
2020-09-28T14:06:06.5251092Z [writer] STDOUT: 	at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:199)
2020-09-28T14:06:06.5259572Z [writer] STDOUT: 	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:143)
2020-09-28T14:06:06.5268043Z [writer] STDOUT: 	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:125)
2020-09-28T14:06:06.5271642Z [writer] STDOUT: 	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source)
2020-09-28T14:06:06.5280027Z [writer] STDOUT: 	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source)
2020-09-28T14:06:06.5284980Z [writer] STDOUT: 	at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
2020-09-28T14:06:06.5297303Z [writer] STDOUT: 	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source)
2020-09-28T14:06:06.5310574Z [writer] STDOUT: 	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage$WrappedErrorForwardingResponseHandler.onError(MakeAsyncHttpRequestStage.java:122)
2020-09-28T14:06:06.5333165Z [writer] STDOUT: 	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor.handleFailure(NettyRequestExecutor.java:269)
2020-09-28T14:06:06.5373335Z [writer] STDOUT: 	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor.lambda$writeRequest$7(NettyRequestExecutor.java:223)
2020-09-28T14:06:06.5384060Z [writer] STDOUT: 	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:502)
2020-09-28T14:06:06.5393842Z [writer] STDOUT: 	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:476)
2020-09-28T14:06:06.5403632Z [writer] STDOUT: 	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:415)
2020-09-28T14:06:06.5420550Z [writer] STDOUT: 	at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:540)
2020-09-28T14:06:06.5482827Z [writer] STDOUT: 	at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:533)
2020-09-28T14:06:06.5492093Z [writer] STDOUT: 	at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:114)
2020-09-28T14:06:06.5502588Z [writer] STDOUT: 	at io.netty.util.internal.PromiseNotificationUtil.tryFailure(PromiseNotificationUtil.java:64)
2020-09-28T14:06:06.5515301Z [writer] STDOUT: 	at io.netty.channel.AbstractChannelHandlerContext.notifyOutboundHandlerException(AbstractChannelHandlerContext.java:807)
2020-09-28T14:06:06.5527837Z [writer] STDOUT: 	at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:708)
2020-09-28T14:06:06.5539445Z [writer] STDOUT: 	at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:698)
2020-09-28T14:06:06.5552732Z [writer] STDOUT: 	at io.netty.channel.AbstractChannelHandlerContext.access$1700(AbstractChannelHandlerContext.java:38)
2020-09-28T14:06:06.5579932Z [writer] STDOUT: 	at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1091)
2020-09-28T14:06:06.5591573Z [writer] STDOUT: 	at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1138)
2020-09-28T14:06:06.5605018Z [writer] STDOUT: 	at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1062)
2020-09-28T14:06:06.5624474Z [writer] STDOUT: 	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
2020-09-28T14:06:06.5642860Z [writer] STDOUT: 	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
2020-09-28T14:06:06.5659141Z [writer] STDOUT: 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:495)
2020-09-28T14:06:06.5683323Z [writer] STDOUT: 	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905)
2020-09-28T14:06:06.5690261Z [writer] STDOUT: 	at java.base/java.lang.Thread.run(Unknown Source)
2020-09-28T14:06:06.5722992Z [writer] STDOUT: Caused by: java.lang.IllegalArgumentException: Duplicate handler name: HttpStreamsClientHandler#0-body-subscriber
2020-09-28T14:06:06.5734400Z [writer] STDOUT: 	at io.netty.channel.DefaultChannelPipeline.checkDuplicateName(DefaultChannelPipeline.java:1066)
2020-09-28T14:06:06.5744455Z [writer] STDOUT: 	at io.netty.channel.DefaultChannelPipeline.filterName(DefaultChannelPipeline.java:284)
2020-09-28T14:06:06.5754272Z [writer] STDOUT: 	at io.netty.channel.DefaultChannelPipeline.addAfter(DefaultChannelPipeline.java:301)
2020-09-28T14:06:06.5776073Z [writer] STDOUT: 	at io.netty.channel.DefaultChannelPipeline.addAfter(DefaultChannelPipeline.java:290)
2020-09-28T14:06:06.5788272Z [writer] STDOUT: 	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.unbufferedWrite(HttpStreamsHandler.java:321)
2020-09-28T14:06:06.5815829Z [writer] STDOUT: 	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.flushNext(HttpStreamsHandler.java:368)
2020-09-28T14:06:06.5846113Z [writer] STDOUT: 	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.write(HttpStreamsHandler.java:261)
2020-09-28T14:06:06.5850359Z [writer] STDOUT: 	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsClientHandler.write(HttpStreamsClientHandler.java:59)
2020-09-28T14:06:06.5853877Z [writer] STDOUT: 	at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:706)
2020-09-28T14:06:06.5856788Z [writer] STDOUT: 	... 10 common frames omitted

I wonder if it could be an async race condition? Unfortunately I cannot reliable reproduce it.

@ttj4
Copy link

ttj4 commented Mar 10, 2021

I can also confirm this is occuring again:

software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Duplicate handler name: HttpStreamsClientHandler#0-body-subscriber
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:205)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:201)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:143)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:125)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:74)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage$WrappedErrorForwardingResponseHandler.onError(MakeAsyncHttpRequestStage.java:126)
	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor.handleFailure(NettyRequestExecutor.java:299)
	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor.lambda$writeRequest$8(NettyRequestExecutor.java:253)
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
	at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
	at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:608)
	at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
	at io.netty.util.internal.PromiseNotificationUtil.tryFailure(PromiseNotificationUtil.java:64)
	at io.netty.channel.AbstractChannelHandlerContext.notifyOutboundHandlerException(AbstractChannelHandlerContext.java:812)
	at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
	at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:762)
	at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1089)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Duplicate handler name: HttpStreamsClientHandler#0-body-subscriber
	at io.netty.channel.DefaultChannelPipeline.checkDuplicateName(DefaultChannelPipeline.java:1055)
	at io.netty.channel.DefaultChannelPipeline.filterName(DefaultChannelPipeline.java:284)
	at io.netty.channel.DefaultChannelPipeline.addAfter(DefaultChannelPipeline.java:301)
	at io.netty.channel.DefaultChannelPipeline.addAfter(DefaultChannelPipeline.java:290)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.unbufferedWrite(HttpStreamsHandler.java:321)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.flushNext(HttpStreamsHandler.java:368)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.write(HttpStreamsHandler.java:261)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsClientHandler.write(HttpStreamsClientHandler.java:59)
	at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:715)

@wim82
Copy link

wim82 commented Aug 20, 2021 •
edited

I have these issues as well (exactly the same exception and cause), but can not reproduce it consistently. I have multiple sessions open as well (around 40). The use case is periodically dumping data from a db in files by certain properties.. sometimes 0 bytes for the file, sometimes a few hundred megabytes.

aws-sdk-java-automation added a commit that referenced this issue Jan 19, 2022
@aws-sdk-java-automation
Merge pull request #1903 from aws/staging/076b46f0-e15d-414b-aaaa-62b…
e0a61cd 
…ea2251150

Pull request: release <- staging/076b46f0-e15d-414b-aaaa-62bea2251150
@sorran
Copy link

sorran commented May 24, 2022

still present in 2.17.191.

We have a scenario that reproduces consistently, its doing 16 S3 async put operations with chained futures that trigger a corresponding 16 S3 put acls (on a 0.5vCPU fargate task). It started as soon as we started chaining the async put acl on to the result of the put future.

When changing the S3 clients completion executor to use the same executor that our non blocking futures use the issue stopped.

.asyncConfiguration{b: ClientAsyncConfiguration.Builder => b.advancedOption(SdkAdvancedAsyncClientOption.FUTURE_COMPLETION_EXECUTOR, completionExecutor); ()}

@vttranlina vttranlina mentioned this issue Jul 13, 2023
Closed
@megglos megglos mentioned this issue May 2, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug This issue is a bug. closed-for-staleness response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@nastra @debora-ito @sorran @oripwk @wim82 @ttj4