Session handling error: Pool#acquire(Duration) has been pending for more than the configured timeout of 45000ms

[viral-kondhia]

2022-04-26 18:39:34.868 ERROR 102456 --- [     parallel-2] o.s.g.client.WebSocketGraphQlTransport   : Session handling error: Pool#acquire(Duration) has been pending for more than the configured timeout of 45000ms
2022-04-26 18:39:34.869 ERROR 102456 --- [     parallel-2] reactor.core.publisher.Operators         : Operator called default onErrorDropped
reactor.netty.internal.shaded.reactor.pool.PoolAcquireTimeoutException: Pool#acquire(Duration) has been pending for more than the configured timeout of 45000ms
	at reactor.netty.internal.shaded.reactor.pool.AbstractPool$Borrower.run(AbstractPool.java:415) ~[reactor-netty-core-1.0.17.jar:1.0.17]
	at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) ~[reactor-core-3.4.16.jar:3.4.16]
	at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) ~[reactor-core-3.4.16.jar:3.4.16]
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:831) ~[na:na]

[viral-kondhia]

@spring-projects-issues Looks like connection is not getting closed and returned to pool
Any workarounds or help will be appreciated

[rstoyanchev]

Can you provide more details around the scenario? Is it possible to reproduce?

[viral-kondhia]

Yes its possible to reproduce.

            String endPoint = "ws://localhost:8080" + "/graphql";

	ReactorNettyWebSocketClient client = new ReactorNettyWebSocketClient();

	WebSocketGraphQlClient graphQlClient = WebSocketGraphQlClient.builder(endPoint, client).build();

	Flux<LocationResponse> locationCreateResponse = graphQlClient.documentName("document")
			.variable("number", getLocationRequest.getNumber())
			.variable("transactionid", getLocationRequest.getTransactionId()).retrieveSubscription("getLocation")
			.toEntity(LocationResponse.class);`

Execute above code for 500 times

Basically I have created the REST wrapper for GraphQLClient Microservices

I have below pipeline and two microservices.

Browser-> GraphqlClient<->GraphQLServer

Please refer below for detailed explanation
https://stackoverflow.com/questions/72024820/spring-boot-starter-graphqlsession-handling-error-poolacquireduration-has

I suspect the websocket connection not getting closed which indeed reaches 500 max limit for TCP/UDP connections.

Even I tried putting graphQlClient.stop(); but that also not working, I think there seems to be issue in org.springframework.graphql.client.WebSocketGraphQlTransport.stop() as org.springframework.graphql.client.WebSocketGraphQlTransport.GraphQlSession.close() this is not getting invoked
I tried implementing WebSocketGraphQlTransport.stop() but it am still getting that error and it seems connection not getting closed.

[viral-kondhia]

@rstoyanchev Hi Just checking
any other info required le me know

[rstoyanchev]

@viral-kondhia the GraphQlWebSocketClient provides multiplexing over a single, shared connection, which means you can use it to make many requests over the same connection but you need to use the same client instance. In other words, if you execute the above code 500 times it is expected that it will create 500 connections. We have an issue #368 to improve the docs.

As for why stop doesn't work, I wonder if you've noticed that it returns Mono, which is deferred and that means nothing happens until something subscribes to it.

[viral-kondhia]

Thanks @rstoyanchev

So I tried based on your input good news is stop is working.
But other thing that i tried is making graphqlClient shared for sending multiple request but it's giving error for some requests. FYI I am using below graphQlClient for sending 40 requests per second for 5 minutes.
It works for most of the requests but for some its failing with below error.

private static final ReactorNettyWebSocketClient client = new ReactorNettyWebSocketClient();
private static final WebSocketGraphQlClient graphQlClient = WebSocketGraphQlClient.builder(endPoint, client).build();

Caused by: java.lang.IllegalStateException: Failed to send request: FAIL_NON_SERIALIZED
	at org.springframework.util.Assert.state(Assert.java:76) ~[spring-core-5.3.19.jar:5.3.19]
	at org.springframework.graphql.client.WebSocketGraphQlTransport$GraphQlSession.trySend(WebSocketGraphQlTransport.java:445) ~[spring-graphql-1.0.0-RC1.jar:na]
	at org.springframework.graphql.client.WebSocketGraphQlTransport$GraphQlSession.executeSubscription(WebSocketGraphQlTransport.java:421) ~[spring-graphql-1.0.0-RC1.jar:na]
	at org.springframework.graphql.client.WebSocketGraphQlTransport.lambda$executeSubscription$5(WebSocketGraphQlTransport.java:152) ~[spring-graphql-1.0.0-RC1.jar:na]
	at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:163) ~[reactor-core-3.4.17.jar:3.4.17]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) ~[reactor-core-3.4.17.jar:3.4.17]

[rstoyanchev]

@viral-kondhia the failure to send with "FAIL_NON_SERIALIZED" is a currently limitation of the implementation. I've created a separate issue #388 for that. Please, subscribe to that and continue the conversation there. I'll close this one here.