Wire JDK HttpClient with virtual-thread executor when spring.threads.virtual.enabled=true#4155
Closed
venkatapgummadi wants to merge 1 commit intospring-cloud:mainfrom
Closed
Conversation
…virtual.enabled=true When spring.threads.virtual.enabled=true is set, Spring Boot switches the inbound Tomcat handlers to virtual threads, but Spring Cloud Gateway MVC's outbound JDK HttpClient still falls back to Executors.newCachedThreadPool(), spawning ~145 platform threads and capping proxy throughput at roughly 25% of what virtual threads can sustain (2.3K -> 9.8K req/s, P99 439ms -> 152ms in the reporter's benchmark). This change registers a ClientHttpRequestFactory bean that explicitly wires the JDK HttpClient with a VirtualThreadTaskExecutor so outbound proxy calls run on virtual threads end-to-end. Fixes spring-cloudgh-4150 Signed-off-by: Venkata Pavan Kumar Gummadi <venkata.p.gummadi@ieee.org>
Member
|
While I appreciate the enthusiasm, the open issue is still marked for triage. Conversation will happen on the issue before any time is taken here. |
Author
Apologies for jumping ahead of triage on this I'll move the discussion to #4150 with a short summary of the approach I prototyped, and happy to hold any further work until the issue has been triaged and an approach is agreed upon. Let me know if you'd prefer I close this PR in the meantime; happy to do so. |
Member
|
Happy to keep it in waiting for triage. I commented on the other issue. |
Member
|
Based on Phil's comment, I'm going to close this for now #4150 (comment) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4150.
Problem
When
spring.threads.virtual.enabled=trueis set, Spring Boot switches the inbound Tomcat handlers to virtual threads, but Spring Cloud Gateway MVC's outbound JDKHttpClientstill falls back toExecutors.newCachedThreadPool()— spawning ~145 platform threads namedHttpClient-N-Worker-Mand capping proxy throughput at roughly 25% of what the virtual-thread front end can sustain.The reporter's benchmark on the issue:
Fix
Registers a
ClientHttpRequestFactorybean inGatewayServerMvcAutoConfigurationthat builds aJdkClientHttpRequestFactorywhose underlyingjava.net.http.HttpClientis wired with aVirtualThreadTaskExecutor, so outbound proxy calls run on virtual threads end-to-end.The bean is conditional on:
spring.threads.virtual.enabled=truejava.net.http.HttpClienton the classpathspring.http.clients.imperative.factory=jdk, or auto-detected when no Apache / Jetty / Reactor Netty is on the classpathClientHttpRequestFactory(@ConditionalOnMissingBean)Tests
Four new unit tests in
GatewayServerMvcAutoConfigurationTestscovering each conditional path:VirtualThreadTaskExecutorwhen virtual threads are enabledspring.http.clients.imperative.factory=simpleClientHttpRequestFactoryThe Java-21-specific tests are gated with
@EnabledForJreRange(min = JRE.JAVA_21). All 9 tests pass locally on JDK 21.