Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The onstatus method of webclient causes a memory leak. [SPR-17473] #22005

Closed
spring-issuemaster opened this issue Nov 7, 2018 · 3 comments
Closed

Comments

@spring-issuemaster
Copy link
Collaborator

@spring-issuemaster spring-issuemaster commented Nov 7, 2018

KimSeongIl opened SPR-17473 and commented

 

private WebClient webClient;

    private final static int DEFAULT_CONNECT_TIMEOUT = 900;
    private final static int DEFAULT_SOCKET_TIMEOUT = 900;

    @PostConstruct
    public void initialize() {
        ReactorClientHttpConnector connector = new ReactorClientHttpConnector(options ->
                options.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, DEFAULT_CONNECT_TIMEOUT)
                        .poolResources(PoolResources.fixed("httpPool", 1000))
                        .afterNettyContextInit(ctx -> {
                            ctx.addHandlerLast(new ReadTimeoutHandler(DEFAULT_SOCKET_TIMEOUT, TimeUnit.MILLISECONDS));
                        }));

        webClient = WebClient.builder()
                .clientConnector(connector)
                .build();

    }

@GetMapping("/test")
    public Mono<String> webclientTest() {

        return webClient
                .get()
                .uri("http://testurl.com/response500test")
                .retrieve()
                .onStatus(httpStatus -> httpStatus != HttpStatus.OK, clientResponse -> Mono.error(new RuntimeException(String.format("api status: %s", clientResponse.statusCode()))))
                .bodyToMono(String.class);
    }

The above source code called an API that responded to 500 errors with 600 tps.

 

io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 8573157383, max: 8589934592)

The onstatus method of webclient causes a memory leak.

 

!스크린샷 2018-11-07 오후 2.01.06.png!

 

In the photo attached above
We used the onStatus method from 13:40 to 13:48,
We removed the onStatus method from 13:55 to 14:00.
Is this a bug?
Why does the onStatus method cause a memory leak?

I used SpringBoot version 2.0.4 and upgraded to 2.1.0, but I still have a problem.


Affects: 5.0.8, 5.1.2

Attachments:

Issue Links:

  • #22096 WebClient logs "Only one connection receive subscriber allowed" when response status is an error

Referenced from: commits f73a522, c187cb2

Backported to: 5.0.11

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 7, 2018

Rossen Stoyanchev commented

Technically, you need to always consume the body of the response, or call bodyToMono(Void.class) if you don't expect or don't care for any content that may be in the response. This is explained in the Javadoc of ClientResponse and the exchange() method, and is also mentioned in the reference. It is not explicitly mentioned on the retrieve() method because then you have to choose a bodyToXxx method, but that's not the case if using the onStatus hook as you have found out.

In short technically this is not a bug, since the application is generally expected to consume the body. However unlike the case with exchange() where the application gets the response and the framework does not have a good place to ensure the response is consumed or released, in the case of the onStatus hook, we can do that after the error function is applied. So I'm turning this into an improvement. Meanwhile please use clientResponse.bodyToMono(Void.class).then(Mono.error(new RuntimeException(...))).

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 8, 2018

KimSeongIl commented

Rossen Stoyanchev

Thank you for the reply.

there was no memory leak after the code was modified.

i expect version 5.1.3 :)

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 9, 2018

Rossen Stoyanchev commented

This should be now fixed if you want to give it a try to 5.1.3 snapshots once the current build completes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.