HttpLoggingInterceptor hangs when response uses "Transfer-Encoding: chunked"

[fabiohecht]

Hi, I have a server that responds with "Transfer-Encoding: chunked" and does not close the connection. Without using the HttpLoggingInterceptor with BODY level, execute() returns a partial response with each chunk. As soon as I enable logging with BODY level, the HttpLoggingInterceptor hangs on source.request(Long.MAX_VALUE) // Buffer the entire body.. I did not expect that enabling logging should have an effect on how the client works.

[yschimke]

Yep - this is a limitation of the HttpLoggingInterceptor, we are unlikely to fix this. If you need something more specific you should copy and alter it's implementation to your requirements.

[merlinran]

Alas, spent hours trying to figure out why a pretty normal client.newCall(request).execute() blocks forever when connecting to a Server-Sent Event endpoint (meant to use okhttp-sse).

I'm relatively new to the Java/Kotlin ecosystem so could be wrong, but feels to me it's not particularly hard to replace source.request(Long.MAX_VALUE) with a forwarding source like what this Progress.java recipe shows, but duplicating the bytes instead. Can try playing with it if no objection.

Or at least add a few words in the comment for Level.BODY to remind the existence of the trap.

[JakeWharton]

That means multiple response bodies can now have their parts interleave in the logs. It also means you may split across a codepoint or grapheme cluster making the rendering inaccurate.

[merlinran]

good point. maybe buffer the duplicated bytes to up to 100kB or until the source closes? In practice I don't think someone would rely on logs to verify the response body larger than 100kB. Feels to me it's pretty important to prevent a utility in this library from violating the core assumption - execute() doesn't have to wait for the full response body.

[yschimke]

We don't actively enhance this functionality or make it more configurable. We might be best just disabling it for these types of uses based on content type as its clearly broken if causing delays. I'll follow up with a PR to do that.

To unblock you, we've previously suggested it's worth copy pasting the implementation to make it do exactly what you need.

[merlinran]

Nice solution! This is no longer an issue to me once the cause was found, but it may help others from entering similar situations. Thank you!