Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OOM when receiving large streaming requests #3173

Open
davidlar opened this issue Sep 26, 2024 · 1 comment · May be fixed by #3174
Open

OOM when receiving large streaming requests #3173

davidlar opened this issue Sep 26, 2024 · 1 comment · May be fixed by #3174
Labels
bug Something isn't working

Comments

@davidlar
Copy link
Contributor

Describe the bug
We have a service that proxies large streams to different backends. It's compiled with GraalVM native-image and is running in docker with a quite small heap ~80Mb. It has worked fine for a long time but when we updated to zio-http 3.0.0 from 3.0.0-RC9 it started to get OOM instantly.
I did some investigation and the problem is the unbounded queue in AsyncBody.asStream that was introduced. It completely disables back pressure for incoming streams. So if the producer is faster than the consumer, the data will be buffered in the unbounded queue. This could lead to OOM.

To Reproduce
Steps to reproduce the behaviour:

  1. Create a service with request streaming enabled and limit the heap. Make processing of incoming data a bit slow
  2. Create a test case that sends a request larger than the heap of the service.
  3. Watch OOM in the service

Expected behaviour
It should work like it did before (but without blocking netty, of course)

@davidlar davidlar added the bug Something isn't working label Sep 26, 2024
@davidlar davidlar linked a pull request Sep 26, 2024 that will close this issue
@davidlar
Copy link
Contributor Author

If enabling request streaming it only takes one large request to kill the server...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant