Skip to content

Fix memory leak in http-relay-server.#760

Merged
ensonic merged 1 commit into
mainfrom
ensonic/relay
Jun 29, 2026
Merged

Fix memory leak in http-relay-server.#760
ensonic merged 1 commit into
mainfrom
ensonic/relay

Conversation

@ensonic

@ensonic ensonic commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

When a request is stopped (e.g., if the user-client times out or closes the connection), StopRelayRequest was simply deleting the request from the internal map. However, it was not closing the associated responseStream and requestStream channels. This caused the goroutines ranging over these channels (started in Server.responseFilter and Server.bidirectionalStream) to block indefinitely, leading to a steady increase in goroutines and memory usage over time.

This change refactores the cleanup logic into a common removeRequest helper that explicitly closes all channels and signals the goroutines to exit before removing the request from the map. It ensures that resources are always reclaimed regardless of whether the request timed out or was explicitly stopped.

In addition this adds a new test to cover this and pprof support (so that we can easily identify similar cases in the future).

When a request is stopped (e.g., if the user-client times out or closes the
connection), StopRelayRequest was simply deleting the request from the internal
map. However, it was not closing the associated responseStream and requestStream
channels. This caused the goroutines ranging over these channels (started in
Server.responseFilter and Server.bidirectionalStream) to block indefinitely,
leading to a steady increase in goroutines and memory usage over time.

This change refactores the cleanup logic into a common removeRequest helper that
explicitly closes all channels and signals the goroutines to exit before
removing the request from the map. It ensures that resources are always
reclaimed regardless of whether the request timed out or was explicitly stopped.

In addition this adds a new test to cover this and pprof support (so that we can
easily identify similar cases in the future).
@ensonic ensonic requested review from drigz and koonpeng June 29, 2026 09:20
@ensonic

ensonic commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

Tested pprof via:

kubectl --context=<myctx> port-forward -n app-prometheus deployment/prometheus-relay-server 7654:8765

@drigz

drigz commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Nice find, thank you!

@ensonic ensonic merged commit 626c1e0 into main Jun 29, 2026
7 checks passed
@ensonic ensonic deleted the ensonic/relay branch June 29, 2026 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants