Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quinn endpoints leak causes increased memory on public peers and eventual refusion to accept new peers #309

Closed
LeeSmet opened this issue Jun 28, 2024 · 1 comment
Assignees
Labels
type_bug Something isn't working

Comments

@LeeSmet
Copy link
Contributor

LeeSmet commented Jun 28, 2024

It seems that sometimes an Endpoint (possibly wrapped in some type) can leak. This increases memory consumption on the node, and eventually, if enough of them leak (65K), no new quic connnections can be created.

@LeeSmet
Copy link
Contributor Author

LeeSmet commented Jul 1, 2024

After more debugging, it turns out the Incoming mentioned in the errors are actually still stored in the internal quinn queue for new connections. The problem is that since accepting a connection then waits for client data, the clients can stall the server by not sending the handshake. Potentially this could be triggered by random udp packets as well (not sure).

To solve this, we need to wrap the quic setup (after the initial accept) in a timeout, and potentially also resolve multiple peers side by side to avoid a client not finishing the handshake and delaying others.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type_bug Something isn't working
Projects
Status: Done
Development

No branches or pull requests

1 participant