Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server / client should implement some sanity checks to prevent unbounded memory growth #103

Closed
gideonw opened this issue Dec 24, 2022 · 9 comments · Fixed by #107
Closed
Labels
bug Something isn't working
Milestone

Comments

@gideonw
Copy link
Collaborator

gideonw commented Dec 24, 2022

Potential client disconnect memory leak where we need to clean up resources explicitly

@gideonw gideonw added the bug Something isn't working label Dec 24, 2022
@gideonw gideonw added this to the v0.2.0 milestone Dec 24, 2022
@gideonw
Copy link
Collaborator Author

gideonw commented Dec 24, 2022

I actually believe this is due to length being parsed incorrectly and then allocating a massive buffer. We might need to make some sane checks here.

Possibly #99

	length := binary.BigEndian.Uint32(lengthPrefix)
	b := make([]byte, length)
	n, err := io.ReadFull(r, b)

@dburkart
Copy link
Owner

Yeah that would make sense if there was mismatch between server and client (one was little and the other big).

I guess there's one clever thing we could do which is to reject any client that looks to be little endian during version negotiation. The lengths should always be the same (for v1.0.0 anyway), so we can check for a specific length. This would at least make it easier to not get in trouble with mismatched dev versions.

@gideonw
Copy link
Collaborator Author

gideonw commented Dec 24, 2022

I'm also thinking about potential bad actors here too. You're right though, this shouldn't be an issue in the future.

@gideonw
Copy link
Collaborator Author

gideonw commented Dec 24, 2022

@dburkart This also comes about if someone opens a TCP connection and sends data that isn't our proto. Do you think it makes sense to add a magic first byte to the version message or the message in general to immediately disregard malformed requests.

@dburkart dburkart modified the milestones: v0.2.0, v0.1.4 Dec 24, 2022
@dburkart
Copy link
Owner

Moving to 0.1.4 for investigation.

@dburkart
Copy link
Owner

dburkart commented Dec 24, 2022

Which resources are not being cleaned up? I checked and it looks like we clean up the net.TCPConn with a defer, but we don't explicitly nil out any fields on the conn struct. Not sure if that's necessary tho

@gideonw
Copy link
Collaborator Author

gideonw commented Dec 24, 2022

I think that it is more likely the symptoms are from what I mention over chat. The first byte being used for the length of the buffer causing a massive consumption of memory of it isn't the proper format.

@dburkart
Copy link
Owner

Ahh right. Let me take a look at that.

@dburkart
Copy link
Owner

Fixed the main issue here by backporting the big-endian change to release/0.1.x. Will use this issue to track some defensive programming around message sizes and the like.

@dburkart dburkart changed the title Memory leak when clients disconnect Server / client should implement some sanity checks to prevent unbounded memory growth Dec 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants