Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gRPC Client leaks memory for messages and streams #124

Closed
NIbbuchanan opened this issue Aug 25, 2022 · 5 comments
Closed

gRPC Client leaks memory for messages and streams #124

NIbbuchanan opened this issue Aug 25, 2022 · 5 comments
Labels
type: showstopper Issues that are blocking a client's release and need immediate attention.

Comments

@NIbbuchanan
Copy link
Contributor

NIbbuchanan commented Aug 25, 2022

We have memory leaks with both unary messages and streaming data:

stream leak is proportional to data size. For small strings, I saw 50 bytes/iteration and with a string size of 1KB, the leak went up to 800bytes/iteration.

Unary messages were always leaking about 10 bytes/message regardless of the message size.

I modified this example and called it from two different LV process to verify the leak only happens on the client side for both messages and streams. I also tried with 32 and 64-bit LV 2021 and both have the problem.

I'm attaching a simple example to illustrate the leak
leak test.zip
.

@NIbbuchanan
Copy link
Contributor Author

Turns out I haven't written a LV memory leak test in awhile and I forgot to disconnect a build array on the While loop. When I remove this, the above test doesn't leak. Closing this issue due to user error. Sorry about that!

@NIbbuchanan
Copy link
Contributor Author

NIbbuchanan commented Aug 29, 2022

After further testing, the approach I was using in the VI to monitor memory wasn't catching the leak. If you look at Task Manager, you will see the LabVIEW process memory continuously growing quickly more than 1MB every 10 seconds when sending unary messages. Using streaming data doesn't have this problem.

@NIbbuchanan
Copy link
Contributor Author

I did some more digging into the memory leak for grpc. When I changed the cluster to be two strings instead of a string and byte array it no longer leaks in 32-bit LV (still leaked in 64-bit LV, but I didn't try the 64-bit leak fix yet). Our desired cluster is a string and byte array which leaks in both 32 and 64-bit LV. I think the leak may be in ClusterDataCopier::CopyToCluster. Here's a simple example to test the leak. Run the server test VI first and then the client.
Simple Leak Test.zip

@eyesonvis eyesonvis added the type: showstopper Issues that are blocking a client's release and need immediate attention. label Aug 29, 2022
@NIbbuchanan
Copy link
Contributor Author

I tried with the latest 64-bit leak fix and it this leaks on 64-bit LV with either a cluster of 2 strings or a cluster with a sting and byte array.

@shivaprasad-basavaraj
Copy link
Collaborator

The issue has been fixed in #132 and has been picked up by the 0.4.9.1 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: showstopper Issues that are blocking a client's release and need immediate attention.
Projects
None yet
Development

No branches or pull requests

3 participants