Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Leak memory when creating a consumer name with jetstream. #4289

Closed
VuongUranus opened this issue Jul 6, 2023 · 10 comments · Fixed by #4329
Closed

Leak memory when creating a consumer name with jetstream. #4289

VuongUranus opened this issue Jul 6, 2023 · 10 comments · Fixed by #4329

Comments

@VuongUranus
Copy link

I have problem leaking mem in sync.(*Map).Store in function github.com/nats-io/nats-server/v2/server.getHashSize.
I debug using go tool pprof. Help me please!

@VuongUranus
Copy link
Author

Screenshot 2023-07-06 at 15 39 35

@derekcollison
Copy link
Member

As a system user what does nats server ls show? How about nats server report jetstream?

@derekcollison
Copy link
Member

How many JetStream assets do you have running?

@VuongUranus
Copy link
Author

I build a cluster of 3 jetstream nodes to serve to create a lot of ephemeral consumers to receive messages. The number of consumers is more than 10000. And I have a memory leak. The problem is that sync.Map stores the names of the consumers and doesn't release. I use the Go programming language and the nats.go library to do this. Don't know what I did wrong?

@derekcollison
Copy link
Member

So you create 10k consumers, then go back to 0 and want the memory to be returned to the kernel?

Or do you think we are not cleaning up an internal client when we create consumers?

@VuongUranus
Copy link
Author

VuongUranus commented Jul 19, 2023

I want memory to be freed when consumers are decremented. I see consumers are released very normally, but the problem is that sync.Map in this your code is not freed, leading to memory leak.
image

@VuongUranus
Copy link
Author

image

@derekcollison
Copy link
Member

Thanks will take a look, this is helpful. Much appreciated.

derekcollison added a commit that referenced this issue Jul 20, 2023
If we created lots of hashes, beyond server names, like for consumer or
stream NRG group names etc, these maps would grow and not release
memory. Performance hit is ~300ns per call, and we can use string intern
trick if need be at a future date since it is GC friendly.

Signed-off-by: Derek Collison <derek@nats.io>

Resolves #4289
@derekcollison
Copy link
Member

Should be resolved and will be in next release, 2.9.21 probably next week.

@VuongUranus
Copy link
Author

Thank you very much!!

@bruth bruth removed the 🐞 bug label Aug 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants