Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Go] [Question]: Retain/Release benchmarks vs Go garbage collection #35232

Closed
yevgenypats opened this issue Apr 19, 2023 · 2 comments
Closed

Comments

@yevgenypats
Copy link
Contributor

Describe the enhancement requested

Hi! Im curious if a benchmark was ever done around the value of Retain/Release in the Go library vs not having it at all and getting the garbage collection taking care of that. Maybe we should have a mode where Retain/Release is not necessary? Or there is other reason Im not aware of?

cc @zeroshade

Component(s)

Go

@zeroshade
Copy link
Member

To my knowledge I don't believe a benchmark has actually been done around it as those predate the original donation of the code from InfluxDB to the repository.

In theory, if you know you're using the default Go allocator, you don't actually have to call Retain/Release in your consumer code and can just let the Go garbage collector manage everything. We just need to make sure that we're calling Retain/Release internally here in the library so that if a consumer is using a non-Go-GC allocator we can be sure that they won't end up with memory leaks.

The primary reasons for their existence currently are:

  • Allowing the user to control when buffers and internals are set to nil to be marked for garbage collection early if possible
  • Allowing for proper usage of allocators that don't rely on the Go garbage collector (such as the mallocator impl which uses malloc/free to manage C memory instead of Go garbage collection) if a user desires.
  • While a finalizer could be utilized via SetFinalizer the general advice is that it's better to have an explicit release rather than relying on a finalizer since there's no actual guarantee that finalizers will ever run. In addition, finalizers only run during GC, which means that if your allocator is allocating C memory or something and the Go memory stays low it's possible that you'll end up with a ton of C memory allocated blowing out your memory before any finalizers run to actually call free on things.

@yevgenypats
Copy link
Contributor Author

Got it. Ok that makes tons of sense. Thanks for the explanation. Going to close it now.

kodiakhq bot pushed a commit to cloudquery/plugin-sdk that referenced this issue Apr 20, 2023
Following this short discussion - apache/arrow#35232

It seems we don't really need to use Retain/Release outside the arrow library. We can always bring this back in the future if we would like to experiment if this brings better memory performance then the default go allocator.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants