New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid unnecessary container allocations #105
Comments
Example:
|
hmmm When I read this issue I was thinking about reusing containers, which would may help here too. I mean, when roaring decides to automatically optimize a container (like on Personally, I think reusing a container could help here too. Yeah, the algorithm you suggested could probably be better for the intersection case, but in cases where adding/removing integers to the bitmap causes a bunch of containers to be allocated and deallocated it could be interesting to reuse containers directly in the library, internally. So what I suggest is: What do you think? |
@fjorgemota Yes, this is sensible, but one would like to see actual performance gains in realistic scenarios. |
Oh, for sure. Only benchmarks to test these sort of things. And for sure no-allocation (like in the algorithm for intersections) is always better than one allocation here and there using pools. :) |
I have observed high CPU usage caused by the memory allocations when using Bitmap.ReadFrom and bitmap.Or/Clone frequently. By implementing memory reuse, we should be able to effectively resolve the majority of this problem. |
I have submitted a pull request to resolve the issue with ReadFrom #395 |
In many instances, especially when computing intersections, we generate empty containers that are immediately garbage collected.
There are clever tricks we could use to start scanning the input containers, advance up to the point where there might be values to be intersected, and bail out without generating an empty container if the intersection is empty.
The text was updated successfully, but these errors were encountered: