ARROW-360: C++: Add method to shrink PoolBuffer using realloc #272

Closed
wants to merge 2 commits into
from

Projects

None yet

2 participants

@xhochy
Member
xhochy commented Jan 6, 2017

Added no explicit unit test for this as I want to keep the freedom to
the allocator in the future to advise the PoolBuffer on an acceptable
minimal size. In some cases it might be worth it to occupy a whole
page.

Resizing to a smaller size is tested though, so we already have a unit test ensuring that this code runs smoothly.

xhochy added some commits Jan 6, 2017
@xhochy xhochy ARROW-360: C++: Add method to shrink PoolBuffer using realloc
Added no explicit unit test for this as I want to keep the freedom to
the allocator in the future to advise the PoolBuffer on an acceptable
minimale size. In some cases it might be worth it to occupy a whole
page.
040d3b4
@xhochy xhochy Adjust DCHECK for zero size arrays
Change-Id: I7f3519e4b2f7de3d60d9d8e203dbdc6fdc858fda
f4992ee
@wesm
wesm approved these changes Jan 6, 2017 View changes

+1. We should add application logic to avoid unnecessary reallocations in the event that the size and the capacity are "close enough" to each other.

@xhochy
Member
xhochy commented Jan 7, 2017

Made https://issues.apache.org/jira/browse/ARROW-464 as a follow-up (I have some reading material on my desk I want to dig into before I tackle that task). But shrinking memory with {{jemalloc}} in general is very cheap (this will never touch the data, only remove references to physical pages in the kernel) Only the normal allocator will do allocate-move-free. This is something that we should avoid by introducing some margin of allocation waste.

@asfgit asfgit pushed a commit that closed this pull request Jan 7, 2017
@xhochy @wesm xhochy + wesm ARROW-360: C++: Add method to shrink PoolBuffer using realloc
Added no explicit unit test for this as I want to keep the freedom to
the allocator in the future to advise the PoolBuffer on an acceptable
minimal size. In some cases it might be worth it to occupy a whole
page.

Resizing to a smaller size is tested though, so we already have a unit test ensuring that this code runs smoothly.

Author: Uwe L. Korn <uwelk@xhochy.com>

Closes #272 from xhochy/ARROW-360 and squashes the following commits:

f4992ee [Uwe L. Korn] Adjust DCHECK for zero size arrays
040d3b4 [Uwe L. Korn] ARROW-360: C++: Add method to shrink PoolBuffer using realloc
7d1f1cf
@asfgit asfgit closed this in 7d1f1cf Jan 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment