Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigArrays: reuse pages more agressively #5299

Closed
jpountz opened this issue Feb 28, 2014 · 3 comments
Closed

BigArrays: reuse pages more agressively #5299

jpountz opened this issue Feb 28, 2014 · 3 comments
Assignees
Labels

Comments

@jpountz
Copy link
Contributor

jpountz commented Feb 28, 2014

Current implementation starts using paged arrays when the size of the array is
greater than the size of a page. This means that if an array has a length of
${page_size}+1, it will use two pages, wasting about half of the memory.

I think this behavior should be symetric when the size is less than the page
size but higher than half of the page size. This means that we would start
reusing memory at ${page_size}/2 instead of ${page_size}.

@jpountz jpountz self-assigned this Feb 28, 2014
@nik9000
Copy link
Member

nik9000 commented Feb 28, 2014

Makes sense to me though it might be worth running performance numbers on it. It might be that you should kick in even earlier. I don't have any ideas on how to measure the performance of such a change though.

jpountz added a commit to jpountz/elasticsearch that referenced this issue Feb 28, 2014
Pages are now going to be reused when the size of the array is greater than
${page_size}/2 (as opposed to ${page_size} currently).

Close elastic#5299
@jpountz
Copy link
Contributor Author

jpountz commented Feb 28, 2014

I thought about making it kick in earlier but then I would be afraid of wasting too much memory. However, since we are wasting 50% of space in the worst case anyway, I thought it would be useful to take advantage of it to reuse pages more agressively.

@nik9000
Copy link
Member

nik9000 commented Feb 28, 2014

That's pretty sound logic to me.

jpountz added a commit that referenced this issue Feb 28, 2014
Pages are now going to be reused when the size of the array is greater than
${page_size}/2 (as opposed to ${page_size} currently).

Close #5299
@clintongormley clintongormley added the :Core/Infra/Core Core issues without another label label Jun 7, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants