Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large overhead from expires times when highly populated #8

Closed
kpdecker opened this issue Aug 15, 2014 · 3 comments
Closed

Large overhead from expires times when highly populated #8

kpdecker opened this issue Aug 15, 2014 · 3 comments
Assignees
Milestone

Comments

@kpdecker
Copy link
Contributor

@kpdecker kpdecker commented Aug 15, 2014

Commonly seeing 20-40k timer instances due to a timer being created for each cache entry. Outside of the CPU overhead that this might incur (I don't know the node internals, other than that there is some sort of linked list going on), this does create a lot of memory overhead.

In one particular dump seeing 30megs+ of retained data for these timers, domains, protects, etc that are all associated with each timer instance.

Under our use pattern these timers commonly trigger at very similar times and it seems like with some smart bucketing and a prune setInterval or similar much of this overhead could be dramatically reduced.

@hueniverse hueniverse added the request label Aug 15, 2014
@hueniverse

This comment has been minimized.

Copy link
Member

@hueniverse hueniverse commented Nov 19, 2014

There are a few ways to solve this. One is to set a threshold, anything under gets a timer, anything above gets collected into a single interval that cleans things up. Catbox itself will never serve expired data so this is more about memory management.

@ldesplat ldesplat mentioned this issue Jun 19, 2015
@hueniverse

This comment has been minimized.

Copy link
Member

@hueniverse hueniverse commented Aug 26, 2016

This combines with #36 should be resolved by replacing all the individual timeouts with a single interval that checks what expired and cleans it up. When you ask for expired data, clean it. The interval should be configurable.

@hueniverse

This comment has been minimized.

Copy link
Member

@hueniverse hueniverse commented Oct 30, 2018

@geek @kanongil @cjihrig would love a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.