Recommended gc settings?

[kurtome]

Hello,

I'm trying to build a basic distributed cache (mostly so in-memory data survives to the next node during rolling deployments in the cluster), and I'm confused about the gc_interval setting. I'm seeing in your getting started guide examples with a gc_interval of 1 day, is that normal? It seems very very long to me, but perhaps I'm way off base. The data I plan to cache is going to have a ttl of 1 hour, and maybe updated repeatedly for a couple of hours before it goes stale.

Generally, it would be super nice if if there were a little more "this is the config to use if you don't know what you're doing" type of documentation. Overall though, the docs have been fantastic, so thank you for all the detailed info!

[cabol]

Hey @kurtome !!

First of all, thanks for the positive feedback, and sure, I will try to improve the docs a little bit more according to your suggestions.

On the other hand, each scenario is different, I'd say there is no as a general rule about the what the config might be, but you're right, perhaps I can give you some advice:

1. If you are expecting high intensive usage of the cache, perhaps you may start setting up the :gc_interval to 1 hr. On the other hand, if you are not sure, you can make it a bit longer, like every 4 / 8 hrs, and try to monitor it, so you can have a better idea about how to tweak it in the best way.
2. If you are setting the ttl: option explicitly to the keys, that will help as well, perhaps, in that case, you can afford to manage a longer :gc_interval, but at the same time you may want to control the cache memory consumption, and this is where the other config parameters come in ( : max_size, : allocated_memory, : gc_cleanup_min_timeout, : gc_cleanup_max_timeout). For example, to start, you can may set the :gc_interval to 4 hrs, handing a ttl: of 1 hr and additionally protecting the memory growth by setting up max_size: 200_000 (200K entries), allocated_memory: 2_000_000_000 (2 GB), gc_cleanup_min_timeout: 10_000 (min timeout for running the GC), and finally gc_cleanup_max_timeout: 600_000 (max timeout, when we have 0 entries, 10 min is a good starting value) – this work with an inverse linear backoff, means, the polling timeout for running the GC is inverse proportional to the memory growth.
3. Try to measure, and once you have some measurements or data, you will be able to define more accurate values not only for :gc_interval but also for : max_size, : allocated_memory, : gc_cleanup_min_timeout, : gc_cleanup_max_timeout; we may want to control also the memory growth.

I will include these tips in the docs anyway, and also I will change the examples using these config params with the recommended values.

Thanks again, and I hope this info may help, otherwise just let me know, I'm glad to help!

[kurtome]

Thanks so much, this is very helpful!

The only thing I'm confused about now is: If I'm always setting the :ttl option when adding to the cache, how do those values interact with the gc_interval:? For example, if I add a brand new value to the cache with ttl: Nebulex.Expire.Time(1, :hour), and then gc runs 1 minute later, what happens? Also, do expired :ttl values get removed from the cache regardless of gc? or is gc part of what makes :ttl work?

I'm familiar with the concept of how garbage collection generations work generally, but it's a bit confusing how the generations interact with all the other options. Which values survive to the next generation and is a new generation required for expiration, etc.

I think I have all the info I need for my project since the memory requirements are pretty relaxed, but mostly I'm just curious now :)

[cabol]

That a good question, normally the ttl: should be lower than :gc_timeout precisely to avoid any unexpected behavior, like the key was expired and the ttl: hasn't happened yet. Assuming you have gc_timeout set to 2 hrs, and you are managing ttl of 1 hr, if something like you describe happens, if I add a brand new value to the cache with ttl: Nebulex.Expire.Time(1, :hour), and then gc runs 1 minute later, in this case, a new generation will be placed, so the key will be still available in the older generation, so if at some point is fetched before the ttl expires, you will retrieve it. On the other hand, suppose you set the key with TTL of 1hr, then the GC runs 1 min later, and the key is not requested for the next GC cycle (e.g.: 2 hrs), the key will be evicted.

Something to keep in mind is, the ttl is evaluated on-demand when the key is requested (if you fetch it and the TTL hasn't expired, the value is returned, otherwise, the key is evicted and nil is returned), means if you set a ttl of 1 hr and you never fetch it before that, the key does exist yet in the cache, only when the GC runs is when the keys are evicted.

I hope this has been helpful, otherwise, just let me know, stay tuned! Thanks!

[Ivor]

Thanks for the great question and answers.

If I understand this correctly then you can have a cache full of records that have expired but if they are never retrieved (to trigger eviction) or garbage collected they will just remain in the cache?

TTL is then not a way to maintain cache size. It is handy for maintaining the integrity of the cache, but you have to set the GC values in the config to prevent the default behaviour which would be to grow without being cleaned up.

[cabol]

If I understand this correctly then you can have a cache full of records that have expired but if they are never retrieved (to trigger eviction) or garbage collected they will just remain in the cache?

Yes, that is correct, the real eviction mechanism is not the TTL, it is the generational process when the GC runs creating a new generation so that the oldest one is evicted (since the data in the oldest one is the least frequently used). Also, pushing a new generation is also triggered when the configured max size or memory is reached.

TTL is then not a way to maintain cache size. It is handy for maintaining the integrity of the cache, but you have to set the GC values in the config to prevent the default behaviour which would be to grow without being cleaned up.

Exactly, for taking care of the cache size, you have to configure the GC values.

Thanks!

[cabol]

Added section "Eviction configuration" into the local adapter to cover GC/eviction settings, caveats, recommendations, etc.