Improve http.send cache cleaning

[lukyer]

Possibly related: #5300

What is the underlying problem you're trying to solve?

We store a lot (thousands) http.send responses with low TTL (10s). Currently it is unnecessarily hanging in the cache until caching.inter_query_builtin_cache.max_size_bytes is reached. When it is, only minimum items are wiped to get the cache size under the bytes limit and items are wiped only in FIFO manner instead of proactively wiping cache items with respect to the TTL.

Describe the ideal solution

Cache items are wiped and memory is freed not only when the cache is full but also when max-age (and others) is reached.

Describe a "Good Enough" solution

Periodically running goroutine wiping stale items from the cache.

[ashutosh-narkar]

The other optimization we could think about is to start removing items from the cache when it hits a user-defined threshold. For example, when the cache is 80% full, start removing items from it before inserting new ones. By default, the threshold would be 100% (ie. remove items once cache is full).

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[lukyer]

this is still valid issue, we are waiting for fix 👍

[anderseknert]

@lukyer that an issue becomes inactive does not mean it is invalid, only that there are no immediate plans to work on it — unless someone steps up to do that, of course :)

[lukyer]

any updates? I believe this should be enough to fix our memory leaks here: #5381

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[rudrakhp]

@ashutosh-narkar I believe this issue is still open? Do correct me as I could not find a related option in the documentation yet. We see a need for this as well.
IMO user should have two options to configure, the cache eviction routine will trigger if any of these events occur:

• caching.inter_query_builtin_cache.eviction_threshold_size_percentage: X% of max_size_bytes (only valid if max_size_bytes is set since it's optional, 100% by default)
• caching.inter_query_builtin_cache.eviction_period_seconds: X seconds since last eviction (always valid, 0 by default so never triggers)

The cache eviction routine should clean "all stale entries" and then some in FIFO fashion if required to bring it below X% in each run.
Please let me know if it sounds good and I will take a shot at contributing here. Thanks 🙂

[ashutosh-narkar]

@rudrakhp feel free to look into this. What has been laid out here seems reasonable.

[rudrakhp]

@ashutosh-narkar raised a initial draft of PR for this issue, please do review when you get a chance. Thanks 🙇

[rudrakhp]

@ashutosh-narkar bumping this up. Please do review the updated PR when you have time. Thanks!

[ashutosh-narkar]

@rudrakhp I'm currently out but I'll tag other maintainers to review in the meantime. Thanks for your patience.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.