Use a 1024 byte minimum weight for filter cache entries #8304

Merged
merged 1 commit into from Oct 31, 2014

Projects

None yet

3 participants

@dakrone
Member
dakrone commented Oct 31, 2014

This changes the weighing function for the filter cache to use a
configurable minimum weight for each filter cached. This value defaults
to 1kb and can be configured with the
indices.cache.filter.minimum_entry_weight setting.

This also fixes an issue with the filter cache where the concurrency
level of the cache was exposed as a setting, but not used in cache
construction.

Relates to #8268
Fixes #8249

@jpountz jpountz was assigned by dakrone Oct 31, 2014
@dakrone
Member
dakrone commented Oct 31, 2014

I also manually tested this, where I ran a shell script containing non-existing term filters over the wikipedia corpus, I was able to see:

        "filter_cache" : {
          "memory_size" : "396.2kb",
          "memory_size_in_bytes" : 405760,
          "evictions" : 55018
        }

to show that filter cache entries are being evicted even though the size limit has not been reached due to the minimum size.

@jpountz jpountz commented on the diff Oct 31, 2014
...icsearch/indices/cache/filter/IndicesFilterCache.java
@@ -145,6 +154,7 @@ public void addReaderKeyToClean(Object readerKey) {
public void close() {
closed = true;
cache.invalidateAll();
+ cache.cleanUp();
@jpountz
jpountz Oct 31, 2014 Contributor

Is it needed because of this change or is it unrelated?

@dakrone
dakrone Oct 31, 2014 Member

Not needed, but always good to actually clean the cache when closing, happy to remove it if you'd like

@jpountz
jpountz Oct 31, 2014 Contributor

I was just wondering if there could be bad side-effects such as causing some operations to take longer.

@dakrone
dakrone Oct 31, 2014 Member

There aren't any bad side-effects, without this, if the cache were closed and everything was invalidated, they would not be immediately removed, this just immediately removes them.

@jpountz
Contributor
jpountz commented Oct 31, 2014

LGTM

@dakrone dakrone Use a 1024 byte minimum weight for filter cache entries
This changes the weighing function for the filter cache to use a
configurable minimum weight for each filter cached. This value defaults
to 1kb and can be configured with the
`indices.cache.filter.minimum_entry_weight` setting.

This also fixes an issue with the filter cache where the concurrency
level of the cache was exposed as a setting, but not used in cache
construction.

Relates to #8268
42b6e01
@dakrone dakrone merged commit 42b6e01 into elastic:master Oct 31, 2014
@clintongormley clintongormley changed the title from Use a 1024 byte minimum weight for filter cache entries to Internal: Use a 1024 byte minimum weight for filter cache entries Nov 3, 2014
@dakrone dakrone deleted the dakrone:bound-filter-cache-size branch Nov 11, 2014
@clintongormley clintongormley changed the title from Internal: Use a 1024 byte minimum weight for filter cache entries to Use a 1024 byte minimum weight for filter cache entries Jun 7, 2015
@clintongormley clintongormley added :Cache and removed :Internal labels Jun 7, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment