Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter cache stats may report incorrect values when index is recreated #2609

Closed
imotov opened this issue Feb 1, 2013 · 0 comments
Closed

Comments

@imotov
Copy link
Contributor

imotov commented Feb 1, 2013

To reproduce, start a single node and as soon as node starts run the following script:

# Create and delete index a few times
for i in {1..4}
do
    curl -XDELETE localhost:9200/test
    curl -XPUT localhost:9200/test -d '{
        "settings": {
            "index.number_of_shards": 1,
            "index.number_of_replicas": 0
        }
    }'
    curl -XPUT localhost:9200/test/doc/1 -d '{"foo": "bar"}'
    curl -XPUT localhost:9200/test/doc/2 -d '{"foo": "bar"}'
    curl -XPUT localhost:9200/test/doc/3 -d '{"foo": "bar"}'
    curl -XPOST localhost:9200/test/_refresh
    # Do search to fill filter cache
    curl "localhost:9200/test/doc/_search"
done
# Sleep for indices.cache.filter.clean_interval
# Reduce it to make this repro run faster
echo
echo "Waiting 1 min for cache to be cleaned"
sleep 60
curl -s -XGET 'http://localhost:9200/_nodes/stats?pretty=true'

The result of this script is

...
        "cache" : {
          "filter_count" : -2,
          "filter_evictions" : 3,
          "filter_size" : "-48b",
          "filter_size_in_bytes" : -48,
          "id_cache_size" : "0b",
          "id_cache_size_in_bytes" : 0
        }
...

Analysis:

  • IndicesFilterCache is using index name to dispatch removal notifications.
  • When index closes, it doesn't clean the cache immediately. So, it's possible that the cache for the old index will be cleaned when a new index with the same name is already created.
  • As a result, it's possible for WeightedFilterCache to receive notifications targeted for the previous incarnations of the index with the same name.
  • WeightedFilterCache decrements stats counters without checking if removal notification was sent for this index or for one of its previous incarnations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant