Warmers do not load field data cache for sorting on new segments #2649

bleskes · 2013-02-13T15:57:55Z

Hi,

One the use cases for warmers is to pre-load the fields data cache with fields used for sorting.

The documentation doesn't specify how to load a field for sorting, but the following loads the id field into the cache if you put a new warmer after data has already been indexed:

PUT to /tests/_warmer/warmer1

{
    "query": {
        "match_all": {}
    },
    "sort": [
        { "id": { "order" : "desc" }}
    ]

}

However, new segments will not have the id field loaded.

To reproduce follow this sequence (on a fresh node). I tried it on 0.20.4:
Create an empty tests index with the folldwing mapping:

{
   "test" : {
       "properties" : {
           "id" : { "type": "string" }
       }
   }
}

Load the warmer: PUT to /tests/_warmer/warmer1

{
    "query": {
        "match_all": {}
    },
    "sort": [
        { "id": { "order" : "desc" }}
    ]

}

Load two documents with the following structure (different ids): PUT to /tests/test/2

{
    "id": "2"
}

Call refresh: GET to _refresh
Get the nodes stats. Cache is empty: GET to /_cluster/nodes/stats

            "cache": {
               "field_evictions": 0,
               "field_size": "0b",
               "field_size_in_bytes": 0,
               "filter_count": 0,
               "filter_evictions": 0,
               "filter_size": "0b",
               "filter_size_in_bytes": 0,
               "bloom_size": "0b",
               "bloom_size_in_bytes": 0,
               "id_cache_size": "0b",
               "id_cache_size_in_bytes": 0
            },

Put the warmer in again: PUT to /tests/_warmer/warmer1

{
    "query": {
        "match_all": {}
    },
    "sort": [
        { "id": { "order" : "desc" }}
    ]

}

Now the cache will be loaded: GET to /_cluster/nodes/stats :

"cache": {
               "field_evictions": 0,
               "field_size": "132b",
               "field_size_in_bytes": 132,
               "filter_count": 0,
               "filter_evictions": 0,
               "filter_size": "0b",
               "filter_size_in_bytes": 0,
               "bloom_size": "0b",
               "bloom_size_in_bytes": 0,
               "id_cache_size": "0b",
               "id_cache_size_in_bytes": 0
            },

Cheers,
Boaz

NOTE: This may just be a documentation issue as I'm guessing how you should "warm" a sort field. If this is the wrong way, I'd be happy know.

kimchy · 2013-02-13T16:50:58Z

Yea, we use COUNT search type, which does not kick in sorting, will change it to regular search. Btw, its much simpler to just have regular curl recreation, so we can more easily run it.

fixes #2649

fixes elastic#2649

kimchy added a commit that referenced this issue Feb 13, 2013

Warmers do not load field data cache for sorting on new segments

439417d

fixes #2649

kimchy closed this as completed in 681239b Feb 13, 2013

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

Warmers do not load field data cache for sorting on new segments

b8562d9

fixes elastic#2649

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Warmers do not load field data cache for sorting on new segments #2649

Warmers do not load field data cache for sorting on new segments #2649

bleskes commented Feb 13, 2013

kimchy commented Feb 13, 2013

Navigation Menu

Warmers do not load field data cache for sorting on new segments #2649

Warmers do not load field data cache for sorting on new segments #2649

Comments

bleskes commented Feb 13, 2013

kimchy commented Feb 13, 2013