Add the ability to disallow field data loading #4431
Assignees
Labels
Comments
ghost
assigned
jpountz
added a commit
to jpountz/elasticsearch
that referenced
this issue
Dec 12, 2013
…ility to disallow field data loading. This commit changes field data configuration updates so that they are immediately taken into account for loading new segments. The way it works is that field data configuration is now cached separately from the field data cache, meaning that it is now possible to clear the field data configuration from IndexFieldDataService while the cache will stay around. On the next time that Elasticsearch will reload field data configuration, it will check if there is already a cache entry, and reuse it if it exists. To disable field data loading, all that is required is to change the field data format to "none" (supported by all field data types) using the update mapping API. Elasticsearch will then refuse to load field data on any new segment, but field data which has been loaded on the previous segments will remain available. So you need to clear the field data cache in order to reclaim memory (otherwise memory will be reclaimed slower, as segments get merged). Close elastic#4430 Close elastic#4431
|
huge +1 to the feature :) |
|
nice screw-up protection. what about documentation? |
|
I just added a new commit with documentation. Shame on me for having forgotten it! |
jpountz
added a commit
that referenced
this issue
Dec 16, 2013
…ility to disallow field data loading. This commit changes field data configuration updates so that they are immediately taken into account for loading new segments. The way it works is that field data configuration is now cached separately from the field data cache, meaning that it is now possible to clear the field data configuration from IndexFieldDataService while the cache will stay around. On the next time that Elasticsearch will reload field data configuration, it will check if there is already a cache entry, and reuse it if it exists. To disable field data loading, all that is required is to change the field data format to "none" (supported by all field data types) using the update mapping API. Elasticsearch will then refuse to load field data on any new segment, but field data which has been loaded on the previous segments will remain available. So you need to clear the field data cache in order to reclaim memory (otherwise memory will be reclaimed slower, as segments get merged). Close #4430 Close #4431
brusic
pushed a commit
to brusic/elasticsearch
that referenced
this issue
Jan 19, 2014
…ility to disallow field data loading. This commit changes field data configuration updates so that they are immediately taken into account for loading new segments. The way it works is that field data configuration is now cached separately from the field data cache, meaning that it is now possible to clear the field data configuration from IndexFieldDataService while the cache will stay around. On the next time that Elasticsearch will reload field data configuration, it will check if there is already a cache entry, and reuse it if it exists. To disable field data loading, all that is required is to change the field data format to "none" (supported by all field data types) using the update mapping API. Elasticsearch will then refuse to load field data on any new segment, but field data which has been loaded on the previous segments will remain available. So you need to clear the field data cache in order to reclaim memory (otherwise memory will be reclaimed slower, as segments get merged). Close elastic#4430 Close elastic#4431
|
i think so there is a problem in the documentation wich says that "The disabled format is supported by all field types." it works for string, long but not for nested ! (i use ES 0.9.11) |
|
There is no field data concept for the nested type so indeed the documentation should rather state that it is supported on all core types. I will fix it soon, thanks! |
|
@CamiloSierraH to be sure I got you right, what were your expectations when setting the field data format on a nested field? |
|
@jpountz i have some nested that i like disabled, i only use this nested in filters (no facets, no sort) because i have some millions of documents and i need to use the ram only if is necessary! Thanks |
|
@CamiloSierraH In that case you should use the {
"type1" : {
"properties" : {
"obj1" : {
"type" : "nested",
"properties": {
"prop1": {
"type": "long",
"fielddata": {
"format": "disabled"
}
}
}
}
}
}
}Note: |
|
ok thank you @jpountz !! (i hope i can release some memory ram 👍 ) |
mute
pushed a commit
to mute/elasticsearch
that referenced
this issue
Jul 29, 2015
…ility to disallow field data loading. This commit changes field data configuration updates so that they are immediately taken into account for loading new segments. The way it works is that field data configuration is now cached separately from the field data cache, meaning that it is now possible to clear the field data configuration from IndexFieldDataService while the cache will stay around. On the next time that Elasticsearch will reload field data configuration, it will check if there is already a cache entry, and reuse it if it exists. To disable field data loading, all that is required is to change the field data format to "none" (supported by all field data types) using the update mapping API. Elasticsearch will then refuse to load field data on any new segment, but field data which has been loaded on the previous segments will remain available. So you need to clear the field data cache in order to reclaim memory (otherwise memory will be reclaimed slower, as segments get merged). Close elastic#4430 Close elastic#4431
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Field data can be a black hole for memory (eg. on full-text fields), so it should be possible to disallow field data loading on a per-field basis.
The text was updated successfully, but these errors were encountered: