Skip to content

Commit

Permalink
When defining script fields in top-hits, de-emphasize stored fields.
Browse files Browse the repository at this point in the history
  • Loading branch information
jtibshirani committed Jun 1, 2020
1 parent 321df98 commit 647321a
Show file tree
Hide file tree
Showing 2 changed files with 66 additions and 39 deletions.
103 changes: 65 additions & 38 deletions docs/reference/scripting/fields.asciidoc
Expand Up @@ -26,8 +26,10 @@ Depending on how many documents you have, this could mean millions or billions
of executions: these scripts need to be fast!

Field values can be accessed from a script using
<<modules-scripting-doc-vals,doc-values>>, or
<<modules-scripting-stored,stored fields or `_source` field>>, which are explained below.
<<modules-scripting-doc-vals,doc-values>>,
<<modules-scripting-source, the `_source` field>>, or
<<modules-scripting-stored, stored fields>>,
each of which is explained below.

[[scripting-score]]
[float]
Expand Down Expand Up @@ -137,32 +139,27 @@ access `text` fields from scripts.
===================================================

[float]
[[modules-scripting-stored]]
=== Stored fields and `_source`

_Stored fields_ -- fields explicitly marked as
<<mapping-store,`"store": true`>> -- can be accessed using the
`_fields['field_name'].value` or `_fields['field_name']` syntax.
[[modules-scripting-source]]
=== The document `_source`

The document <<mapping-source-field,`_source`>>, which is really just a
special stored field, can be accessed using the `_source.field_name` syntax.
The `_source` is loaded as a map-of-maps, so properties within object fields
can be accessed as, for example, `_source.name.first`.
The document <<mapping-source-field,`_source`>> can be accessed using the
`_source.field_name` syntax. The `_source` is loaded as a map-of-maps, so
properties within object fields can be accessed as, for example,
`_source.name.first`.

[IMPORTANT]
.Prefer doc-values to stored fields
.Prefer doc-values to _source
=========================================================
Stored fields (which includes the stored `_source` field) are much slower than
doc-values. They are optimised for returning several fields per result,
while doc values are optimised for accessing the value of a specific field in
many documents.
Accessing the `_source` field is much slower than using doc-values. The
_source field is optimised for returning several fields per result, while doc
values are optimised for accessing the value of a specific field in many
documents.
It makes sense to use `_source` or stored fields when generating a
<<request-body-search-script-fields,script field>> for the top ten hits from a search
result but, for other search and aggregation use cases, always prefer using
doc values.
It makes sense to use `_source` when generating a
<<request-body-search-script-fields,script field>> for the top ten hits from a
search result but, for other search and aggregation use cases, always prefer
using doc values.
=========================================================


Expand All @@ -174,50 +171,80 @@ PUT my_index
{
"mappings": {
"properties": {
"title": { <1>
"type": "text"
},
"first_name": {
"type": "text",
"store": true
"type": "text"
},
"last_name": {
"type": "text",
"store": true
"type": "text"
}
}
}
}
PUT my_index/_doc/1?refresh
{
"title": "Mr",
"first_name": "Barry",
"last_name": "White"
}
GET my_index/_search
{
"script_fields": {
"source": {
"full_name": {
"script": {
"lang": "painless",
"source": "params._source.title + ' ' + params._source.first_name + ' ' + params._source.last_name" <2>
"source": "params._source.first_name + ' ' + params._source.last_name"
}
}
}
}
-------------------------------

[float]
[[modules-scripting-stored]]
=== Stored fields

_Stored fields_ -- fields explicitly marked as
<<mapping-store,`"store": true`>> in the mapping -- can be accessed using the
`_fields['field_name'].value` or `_fields['field_name']` syntax:

[source,console]
-------------------------------
PUT my_index
{
"mappings": {
"properties": {
"full_name": {
"type": "text",
"store": true
},
"title": {
"type": "text",
"store": true
}
},
"stored_fields": {
}
}
}
PUT my_index/_doc/1?refresh
{
"full_name": "Alice Ball",
"title": "Professor"
}
GET my_index/_search
{
"script_fields": {
"name_with_title": {
"script": {
"lang": "painless",
"source": "params._fields['first_name'].value + ' ' + params._fields['last_name'].value"
"source": "params._fields['title'].value + ' ' + params._fields['full_name'].value"
}
}
}
}
-------------------------------

<1> The `title` field is not stored and so cannot be used with the `_fields[]` syntax.
<2> The `title` field can still be accessed from the `_source`.

[TIP]
.Stored vs `_source`
=======================================================
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/search/request/highlighting.asciidoc
Expand Up @@ -330,7 +330,7 @@ GET /_search
"rescore_query_weight" : 10
}
},
"_source": false
"_source": false,
"highlight" : {
"order" : "score",
"fields" : {
Expand Down

0 comments on commit 647321a

Please sign in to comment.