Skip to content

Commit

Permalink
Avoid unnecessary use of stored_fields in our docs. (#57488)
Browse files Browse the repository at this point in the history
Generally we don't advocate for using `stored_fields`, and we're interested in
eventually removing the need for this parameter. So it's best to avoid using
stored fields in our docs examples when it's not actually necessary.

Individual changes:
* Avoid using 'stored_fields' in our docs.
* When defining script fields in top-hits, de-emphasize stored fields.
  • Loading branch information
jtibshirani committed Jun 2, 2020
1 parent 73f1e2b commit 681ca22
Show file tree
Hide file tree
Showing 3 changed files with 67 additions and 44 deletions.
6 changes: 1 addition & 5 deletions docs/painless/painless-guide/painless-walkthrough.asciidoc
Expand Up @@ -132,10 +132,6 @@ First, let's look at the source data for a player by submitting the following re
----------------------------------------------------------------
GET hockey/_search
{
"stored_fields": [
"_id",
"_source"
],
"query": {
"term": {
"_id": 1
Expand Down Expand Up @@ -189,7 +185,7 @@ Date fields are exposed as
`ZonedDateTime`, so they support methods like `getYear`, `getDayOfWeek`
or e.g. getting milliseconds since epoch with `getMillis`. To use these
in a script, leave out the `get` prefix and continue with lowercasing the
rest of the method name. For example, the following returns every hockey
rest of the method name. For example, the following returns every hockey
player's birth year:

[source,console]
Expand Down
103 changes: 65 additions & 38 deletions docs/reference/scripting/fields.asciidoc
Expand Up @@ -26,8 +26,10 @@ Depending on how many documents you have, this could mean millions or billions
of executions: these scripts need to be fast!

Field values can be accessed from a script using
<<modules-scripting-doc-vals,doc-values>>, or
<<modules-scripting-stored,stored fields or `_source` field>>, which are explained below.
<<modules-scripting-doc-vals,doc-values>>,
<<modules-scripting-source, the `_source` field>>, or
<<modules-scripting-stored, stored fields>>,
each of which is explained below.

[[scripting-score]]
[float]
Expand Down Expand Up @@ -137,32 +139,27 @@ access `text` fields from scripts.
===================================================

[float]
[[modules-scripting-stored]]
=== Stored fields and `_source`

_Stored fields_ -- fields explicitly marked as
<<mapping-store,`"store": true`>> -- can be accessed using the
`_fields['field_name'].value` or `_fields['field_name']` syntax.
[[modules-scripting-source]]
=== The document `_source`

The document <<mapping-source-field,`_source`>>, which is really just a
special stored field, can be accessed using the `_source.field_name` syntax.
The `_source` is loaded as a map-of-maps, so properties within object fields
can be accessed as, for example, `_source.name.first`.
The document <<mapping-source-field,`_source`>> can be accessed using the
`_source.field_name` syntax. The `_source` is loaded as a map-of-maps, so
properties within object fields can be accessed as, for example,
`_source.name.first`.

[IMPORTANT]
.Prefer doc-values to stored fields
.Prefer doc-values to _source
=========================================================
Stored fields (which includes the stored `_source` field) are much slower than
doc-values. They are optimised for returning several fields per result,
while doc values are optimised for accessing the value of a specific field in
many documents.
Accessing the `_source` field is much slower than using doc-values. The
_source field is optimised for returning several fields per result, while doc
values are optimised for accessing the value of a specific field in many
documents.
It makes sense to use `_source` or stored fields when generating a
<<request-body-search-script-fields,script field>> for the top ten hits from a search
result but, for other search and aggregation use cases, always prefer using
doc values.
It makes sense to use `_source` when generating a
<<request-body-search-script-fields,script field>> for the top ten hits from a
search result but, for other search and aggregation use cases, always prefer
using doc values.
=========================================================


Expand All @@ -174,50 +171,80 @@ PUT my_index
{
"mappings": {
"properties": {
"title": { <1>
"type": "text"
},
"first_name": {
"type": "text",
"store": true
"type": "text"
},
"last_name": {
"type": "text",
"store": true
"type": "text"
}
}
}
}
PUT my_index/_doc/1?refresh
{
"title": "Mr",
"first_name": "Barry",
"last_name": "White"
}
GET my_index/_search
{
"script_fields": {
"source": {
"full_name": {
"script": {
"lang": "painless",
"source": "params._source.title + ' ' + params._source.first_name + ' ' + params._source.last_name" <2>
"source": "params._source.first_name + ' ' + params._source.last_name"
}
}
}
}
-------------------------------

[float]
[[modules-scripting-stored]]
=== Stored fields

_Stored fields_ -- fields explicitly marked as
<<mapping-store,`"store": true`>> in the mapping -- can be accessed using the
`_fields['field_name'].value` or `_fields['field_name']` syntax:

[source,console]
-------------------------------
PUT my_index
{
"mappings": {
"properties": {
"full_name": {
"type": "text",
"store": true
},
"title": {
"type": "text",
"store": true
}
},
"stored_fields": {
}
}
}
PUT my_index/_doc/1?refresh
{
"full_name": "Alice Ball",
"title": "Professor"
}
GET my_index/_search
{
"script_fields": {
"name_with_title": {
"script": {
"lang": "painless",
"source": "params._fields['first_name'].value + ' ' + params._fields['last_name'].value"
"source": "params._fields['title'].value + ' ' + params._fields['full_name'].value"
}
}
}
}
-------------------------------

<1> The `title` field is not stored and so cannot be used with the `_fields[]` syntax.
<2> The `title` field can still be accessed from the `_source`.

[TIP]
.Stored vs `_source`
=======================================================
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/search/request/highlighting.asciidoc
Expand Up @@ -309,7 +309,6 @@ highlighting would only take the search query into account.
--------------------------------------------------
GET /_search
{
"stored_fields": [ "_id" ],
"query" : {
"match": {
"comment": {
Expand All @@ -331,6 +330,7 @@ GET /_search
"rescore_query_weight" : 10
}
},
"_source": false,
"highlight" : {
"order" : "score",
"fields" : {
Expand Down

0 comments on commit 681ca22

Please sign in to comment.