Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid unnecessary use of stored_fields in our docs. #57488

Merged
merged 2 commits into from Jun 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 1 addition & 5 deletions docs/painless/painless-guide/painless-walkthrough.asciidoc
Expand Up @@ -132,10 +132,6 @@ First, let's look at the source data for a player by submitting the following re
----------------------------------------------------------------
GET hockey/_search
{
"stored_fields": [
"_id",
"_source"
],
"query": {
"term": {
"_id": 1
Expand Down Expand Up @@ -189,7 +185,7 @@ Date fields are exposed as
`ZonedDateTime`, so they support methods like `getYear`, `getDayOfWeek`
or e.g. getting milliseconds since epoch with `getMillis`. To use these
in a script, leave out the `get` prefix and continue with lowercasing the
rest of the method name. For example, the following returns every hockey
rest of the method name. For example, the following returns every hockey
player's birth year:

[source,console]
Expand Down
103 changes: 65 additions & 38 deletions docs/reference/scripting/fields.asciidoc
Expand Up @@ -26,8 +26,10 @@ Depending on how many documents you have, this could mean millions or billions
of executions: these scripts need to be fast!

Field values can be accessed from a script using
<<modules-scripting-doc-vals,doc-values>>, or
<<modules-scripting-stored,stored fields or `_source` field>>, which are explained below.
<<modules-scripting-doc-vals,doc-values>>,
<<modules-scripting-source, the `_source` field>>, or
<<modules-scripting-stored, stored fields>>,
each of which is explained below.

[[scripting-score]]
[float]
Expand Down Expand Up @@ -137,32 +139,27 @@ access `text` fields from scripts.
===================================================

[float]
[[modules-scripting-stored]]
=== Stored fields and `_source`

_Stored fields_ -- fields explicitly marked as
<<mapping-store,`"store": true`>> -- can be accessed using the
`_fields['field_name'].value` or `_fields['field_name']` syntax.
[[modules-scripting-source]]
=== The document `_source`

The document <<mapping-source-field,`_source`>>, which is really just a
special stored field, can be accessed using the `_source.field_name` syntax.
The `_source` is loaded as a map-of-maps, so properties within object fields
can be accessed as, for example, `_source.name.first`.
The document <<mapping-source-field,`_source`>> can be accessed using the
`_source.field_name` syntax. The `_source` is loaded as a map-of-maps, so
properties within object fields can be accessed as, for example,
`_source.name.first`.

[IMPORTANT]
.Prefer doc-values to stored fields
.Prefer doc-values to _source
=========================================================

Stored fields (which includes the stored `_source` field) are much slower than
doc-values. They are optimised for returning several fields per result,
while doc values are optimised for accessing the value of a specific field in
many documents.
Accessing the `_source` field is much slower than using doc-values. The
_source field is optimised for returning several fields per result, while doc
values are optimised for accessing the value of a specific field in many
documents.


It makes sense to use `_source` or stored fields when generating a
<<request-body-search-script-fields,script field>> for the top ten hits from a search
result but, for other search and aggregation use cases, always prefer using
doc values.
It makes sense to use `_source` when generating a
<<request-body-search-script-fields,script field>> for the top ten hits from a
search result but, for other search and aggregation use cases, always prefer
using doc values.
=========================================================


Expand All @@ -174,50 +171,80 @@ PUT my_index
{
"mappings": {
"properties": {
"title": { <1>
"type": "text"
},
"first_name": {
"type": "text",
"store": true
"type": "text"
},
"last_name": {
"type": "text",
"store": true
"type": "text"
}
}
}
}

PUT my_index/_doc/1?refresh
{
"title": "Mr",
"first_name": "Barry",
"last_name": "White"
}

GET my_index/_search
{
"script_fields": {
"source": {
"full_name": {
"script": {
"lang": "painless",
"source": "params._source.title + ' ' + params._source.first_name + ' ' + params._source.last_name" <2>
"source": "params._source.first_name + ' ' + params._source.last_name"
}
}
}
}
-------------------------------

[float]
[[modules-scripting-stored]]
=== Stored fields

_Stored fields_ -- fields explicitly marked as
<<mapping-store,`"store": true`>> in the mapping -- can be accessed using the
`_fields['field_name'].value` or `_fields['field_name']` syntax:

[source,console]
-------------------------------
PUT my_index
{
"mappings": {
"properties": {
"full_name": {
"type": "text",
"store": true
},
"title": {
"type": "text",
"store": true
}
},
"stored_fields": {
}
}
}

PUT my_index/_doc/1?refresh
{
"full_name": "Alice Ball",
"title": "Professor"
}

GET my_index/_search
{
"script_fields": {
"name_with_title": {
"script": {
"lang": "painless",
"source": "params._fields['first_name'].value + ' ' + params._fields['last_name'].value"
"source": "params._fields['title'].value + ' ' + params._fields['full_name'].value"
}
}
}
}
-------------------------------

<1> The `title` field is not stored and so cannot be used with the `_fields[]` syntax.
<2> The `title` field can still be accessed from the `_source`.

[TIP]
.Stored vs `_source`
=======================================================
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/search/request/highlighting.asciidoc
Expand Up @@ -309,7 +309,6 @@ highlighting would only take the search query into account.
--------------------------------------------------
GET /_search
{
"stored_fields": [ "_id" ],
"query" : {
"match": {
"comment": {
Expand All @@ -331,6 +330,7 @@ GET /_search
"rescore_query_weight" : 10
}
},
"_source": false,
"highlight" : {
"order" : "score",
"fields" : {
Expand Down