Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add _source, _doc and _field support to fields #4492

Closed
clintongormley opened this issue Dec 17, 2013 · 6 comments
Closed

Add _source, _doc and _field support to fields #4492

clintongormley opened this issue Dec 17, 2013 · 6 comments
Assignees

Comments

@clintongormley
Copy link

When requesting fields in the search, get, update, etc APIs, we should be able to control where those values come from with these prefixes:

  • _source: extract the field value from the _source field
  • _fields: retrieve a stored field value
  • _doc: retrieve the value from fielddata

The _doc option is particularly useful for retrieving dates as, no matter the format they're specified in in the source, they will be accessible as milliseconds-since-the-epoch from fielddata, without the need to allow dynamic scripts.

While on the subject, perhaps we should support this same syntax in scripting, instead of docs['foo']?

@ghost ghost assigned dakrone Jan 7, 2014
@dakrone
Copy link
Member

dakrone commented Jan 9, 2014

@clintongormley to make sure I understand, you proposing this syntax:

curl 'localhost:9200/_search?q=foo:bar&fields=_source:body,_fields:foo,_doc:date'

and

curl 'localhost:9200/_search' -d'{
  "query": {"match_all": {}},
  "fields": ["_source:body", "_fields:foo", "_doc:date"]
}'

Is that correct?

@dakrone
Copy link
Member

dakrone commented Jan 9, 2014

We discussed this and decided to add a fielddata_fields parameter similar to script_fields to extract fields from the field data for returning. It'll also remove the workaround here: https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/search/fetch/FieldsParseElement.java#L40

@kimchy
Copy link
Member

kimchy commented Jan 11, 2014

@dakrone and the workaround in the GET API....

@dakrone
Copy link
Member

dakrone commented Jan 14, 2014

@kimchy I currently have fielddata_fields working just like script_fields does, however, if I remove the workaround for _source.<name> and doc['name'].value, then it's not possible to retrieve field data values via the GET API.

Is this acceptable, or should I work on adding support for fielddata_fields in the GET API? (script_fields is also not supported in the GET API).

@kimchy
Copy link
Member

kimchy commented Jan 14, 2014

I am personally ok with it, we have source filtering now for the get API, and doc notation is not supported for GET. The reason why doc / fielddata is problematic with GET API is that we support realtime get, and we might fetch the doc from the transaction log and not form Lucene.

@dakrone
Copy link
Member

dakrone commented Jan 14, 2014

Sounds good, thanks!

dakrone added a commit to dakrone/elasticsearch that referenced this issue Jan 21, 2014
Adds a new FetchSubPhase, FieldDataFieldsFetchSubPhase, which loads the
field data cache for a field and returns an array of values for the
field.

Also removes `doc['<field>']` and `_source.<field>` workaround no longer
needed in field name resolving.

Closes elastic#4492
dakrone added a commit that referenced this issue Jan 21, 2014
Adds a new FetchSubPhase, FieldDataFieldsFetchSubPhase, which loads the
field data cache for a field and returns an array of values for the
field.

Also removes `doc['<field>']` and `_source.<field>` workaround no longer
needed in field name resolving.

Closes #4492
dakrone added a commit that referenced this issue Jan 21, 2014
Adds a new FetchSubPhase, FieldDataFieldsFetchSubPhase, which loads the
field data cache for a field and returns an array of values for the
field.

Also removes `doc['<field>']` and `_source.<field>` workaround no longer
needed in field name resolving.

Closes #4492
mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
Adds a new FetchSubPhase, FieldDataFieldsFetchSubPhase, which loads the
field data cache for a field and returns an array of values for the
field.

Also removes `doc['<field>']` and `_source.<field>` workaround no longer
needed in field name resolving.

Closes elastic#4492
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants