Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replaced multi-field type with multi-field syntax. #4566

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/reference/indices/put-mapping.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ which means conflicts are *not* ignored.
The definition of conflict is really dependent on the type merged, but
in general, if a different core type is defined, it is considered as a
conflict. New mapping definitions can be added to object types, and core
type mapping can be upgraded to `multi_field` type.
type mapping can be upgraded by specifying multi fields on a core type.

[float]
[[put-mapping-multi-index]]
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/mapping/misc.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,5 @@ include::mapping/date-format.asciidoc[]
include::mapping/conf-mappings.asciidoc[]

include::mapping/meta.asciidoc[]

include::mapping/multi-fields.asciidoc[]
120 changes: 120 additions & 0 deletions docs/reference/mapping/multi-fields.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
[[multi-fields]]
== Multi Fields

The `fields` options allows to map several core types fields into a single
json source field. This can be useful if a single field need to be
used in different ways. For example a single field is to be used for both
free text search and sorting.

[source,js]
--------------------------------------------------
{
"tweet" : {
"properties" : {
"name" : {
"type" : "string",
"index" : "analyzed",
"fields" : {
"raw" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
}
}
--------------------------------------------------

In the above example the field name gets processed twice. The first time is gets
processed as an analyzed string and this version is accessible under the field name
`name`, this is the main field and is in fact just like any other field. The second time
its get processed as a not analyzed string and is accessible under the name `name.raw`.

[float]
=== Accessing Fields

The multi fields defined in the `fields` are prefixed with the
name of the main field and can be accessed by their full path using the
navigation notation: `name.raw`, or using the typed navigation notation
`tweet.name.raw`. The `path` option allows to control how fields are accessed.
If the `path` option is set to `full`, then the full path of the main field
is prefixed, but if the `path` option is set to `just_name` the actual
multi field name without any prefix is used. The default value for
the `path` option is `full`.

The `just_name` setting, among other things, allows indexing content of multiple
fields under the same name. In the example below the content of both fields
`first_name` and `last_name` can be accessed by using `any_name` or `tweet.any_name`.

[source,js]
--------------------------------------------------
{
"tweet" : {
"properties": {
"first_name": {
"type": "string",
"index": "analyzed",
"path": "just_name",
"fields": {
"any_name": {"type": "string","index": "analyzed"}
}
},
"last_name": {
"type": "string",
"index": "analyzed",
"path": "just_name",
"fields": {
"any_name": {"type": "string","index": "analyzed"}
}
}
}
}
}
--------------------------------------------------

[float]
=== Include in All

The `include_in_all` setting is ignored on any field that is defined in
the `fields` options. Setting the `include_in_all` only makes sense on
the main field, since the raw field value to copied to the `_all` field,
the tokens aren't copied.

[float]
=== Updating a field

In the essence a field can't be updated. However multi fields can be
added to existing fields. This allows for example to have a different
`index_analyzer` configuration in addition to the already configured
`index_analyzer` configuration specified in the main and other multi fields.

Also the new multi field will only be applied on document that have been
added after the multi field has been added and in fact the new multi field
doesn't exist in existing documents.

Another important note is that new multi fields will be merged into the
list of existing multi fields, so when adding new multi fields for a field
previous added multi fields don't need to be specified.

[float]
=== Inherit settings from main field

Any settings defined on the main field are automatically inherited by all
multi fields and act as a default for a multi field.

The first example could also be defined as in the example below:

[source,js]
--------------------------------------------------
{
"tweet" : {
"properties" : {
"name" : {
"type" : "string",
"index" : "analyzed",
"fields" : {
"raw" : {"index" : "not_analyzed"}
}
}
}
}
}
--------------------------------------------------
2 changes: 0 additions & 2 deletions docs/reference/mapping/types.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ include::types/root-object-type.asciidoc[]

include::types/nested-type.asciidoc[]

include::types/multi-field-type.asciidoc[]

include::types/ip-type.asciidoc[]

include::types/geo-point-type.asciidoc[]
Expand Down
7 changes: 2 additions & 5 deletions docs/reference/mapping/types/core-types.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -245,12 +245,9 @@ example:
{
"tweet" : {
"properties" : {
"message" : {
"type" : "multi_field",
"name" : {
"type" : "string",
"fields" : {
"name": {
"type": "string"
},
"word_count": {
"type" : "token_count",
"store" : "yes",
Expand Down
93 changes: 0 additions & 93 deletions docs/reference/mapping/types/multi-field-type.asciidoc

This file was deleted.

10 changes: 5 additions & 5 deletions docs/reference/mapping/types/root-object-type.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,8 @@ when dynamic introduction of fields / objects happens.

For example, we might want to have all fields to be stored by default,
or all `string` fields to be stored, or have `string` fields to always
be indexed as `multi_field`, once analyzed and once not_analyzed. Here
is a simple example:
be indexed with multi fields syntax, once analyzed and once not_analyzed.
Here is a simple example:

[source,js]
--------------------------------------------------
Expand All @@ -145,9 +145,9 @@ is a simple example:
"template_1" : {
"match" : "multi*",
"mapping" : {
"type" : "multi_field",
"type" : "{dynamic_type}",
"index" : "analyzed",
"fields" : {
"{name}" : {"type": "{dynamic_type}", "index" : "analyzed"},
"org" : {"type": "{dynamic_type}", "index" : "not_analyzed"}
}
}
Expand All @@ -168,7 +168,7 @@ is a simple example:
}
--------------------------------------------------

The above mapping will create a `multi_field` mapping for all field
The above mapping will create a field with multi fields for all field
names starting with multi, and will map all `string` types to be
`not_analyzed`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ The following parameters are supported:

NOTE: Even though you are losing most of the features of the
completion suggest, you can opt in for the shortest form, which even
allows you to use inside of multi_field. But keep in mind, that you will
allows you to use inside of multi fields. But keep in mind, that you will
not be able to use several inputs, an output, payloads or weights.

[source,js]
Expand Down
29 changes: 22 additions & 7 deletions rest-api-spec/test/indices.put_mapping/10_basic.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,21 @@
body:
test_type:
properties:
text:
text1:
type: string
analyzer: whitespace
text2:
type: string
analyzer: whitespace

- do:
indices.get_mapping:
index: test_index

- match: {test_index.test_type.properties.text.type: string}
- match: {test_index.test_type.properties.text.analyzer: whitespace}
- match: {test_index.test_type.properties.text1.type: string}
- match: {test_index.test_type.properties.text1.analyzer: whitespace}
- match: {test_index.test_type.properties.text2.type: string}
- match: {test_index.test_type.properties.text2.analyzer: whitespace}

- do:
indices.put_mapping:
Expand All @@ -29,19 +34,29 @@
body:
test_type:
properties:
text:
text1:
type: multi_field
fields:
text:
text1:
type: string
analyzer: whitespace
text_raw:
type: string
index: not_analyzed
text2:
type: string
analyzer: whitespace
fields:
text_raw:
type: string
index: not_analyzed


- do:
indices.get_mapping:
index: test_index

- match: {test_index.test_type.properties.text.type: multi_field}
- match: {test_index.test_type.properties.text.fields.text_raw.index: not_analyzed }
- match: {test_index.test_type.properties.text1.type: string}
- match: {test_index.test_type.properties.text1.fields.text_raw.index: not_analyzed}
- match: {test_index.test_type.properties.text2.type: string}
- match: {test_index.test_type.properties.text2.fields.text_raw.index: not_analyzed}
Original file line number Diff line number Diff line change
Expand Up @@ -572,13 +572,9 @@ public ParsedDocument parse(SourceToParse source, @Nullable ParseListener listen
return doc;
}

public void addFieldMappers(Collection<FieldMapper> fieldMappers) {
addFieldMappers(fieldMappers.toArray(new FieldMapper[fieldMappers.size()]));
}

public void addFieldMappers(FieldMapper... fieldMappers) {
public void addFieldMappers(Iterable<FieldMapper> fieldMappers) {
synchronized (mappersMutex) {
this.fieldMappers.addNewMappers(Arrays.asList(fieldMappers));
this.fieldMappers.addNewMappers(fieldMappers);
}
for (FieldMapperListener listener : fieldMapperListeners) {
listener.fieldMappers(fieldMappers);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@
import org.elasticsearch.index.mapper.geo.GeoShapeFieldMapper;
import org.elasticsearch.index.mapper.internal.*;
import org.elasticsearch.index.mapper.ip.IpFieldMapper;
import org.elasticsearch.index.mapper.multifield.MultiFieldMapper;
import org.elasticsearch.index.mapper.object.ObjectMapper;
import org.elasticsearch.index.mapper.object.RootObjectMapper;
import org.elasticsearch.index.settings.IndexSettings;
Expand Down Expand Up @@ -98,7 +97,7 @@ public DocumentMapperParser(Index index, @IndexSettings Settings indexSettings,
.put(TokenCountFieldMapper.CONTENT_TYPE, new TokenCountFieldMapper.TypeParser())
.put(ObjectMapper.CONTENT_TYPE, new ObjectMapper.TypeParser())
.put(ObjectMapper.NESTED_CONTENT_TYPE, new ObjectMapper.TypeParser())
.put(MultiFieldMapper.CONTENT_TYPE, new MultiFieldMapper.TypeParser())
.put(TypeParsers.MULTI_FIELD_CONTENT_TYPE, TypeParsers.multiFieldConverterTypeParser)
.put(CompletionFieldMapper.CONTENT_TYPE, new CompletionFieldMapper.TypeParser())
.put(GeoPointFieldMapper.CONTENT_TYPE, new GeoPointFieldMapper.TypeParser());

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ public void fieldMapper(FieldMapper fieldMapper) {

public abstract void fieldMapper(FieldMapper fieldMapper);

public void fieldMappers(FieldMapper... fieldMappers) {
public void fieldMappers(Iterable<FieldMapper> fieldMappers) {
for (FieldMapper mapper : fieldMappers) {
fieldMapper(mapper);
}
Expand Down