New timestamp default option changed the default behavior for missing paths #9049
Conversation
… paths PR elastic#7036 changed the behavior for timestamp when provided as a parameter or within the document using `path` attribute. This PR now considers that: * when using timestamp as a parameter, we use a default value of `now`. Which means that if no timestamp is provided, the current time is used when the index operation is performed. * when getting the timestamp from `path`, we use a default value of `null`. Which means that if no value is provided within the document, indexation will fail. If users want to set the default value for `timestamp`, they can explicitly set one or set `"default": "now"`. Closes elastic#8882.
dadoonet
added
review
:Search/Mapping
|
The "default default" should not change dynamically based on how other settings are set, IMO. That would be very confusing for users. Whether the timestamp comes in |
|
@rjernst In that case, how to force a rejection when timestamp is not provided within the json document (when using |
|
How do we require other fields? I thought it was with |
|
The problem is that some languages (.Net) can not explicitly set Let's say we use |
Then isn't this a problem for all field types, not just timestamp? |
|
I think it's the only field for which null has a special meaning. Right @Mpdreamz? |
|
If there are problems with using |
|
Like |
|
Sure, sounds good to me. :) |
|
Closing in favor of #9104. |
Related to elastic#9049. By default, the default value for `timestamp` is `now` which means the date the document was processed by the indexing chain. You can now reject documents which not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`): ```js { "tweet" : { "_timestamp" : { "enabled" : true, "ignore_missing" : false } } } ``` When you update the cluster to 1.5 or master, this index created with 1.4 we automatically migrate an index created with 1.4 to the 1.5 syntax. Let say you have defined this in elasticsearch 1.4.x: ```js DELETE test PUT test { "settings": { "number_of_shards": 1, "number_of_replicas": 0 } } PUT test/type/_mapping { "type" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` After migration, the mapping become: ```js { "test": { "mappings": { "type": { "_timestamp": { "enabled": true, "store": false, "ignore_missing": false }, "properties": {} } } } } ``` Closes elastic#8882.
Related to #9049. By default, the default value for `timestamp` is `now` which means the date the document was processed by the indexing chain. You can now reject documents which not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`): ```js { "tweet" : { "_timestamp" : { "enabled" : true, "ignore_missing" : false } } } ``` When you update the cluster to 1.5 or master, this index created with 1.4 we automatically migrate an index created with 1.4 to the 1.5 syntax. Let say you have defined this in elasticsearch 1.4.x: ```js DELETE test PUT test { "settings": { "number_of_shards": 1, "number_of_replicas": 0 } } PUT test/type/_mapping { "type" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` After migration, the mapping become: ```js { "test": { "mappings": { "type": { "_timestamp": { "enabled": true, "store": false, "ignore_missing": false }, "properties": {} } } } } ``` Closes #8882. (cherry picked from commit fb10346)
When creating an index with:
```
PUT new_index
{
"mappings": {
"power": {
"_timestamp" : {
"enabled" : true,
"default": null
}
}
}
}
```
When restarting the cluster, `now` is applied instead of `null`. So index become:
```
{
"mappings": {
"power": {
"_timestamp" : {
"enabled" : true,
"default": "now"
}
}
}
}
```
This PR fix that and applies `null` when it was explicitly set.
Note that this won't happen anymore in 1.5 as `null` is not allowed anymore as a `default` value. See elastic#9104.
See also:
* elastic#7036
* elastic#9049
* elastic#9426#issuecomment-71534871
When creating an index with:
```
PUT new_index
{
"mappings": {
"power": {
"_timestamp" : {
"enabled" : true,
"default": null
}
}
}
}
```
When restarting the cluster, `now` is applied instead of `null`. So index become:
```
{
"mappings": {
"power": {
"_timestamp" : {
"enabled" : true,
"default": "now"
}
}
}
}
```
This PR fix that and applies `null` when it was explicitly set.
Note that this won't happen anymore in 1.5 as `null` is not allowed anymore as a `default` value. See elastic#9104.
See also:
* elastic#7036
* elastic#9049
* elastic#9426#issuecomment-71534871
PR #7036 changed the behavior for timestamp when provided as a parameter or within the document using
pathattribute.This PR now considers that:
now. Which means that if no timestamp is provided, the current time is used when the index operation is performed.path, we use a default value ofnull. Which means that if no value is provided within the document, indexation will fail.If users want to set the default value for
timestamp, they can explicitly set one or set"default": "now".Closes #8882.