[ML] Make datafeeds work with nanosecond time fields #51180
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Allows ML datafeeds to work with time fields that have the "date_nanos" type. (Previously datafeeds only worked with time fields that had the "date" type.)
|
Pinging @elastic/ml-core (:ml) |
davidkyle
approved these changes
Jan 20, 2020
x-pack/plugin/ml/src/test/java/org/elasticsearch/xpack/ml/extractor/TimeFieldTests.java
Outdated
Show resolved
Hide resolved
Co-Authored-By: David Kyle <david.kyle@elastic.co>
The extractor for the primary time field of an anomaly detection job is automatically created as a TimeField without checking its field caps response, so earlier changes in this PR did enough to make that scenario work for "date_nanos" fields. However, the main field type detection code was still only accepting "date" to create a TimeField. This change makes it accept either or both of "date" and "date_nanos".
…droberts195/elasticsearch into make_datafeed_work_with_nanosecond_time
Profiling showed the low level version is 3 times faster and we do time parsing for every single document fed to a job.
droberts195
added a commit
that referenced
this pull request
Jan 21, 2020
Allows ML datafeeds to work with time fields that have the "date_nanos" type _and make use of the extra precision_. (Previously datafeeds only worked with time fields that were exact multiples of milliseconds. So datafeeds would work with "date_nanos" only if the extra precision over "date" was not used.) Relates #49889
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allows ML datafeeds to work with time fields that have
the "date_nanos" type and make use of the extra precision.
(Previously datafeeds only worked with time fields that were
exact multiples of milliseconds. So datafeeds would work
with "date_nanos" only if the extra precision over "date" was
not used.)
Relates #49889