Create a custom parser for parsing ISO8601 datetime variants #106486
Conversation
thecoop
force-pushed
the
fast-datetime
branch
4 times, most recently
from
f5078af
to
cce3e9c
Compare
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
server/src/test/java/org/elasticsearch/common/time/Iso8601ParserTests.java
Show resolved
Hide resolved
|
@elasticmachine update branch |
There was a problem hiding this comment.
It looks good to me.
I left few questions asking for more javadocs
can we also link the original issue? I wonder if we should also comment there why this approach was taken instead of that user's solution or 3rd party library (https://github.com/ethlo/itu/ )
@thecoop you mentioned that there are some cases where this would be broken? can we have them as testcases? We likely should document them under breaking changes section and let support team know too
server/src/main/java/org/elasticsearch/common/time/Iso8601DateTimeParser.java
Show resolved
Hide resolved
|
Hi @thecoop, I've created a changelog YAML for you. |
|
Hi @thecoop, I've updated the changelog YAML for you. |
|
@pgomulka There are test cases for the invalid strings in |
|
@elasticmachine update branch |
|
Updated with feedback from breaking-changes committee |
| if (seconds == null) return null; | ||
| if (len == pos) return ofHoursMinutesSeconds(hours, minutes, seconds, positive); | ||
|
|
||
| // there's some text left over... |
There was a problem hiding this comment.
is it be worth rephrasing to? // there's some text left over... will return error
Create and use a custom date-time parser for predefined ISO8601 variant date formatters. This resolves #102063.
This shows a ~10% increase in ingest throughput on
http_logsrally track, and ~25% increase in ingest throughput onsorally track, both tested using esbench.For the moment, it only supports
strict_date-time variants, where it is possible to specify all datetime fields. Future work could easily extend this to date formatters where some fields are forbidden, egdate_hourordate_hour_minute_second(adding an extraSet<ChronoField> forbiddenFieldsparameter and checking appropriately).I have tried very hard to keep the new parser as close as possible to the behaviour of the old java.time-based parser, however there are some edge cases that the new parser does not support. If the new parser fails to parse a string, the old parser will then be tried. A JVM option
es.datetime.java_time_parsersis added to force the use of the old parser regardless, if that is desired for any reason.