Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix date rounding for date math parsing #90458

Merged
merged 8 commits into from
Oct 4, 2022

Conversation

pgomulka
Copy link
Contributor

@pgomulka pgomulka commented Sep 28, 2022

in #89693 the rounding logic was only applied when a field was present on a pattern. This is incorrect as for dates like "2020" we want to default to "2020-01-01T23:59:59.999..." when rounding is enabled.

This commit always applies monthOfYear or dayofMonth defaulting (when rounding enabled) except when the dayOfYear is set
closes #90187

@pgomulka pgomulka added v7.17.7 v8.5.1 >bug :Core/Infra/Core Core issues without another label labels Sep 29, 2022
@elasticsearchmachine
Copy link
Collaborator

Hi @pgomulka, I've created a changelog YAML for you.

@pgomulka pgomulka marked this pull request as ready for review September 29, 2022 12:36
@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Sep 29, 2022
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

range:
date:
gte: 1500 #1500-01-01T00:00:00
lte: 1500 #1500-01-01T23:59:59
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is right? If you're asking for things in the range 1500 to 1500, you expect to find stuff on the 31st December 1500, but this is rounding to midnight on the 1st January

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but did we had that behaviour in the past? It might be a bug on its own, but I thought that this is how it always worked

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing on a 7x install it seems you're right, this has always been broken. But it's still a bug. Is it simple to fix here or should we merge this PR as-is and work on that separately?

Copy link
Contributor Author

@pgomulka pgomulka Oct 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree this is a bug (or a missing feature at least). I think this would require some more work. It would make 2 cases for rounding (up and down) I am happy to raise an issue where we could gather all the use cases

#90605

@pgomulka
Copy link
Contributor Author

@elasticmachine update branch

Copy link
Contributor

@grcevski grcevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

@grcevski
Copy link
Contributor

I wonder if we should try to make this fix in the 8.4.3 release too? Is it too late?

@pgomulka
Copy link
Contributor Author

pgomulka commented Oct 3, 2022

I wonder if we should try to make this fix in the 8.4.3 release too? Is it too late?

I think it is too late now.

Copy link
Contributor

@romseygeek romseygeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pgomulka pgomulka merged commit 3f3a95e into elastic:main Oct 4, 2022
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 4, 2022
in elastic#89693 the rounding logic was only applied when a field was present on a pattern. This is incorrect as for dates like "2020" we want to default to "2020-01-01T23:59:59.999..." when rounding is enabled.

This commit always applies monthOfYear or dayofMonth defaulting (when rounding enabled) except when the dayOfYear is set
closes elastic#90187

(cherry picked from commit 3f3a95e)
@pgomulka
Copy link
Contributor Author

pgomulka commented Oct 4, 2022

💚 All backports created successfully

Status Branch Result
8.5
7.17

Questions ?

Please refer to the Backport tool documentation

pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 4, 2022
in elastic#89693 the rounding logic was only applied when a field was present on a pattern. This is incorrect as for dates like "2020" we want to default to "2020-01-01T23:59:59.999..." when rounding is enabled.

This commit always applies monthOfYear or dayofMonth defaulting (when rounding enabled) except when the dayOfYear is set
closes elastic#90187

(cherry picked from commit 3f3a95e)

# Conflicts:
#	server/src/test/java/org/elasticsearch/common/time/DateFormattersTests.java
pgomulka added a commit that referenced this pull request Oct 4, 2022
in #89693 the rounding logic was only applied when a field was present on a pattern. This is incorrect as for dates like "2020" we want to default to "2020-01-01T23:59:59.999..." when rounding is enabled.

This commit always applies monthOfYear or dayofMonth defaulting (when rounding enabled) except when the dayOfYear is set
closes #90187

backports #90458
pgomulka added a commit that referenced this pull request Oct 4, 2022
* Fix date rounding for date math parsing (#90458)

in #89693 the rounding logic was only applied when a field was present on a pattern. This is incorrect as for dates like "2020" we want to default to "2020-01-01T23:59:59.999..." when rounding is enabled.

This commit always applies monthOfYear or dayofMonth defaulting (when rounding enabled) except when the dayOfYear is set
closes #90187
backport(#90458)
(cherry picked from commit 3f3a95e)
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 4, 2022
the test should be enabled as the fix is applied in 7.x branch

relates elastic#90458
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 4, 2022
the elastic#90458 has been backported to all branches so the bwc testing can be enable for this
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 5, 2022
the elastic#90458 has been backported to all branches so the bwc testing can be enable for this
tests were incorrectly relying on sort order. Added sort to make it deterministic

closes elastic#90668
pgomulka added a commit that referenced this pull request Oct 5, 2022
the #90458 has been backported to all branches so the bwc testing can be enable for this tests were incorrectly relying on sort order. Added sort to make it deterministic

closes #90668
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 5, 2022
the elastic#90458 has been backported to all branches so the bwc testing can be enable for this tests were incorrectly relying on sort order. Added sort to make it deterministic

closes elastic#90668
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 5, 2022
the elastic#90458 has been backported to all branches so the bwc testing can be enable for this tests were incorrectly relying on sort order. Added sort to make it deterministic

closes elastic#90668
pgomulka added a commit that referenced this pull request Oct 6, 2022
…90681) (#90690)

the #90458 has been backported to all branches so the bwc testing can be enable for this tests were incorrectly relying on sort order. Added sort to make it deterministic

closes #90668
pgomulka added a commit that referenced this pull request Oct 6, 2022
… (#90689)

the #90458 has been backported to all branches so the bwc testing can be enable for this tests were incorrectly relying on sort order. Added sort to make it deterministic

closes #90668
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 13, 2022
…tic#90633)

in elastic#89693 the rounding logic was only applied when a field was present on a pattern. This is incorrect as for dates like "2020" we want to default to "2020-01-01T23:59:59.999..." when rounding is enabled.

This commit always applies monthOfYear or dayofMonth defaulting (when rounding enabled) except when the dayOfYear is set
closes elastic#90187

backports elastic#90458
pgomulka added a commit that referenced this pull request Oct 13, 2022
in #89693 the rounding logic was only applied when a field was present on a pattern. This is incorrect as for dates like "2020" we want to default to "2020-01-01T23:59:59.999..." when rounding is enabled.

This commit always applies monthOfYear or dayofMonth defaulting (when rounding enabled) except when the dayOfYear is set
closes #90187

backports #90458
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Oct 31, 2022
…elastic#90690)

the elastic#90458 has been backported to all branches so the bwc testing can be enable for this tests were incorrectly relying on sort order. Added sort to make it deterministic

closes elastic#90668
pgomulka added a commit that referenced this pull request Oct 31, 2022
… (#91201)

the #90458 has been backported to all branches so the bwc testing can be enable for this tests were incorrectly relying on sort order. Added sort to make it deterministic

close #90783
@csoulios csoulios added v8.5.0 and removed v8.5.1 labels Nov 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Core/Infra/Core Core issues without another label Team:Core/Infra Meta label for core/infra team v7.17.7 v8.5.0 v8.6.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Match query on date property with format 'uuuu' (year) acts as a range query for years below 1971
6 participants