Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

filter_temporal should be left-closed #34

Closed
soxofaan opened this issue Jan 23, 2020 · 8 comments · Fixed by Open-EO/openeo-opensearch-client#21
Closed

filter_temporal should be left-closed #34

soxofaan opened this issue Jan 23, 2020 · 8 comments · Fixed by Open-EO/openeo-opensearch-client#21
Assignees
@soxofaan @jdries
Labels
bug

Comments

@soxofaan
Copy link
Member

@lforesta noted that filter_temporal (and the related argument in load_collection) on the VITO backend does not exclude the end date of the data selection as specified in the process specification https://open-eo.github.io/openeo-api/processreference/#filter_temporal:

extent
Left-closed temporal interval, i.e. an array with exactly two elements:

  1. The first element is the start of the date and/or time interval. The specified instance in time is included in the interval.
  2. The second element is the end of the date and/or time interval. The specified instance in time is excluded from the interval.
@soxofaan
Copy link
Member Author

example:

Screenshot from 2020-01-23 10-46-56

@soxofaan soxofaan added the bug label Dec 8, 2020
@bossie
Copy link
Collaborator

bossie commented Dec 8, 2020

Pending a final solution, this is worked around in some places e.g. https://github.com/Open-EO/openeo-geotrellis-extensions/blob/9ce404a57dea9b0d60d9b9a5d559d096317fac29/openeo-geotrellis/src/main/scala/org/openeo/geotrellis/layers/OpenSearch.scala#L33-L38

@clausmichele clausmichele mentioned this issue Oct 11, 2021
Closed
@soxofaan soxofaan mentioned this issue Mar 8, 2022
Closed
bossie added a commit that referenced this issue Mar 24, 2022
@bossie
#34: optional soft errors
91a8be8
bossie added a commit that referenced this issue Mar 24, 2022
@bossie
#34: updated test to account for feature flag
66078e2
bossie added a commit that referenced this issue Mar 24, 2022
@bossie
#34: moved soft errors flag from load_collection to job_options
8d0365d
bossie added a commit that referenced this issue Mar 24, 2022
@bossie
fixed batch jobs on Creo #34
393bab1 
2022-03-24 11:07:28,481:P62:INFO:openeogeotrellis.deploy.batch_job:batch_job.py main: fail 2022-03-24 11:07:28.480997, elapsed 0:00:00.000687
Traceback (most recent call last):
 File "/opt/openeo/lib64/python3.8/site-packages/openeogeotrellis/deploy/batch_job.py", line 478, in <module>
   main(sys.argv)
 File "/opt/openeo/lib64/python3.8/site-packages/openeogeotrellis/deploy/batch_job.py", line 229, in main
   raise Exception(
Exception: usage: /opt/openeo/lib64/python3.8/site-packages/openeogeotrellis/deploy/batch_job.py <job specification input file> <job directory> <results output file name> <user log file name> <metadata file name> <api version> <dependencies> <user id> <soft errors>
@soxofaan
Copy link
Member Author

soxofaan commented Dec 7, 2022

This issue (VITO implementation using left+right closing temporal intervals) came up again in dev telco, e.g. see Open-EO/openeo-processes#331 and Open-EO/openeo-processes#394

@soxofaan
Copy link
Member Author

soxofaan commented Dec 7, 2022

I think we should address this, because apparently all other back-end implementations follow the spec (left closed intervals), so this will start giving portability issues

@soxofaan soxofaan mentioned this issue Dec 8, 2022
Closed
@jdries
Copy link
Contributor

jdries commented Jan 26, 2023

In catalog clients:
if start and end date are the same => modify end date to be at end of day
in oscars client, there is an additional filter after loading features, this needs to respect the left-closed behaviour

@jdries
Copy link
Contributor

jdries commented Jul 23, 2023

Started a branch:
https://github.com/Open-EO/openeo-opensearch-client/tree/left_closed

@jdries jdries mentioned this issue Jul 31, 2023
Merged
@jdries jdries linked a pull request Aug 23, 2023 that will close this issue
Left closed temporal filter Open-EO/openeo-opensearch-client#21
Merged
jdries added a commit to Open-EO/openeo-opensearch-client that referenced this issue Aug 23, 2023
@jdries
fix tests
352b733 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-opensearch-client that referenced this issue Aug 23, 2023
@jdries
fix case where start and end date are equal
afb33ea 
Open-EO/openeo-geopyspark-driver#34
@jdries jdries reopened this Aug 23, 2023
jdries added a commit to Open-EO/openeo-opensearch-client that referenced this issue Aug 23, 2023
@jdries
preserve behaviour when start and end date are equal
2df5849 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-opensearch-client that referenced this issue Aug 23, 2023
@jdries
add a test for querying with same start and end date
568dfb2 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-geotrellis-extensions that referenced this issue Aug 24, 2023
@jdries
date fixes
fb0c7d3 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-geotrellis-extensions that referenced this issue Aug 24, 2023
@jdries
date fixes
1c8c9a7 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-geotrellis-extensions that referenced this issue Aug 24, 2023
@jdries
date fixes
628ce47 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-geotrellis-extensions that referenced this issue Aug 24, 2023
@jdries
date fixes
9cf3b95 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-geotrellis-extensions that referenced this issue Aug 24, 2023
@jdries
date fixes
c0adf51 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-geopyspark-integrationtests that referenced this issue Aug 24, 2023
@jdries
fix dates
811065f 
Open-EO/openeo-geopyspark-driver#34
jdries added a commit to Open-EO/openeo-geopyspark-integrationtests that referenced this issue Aug 25, 2023
@jdries
fix dates
f7dc85a 
Open-EO/openeo-geopyspark-driver#34
@jdries jdries closed this as completed Aug 29, 2023
@bossie bossie mentioned this issue Sep 11, 2023
Closed
@bossie
Copy link
Collaborator

bossie commented Sep 13, 2023

Shouldn't this test (and others) have started failing since these changes?

https://github.com/Open-EO/openeo-geopyspark-integrationtests/blob/f7dc85a8ebbb717e48776df265c210941da88600/tests/test_integration.py#L175

@jdries
Copy link
Contributor

jdries commented Sep 13, 2023

No, to prevent all-out failure of cases where start and end date is the same, we treat this as a special case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@soxofaan soxofaan

@jdries jdries

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Left closed temporal filter Open-EO/openeo-opensearch-client
3 participants
@soxofaan @bossie @jdries