Allow open date range in dc load and find_datasets #1443
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
May 11, 2023 07:23
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## develop #1443 +/- ##
===========================================
+ Coverage 91.77% 91.82% +0.05%
===========================================
Files 132 132
Lines 14490 14485 -5
===========================================
+ Hits 13298 13301 +3
+ Misses 1192 1184 -8
☔ View full report in Codecov by Sentry. |
Ariana-B
changed the title
Even with open bounds, dates are still inclusive of the start and end dates. Minor update to wording to make this clearer
robbibt
approved these changes
May 22, 2023
There was a problem hiding this comment.
This is fantastic @Ariana-B - I've tested every edge case I can think of and it all seems to work as intended.
The date ranges do seem to still be inclusive (e.g. (None, '2000') includes data up to 2000-12-31), so I've made a small update to the documentation to make this clearer. Please let me know if this isn't the intention! (although I think being inclusive for open ranges works fine for consistency with existing inclusive time filtering).
SpacemanPaul
added a commit
that referenced
this pull request
Jun 21, 2023
…h. (#1459) * GitHub actions fixes (#1433) * Github action fixes, backported from develop-1.9 * Updates to whats_new.rst * Capitalise "Dependabot" in whats_new.rst * Remove duplicate GHA Workflow for docs We used to deploy to netlify for docs previews, but now that's done to Read The Docs. The old Workflow should be removed. * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/adrienverge/yamllint.git: v1.30.0 → v1.31.0](https://github.com/adrienverge/yamllint.git/compare/v1.30.0...v1.31.0) * Display error message instead of help message if a required argument isn't provided * update whats_new * print error message as well as usage info * add license hook to pre-commit * update license template and instructions, and whats_new * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add > and < to lark grammar * refine logic * use timestamp 0 as lowest bound instead of hardcoded date * update whats_new * update doco * update whats_new * Allow open date range in dc load and find_datasets (#1443) * support open ended date range in query init * allow open ended time ranges in load() and find_datasets(), also simplify logic for cli * update doco and whats_new * get end of datetime.now() to avoid failing tests due to second mismatches * Minor update to documentation Even with open bounds, dates are still inclusive of the start and end dates. Minor update to wording to make this clearer --------- Co-authored-by: Ariana Barzinpour <ariana.barzinpour@ga.gov.au> Co-authored-by: Robbi Bishop-Taylor <Robbi.BishopTaylor@ga.gov.au> * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/adrienverge/yamllint.git: v1.31.0 → v1.32.0](https://github.com/adrienverge/yamllint.git/compare/v1.31.0...v1.32.0) * add archive_less_mature option to add and update * update whats_new * add warning message in memory driver * Pass X and Y Scale factors through to rasterio.warp.project. (#1450) * Pass X and Y Scale factors through to rasterio.warp.project. Update whats_new * Update PR number in whats_new.rst * Remove unused import. * Cleanup. * Cleanup. * Should probably just add it to the dictionary tbh. * Respond to Kirill's comments. * remove lineage from docs * move archive_less_mature to abstract and allow for postgres * move find dupes logic into a separate function * update whats_new * allow for a bit of leniency in datetime comparison when searching for less mature, add test case * update whats_new * properly add new files * fix failing tests * refactor doc_to_ds without adding dataset logic * Add missing PR's to whats_new.rst and prepare for 1.8.13 release. (#1453) * Fix gha pypi publishing condition (#1454) * fix gha pypi publishing condition * update whats_new --------- Co-authored-by: Ariana Barzinpour <ariana.barzinpour@ga.gov.au> * update ubntu installation instructions * update wordlist * update readme * update wordlist again * add a bit more info on db env variables; other misc improvements * update barebones metadata type requirements * fix typos, update wordlist * fix some wording * update integration db names * rename agdcintegration.conf * Always use XSCALE=1,YSCALE=1 in warp. (#1457) * Use SCALEX=1,SCALEY=1 in both warp code-paths. * remove data preparation page, add links to indexing guide * fix typo, del data preparation scripts page * increase buffer to 500ms * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Damien Ayers <damien@omad.net> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Ariana Barzinpour <ariana.barzinpour@ga.gov.au> Co-authored-by: Ariana-B <40238244+Ariana-B@users.noreply.github.com> Co-authored-by: Robbi Bishop-Taylor <Robbi.BishopTaylor@ga.gov.au>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reason for this pull request
As raised by @robbibt, #183 was closed without having been resolved. The logic introduced in #1439 to support open-ended date ranges in
datacube dataset search, can be used to allow them indc.load()anddc.find_datasets()as well.Proposed changes
Handle None dates in
_time_to_search_dimsrather than in a separate functionCloses Allow open-ended time ranges #183
Tests added / passed
Fully documented, including
docs/about/whats_new.rstfor all changes📚 Documentation preview 📚: https://datacube-core--1443.org.readthedocs.build/en/1443/