New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pass X and Y Scale factors through to rasterio.warp.project. #1450
Conversation
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## develop #1450 +/- ##
========================================
Coverage 91.82% 91.82%
========================================
Files 132 132
Lines 14485 14486 +1
========================================
+ Hits 13301 13302 +1
Misses 1184 1184
☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can confirm this resolves our issue! I've re-run all my previous analyses, and there's no longer any trace of interaction between resampling method and data load extent/aspect ratio/size. For example, in #1448 we generated a graph showing the difference between full and fractional loads by resampling method:
As far as I can tell, resampling should now behave the same regardless of the shape and size AOI requested by the user. This means that that we can rely on using Dask with spatial chunking, which is fantastic.
Thanks very much for this fix!
…h. (#1459) * GitHub actions fixes (#1433) * Github action fixes, backported from develop-1.9 * Updates to whats_new.rst * Capitalise "Dependabot" in whats_new.rst * Remove duplicate GHA Workflow for docs We used to deploy to netlify for docs previews, but now that's done to Read The Docs. The old Workflow should be removed. * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/adrienverge/yamllint.git: v1.30.0 → v1.31.0](https://github.com/adrienverge/yamllint.git/compare/v1.30.0...v1.31.0) * Display error message instead of help message if a required argument isn't provided * update whats_new * print error message as well as usage info * add license hook to pre-commit * update license template and instructions, and whats_new * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add > and < to lark grammar * refine logic * use timestamp 0 as lowest bound instead of hardcoded date * update whats_new * update doco * update whats_new * Allow open date range in dc load and find_datasets (#1443) * support open ended date range in query init * allow open ended time ranges in load() and find_datasets(), also simplify logic for cli * update doco and whats_new * get end of datetime.now() to avoid failing tests due to second mismatches * Minor update to documentation Even with open bounds, dates are still inclusive of the start and end dates. Minor update to wording to make this clearer --------- Co-authored-by: Ariana Barzinpour <ariana.barzinpour@ga.gov.au> Co-authored-by: Robbi Bishop-Taylor <Robbi.BishopTaylor@ga.gov.au> * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/adrienverge/yamllint.git: v1.31.0 → v1.32.0](https://github.com/adrienverge/yamllint.git/compare/v1.31.0...v1.32.0) * add archive_less_mature option to add and update * update whats_new * add warning message in memory driver * Pass X and Y Scale factors through to rasterio.warp.project. (#1450) * Pass X and Y Scale factors through to rasterio.warp.project. Update whats_new * Update PR number in whats_new.rst * Remove unused import. * Cleanup. * Cleanup. * Should probably just add it to the dictionary tbh. * Respond to Kirill's comments. * remove lineage from docs * move archive_less_mature to abstract and allow for postgres * move find dupes logic into a separate function * update whats_new * allow for a bit of leniency in datetime comparison when searching for less mature, add test case * update whats_new * properly add new files * fix failing tests * refactor doc_to_ds without adding dataset logic * Add missing PR's to whats_new.rst and prepare for 1.8.13 release. (#1453) * Fix gha pypi publishing condition (#1454) * fix gha pypi publishing condition * update whats_new --------- Co-authored-by: Ariana Barzinpour <ariana.barzinpour@ga.gov.au> * update ubntu installation instructions * update wordlist * update readme * update wordlist again * add a bit more info on db env variables; other misc improvements * update barebones metadata type requirements * fix typos, update wordlist * fix some wording * update integration db names * rename agdcintegration.conf * Always use XSCALE=1,YSCALE=1 in warp. (#1457) * Use SCALEX=1,SCALEY=1 in both warp code-paths. * remove data preparation page, add links to indexing guide * fix typo, del data preparation scripts page * increase buffer to 500ms * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Damien Ayers <damien@omad.net> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Ariana Barzinpour <ariana.barzinpour@ga.gov.au> Co-authored-by: Ariana-B <40238244+Ariana-B@users.noreply.github.com> Co-authored-by: Robbi Bishop-Taylor <Robbi.BishopTaylor@ga.gov.au>
Reason for this pull request
Attempted fix for issue #1448
Proposed changes
Pass integer scale factors through to rasterio.warp.project (as XSCALE and YSCALE parameters).
Closes Distorted data along fractional Dask chunks when loading data with
dc.load
and Dask #1448Tests added / passed
Fully documented, including
docs/about/whats_new.rst
for all changes📚 Documentation preview 📚: https://datacube-core--1450.org.readthedocs.build/en/1450/