Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default floating point bands to nodata=nan when writing COGs. #1602

Merged
merged 8 commits into from
Jun 20, 2024

Conversation

SpacemanPaul
Copy link
Contributor

@SpacemanPaul SpacemanPaul commented Jun 20, 2024

Reason for this pull request

Issue #1595

Proposed changes


📚 Documentation preview 📚: https://datacube-core--1602.org.readthedocs.build/en/1602/

Copy link

codecov bot commented Jun 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.78%. Comparing base (8b46c22) to head (783124f).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #1602   +/-   ##
========================================
  Coverage    91.78%   91.78%           
========================================
  Files          132      132           
  Lines        14637    14640    +3     
========================================
+ Hits         13434    13437    +3     
  Misses        1203     1203           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@SpacemanPaul SpacemanPaul marked this pull request as ready for review June 20, 2024 05:03
Copy link
Contributor

@robbibt robbibt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is working nicely, thanks @SpacemanPaul! If I try and write out a floating point array with no nodata attribute, I now get nodata=nan in the COG:

import datacube
from datacube.utils.cog import write_cog
dc = datacube.Datacube()

ds = dc.load(
    product="ga_ls8c_ard_3",
    measurements=["nbart_red"],
    time=("2019-01-10", "2019-01-15"),
    x=(149.036528545, 149.233835134),
    y=(-35.198132594, -35.384897602), 
    group_by="solar_day",
)

# Convert to float
da_float = ds.nbart_red / 10000
assert "nodata" not in da_float.attrs

# Export to file
write_cog(da_float, "test_missingnodata.tif", overwrite=True)
!gdalinfo test_missingnodata.tif

image

I also get nodata=nan in the COG if it is explicitly declared on the Xarray too:

import numpy as np
da_float.attrs["nodata"] = np.nan

# Export to file
write_cog(da_float, "test_nodatanan.tif", overwrite=True)
!gdalinfo test_nodatanan.tif

image

And custom nodata is still preserved if provided on the Xarray:

da_float.attrs["nodata"] = -999

# Export to file
write_cog(da_float, "test_nodatacustom.tif", overwrite=True)
!gdalinfo test_nodatacustom.tif

image

@SpacemanPaul SpacemanPaul merged commit 238c381 into develop Jun 20, 2024
31 checks passed
@SpacemanPaul SpacemanPaul deleted the cog-float-nan-nodata branch June 20, 2024 06:00
SpacemanPaul added a commit that referenced this pull request Jun 28, 2024
* Update test_extra_dimensions to work with numpy 1.x and numpy 2.x

* Behaviour of num2numpy function changes between numpy 1.x and numpy 2.x

* Pin moto<5 to delay rewriting of tests.

* Default float bands to nodata=nan when writing cogs.

* Test float bands default to nodata=nan when writing cogs.

* Update whats_new.rst

* Lintage and incorporate Dependabot's #1600.

* Add ESRI to wordlist.txt (roll eyes).
SpacemanPaul added a commit that referenced this pull request Jul 1, 2024
* Update test_extra_dimensions to work with numpy 1.x and numpy 2.x

* Behaviour of num2numpy function changes between numpy 1.x and numpy 2.x

* Pin moto<5 to delay rewriting of tests.

* Default float bands to nodata=nan when writing cogs.

* Test float bands default to nodata=nan when writing cogs.

* Update whats_new.rst

* Lintage and incorporate Dependabot's #1600.

* Add ESRI to wordlist.txt (roll eyes).
SpacemanPaul added a commit that referenced this pull request Jul 1, 2024
* Update test_extra_dimensions to work with numpy 1.x and numpy 2.x

* Behaviour of num2numpy function changes between numpy 1.x and numpy 2.x

* Pin moto<5 to delay rewriting of tests.

* Default float bands to nodata=nan when writing cogs.

* Test float bands default to nodata=nan when writing cogs.

* Update whats_new.rst

* Lintage and incorporate Dependabot's #1600.

* Add ESRI to wordlist.txt (roll eyes).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants