Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gradient_search fails when resampling Himawari data #507

Closed
simonrp84 opened this issue Mar 20, 2023 · 7 comments · Fixed by #508
Closed

gradient_search fails when resampling Himawari data #507

simonrp84 opened this issue Mar 20, 2023 · 7 comments · Fixed by #508
Assignees
Labels

Comments

@simonrp84
Copy link
Member

The gradient_search method is failing when trying to resample Himawari data from one resolution to another:

Code Sample, a minimal, complete, and verifiable piece of code

from satpy import Scene
from glob import glob

scn = Scene(glob(f'{input_dir}/*.DAT', reader='ahi_hsd')
scn.load(['true_color_nocorr'])

scn2 = scn.resample(scn.coarsest_area(), resampler='gradient_search')
scn2.save_datasets()

Problem description

The above code produces the following error message:

File ~\miniconda\lib\site-packages\pyresample\resampler.py:202, in resample_blocks(func, src_area, src_arrays, dst_area, dst_arrays, chunk_size, dtype, name, fill_value, **kwargs)
    155 """Resample dask arrays blockwise.
    156 
    157 Resample_blocks applies a function blockwise to transform data from a source
   (...)
    199 
    200 """
    201 if dst_area == src_area:
--> 202     raise ValueError("Source and destination areas are identical."
    203                      " Should you be running `map_blocks` instead of `resample_blocks`?")
    205 name = _create_dask_name(name, func,
    206                          src_area, src_arrays,
    207                          dst_area, dst_arrays,
    208                          fill_value, dtype, chunk_size, kwargs)
    209 dask_graph = dict()

ValueError: Source and destination areas are identical. Should you be running `map_blocks` instead of `resample_blocks`?

If I comment out lines 202 and 203 then it runs fine, so presumably something is being incorrectly compared in that if statement. For Himawari true color, only one band has a different resolution to the others so my guess is that the comparison is being made between B01 and B02, which are the same resolution. Perhaps this should compare all input bands or something?

Expected Output

A resampled image.

Versions of Python, package at hand and relevant dependencies

Python = 3.10
Pyresample = 1.26.1 (from conda-forge).
OS: Windows 10

@mraspaud
Copy link
Member

@simonrp84 thanks for reporting the issue.

It looks like pyresample complains that the source and destination areas are the same, which is actually true for some of the channels of the true color image I suppose, and probably not for others.
My first reaction here is why not use the native resampler instead? Are you after the bilinear interpolation gradient search provides?

@simonrp84
Copy link
Member Author

simonrp84 commented Mar 20, 2023

No, this is just a simple example to prove the bug. I'm actually trying to use the true_color_with_night_ir composite, which is unable to use native as it has non-geos data included. It produces the same error.

@mraspaud
Copy link
Member

I understand. I suppose one solution will be to make this a warning rather than an error...

@djhoese
Copy link
Member

djhoese commented Mar 20, 2023

@mraspaud perhaps Satpy, if it doesn't already, should skip resampling data in these cases and instead do a .copy(deep=True) on the DataArray? Why don't we see this issue with the kdtree version?

@djhoese
Copy link
Member

djhoese commented Mar 20, 2023

Wait...are we completely wasting time with all resamplers by asking to resample data to its same projection?!?!? No. I don't believe it.

@simonrp84
Copy link
Member Author

Yeah, that was going to be my next question: If this is a warning rather than an error then what happens with the data? Sounds like it might still get 'resampled' even for the bands where there's no need...

@mraspaud
Copy link
Member

So, looking at this I can see that in the case of the kdtree resampler, the resampling will take place even if source and target areas are identical. Native works around it by nature (sizes being the same, nothing is done). Gradient search atm raises an error. Working on a PR for the latter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants