AGRI data fails with native resampling`

[simonrp84]

Describe the bug
The native resampler no longer works for AGRI data if the reduce_data=True option (default) is passed to the resampler.

To Reproduce

scn = Scene(files, reader='agri_fy4a_l1')  # where 'files' is some selection of AGRI L1 files
scn.load(['C01', 'solar_zenith_angle'])
scn = scn.resample(scn.coarsest_area(), resampler = 'native')

Expected behavior
The scene is correctly resampled.

Actual results

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In [2], line 19
     17 scn = Scene(files, reader='agri_l1')
     18 scn.load(['C01', 'solar_zenith_angle'])
---> 19 scn2 = scn.resample(scn.coarsest_area(), resampler = 'native')

File ~\PycharmProjects\satpy\satpy\scene.py:958, in Scene.resample(self, destination, datasets, generate, unload, resampler, reduce_data, **resample_kwargs)
    956     destination = self.finest_area(datasets)
    957 new_scn = self.copy(datasets=datasets)
--> 958 self._resampled_scene(new_scn, destination, resampler=resampler,
    959                       reduce_data=reduce_data, **resample_kwargs)
    961 # regenerate anything from the wishlist that needs it (combining
    962 # multiple resolutions, etc.)
    963 if generate:

File ~\PycharmProjects\satpy\satpy\scene.py:873, in Scene._resampled_scene(self, new_scn, destination_area, reduce_data, **resample_kwargs)
    871 kwargs = resample_kwargs.copy()
    872 kwargs['resampler'] = resamplers[source_area]
--> 873 res = resample_dataset(dataset, destination_area, **kwargs)
    874 new_datasets[ds_id] = res
    875 if ds_id in new_scn._datasets:

File ~\PycharmProjects\satpy\satpy\resample.py:1421, in resample_dataset(dataset, destination_area, **kwargs)
   1418     return dataset
   1420 fill_value = kwargs.pop('fill_value', get_fill_value(dataset))
-> 1421 new_data = resample(source_area, dataset, destination_area, fill_value=fill_value, **kwargs)
   1422 new_attrs = new_data.attrs
   1423 new_data.attrs = dataset.attrs.copy()

File ~\PycharmProjects\satpy\satpy\resample.py:1384, in resample(source_area, data, destination_area, resampler, **kwargs)
   1382     res = [resampler_instance.resample(ds, **kwargs) for ds in data]
   1383 else:
-> 1384     res = resampler_instance.resample(data, **kwargs)
   1386 return res

File ~\PycharmProjects\satpy\satpy\resample.py:983, in NativeResampler.resample(self, data, cache_dir, mask_area, **kwargs)
    981 """Run NativeResampler."""
    982 # use 'mask_area' with a default of False. It wouldn't do anything.
--> 983 return super(NativeResampler, self).resample(data,
    984                                              cache_dir=cache_dir,
    985                                              mask_area=mask_area,
    986                                              **kwargs)

File ~\PycharmProjects\satpy\satpy\resample.py:439, in BaseResampler.resample(self, data, cache_dir, mask_area, **kwargs)
    436     kwargs['mask'] = kwargs['mask'].all(dim=flat_dims)
    438 cache_id = self.precompute(cache_dir=cache_dir, **kwargs)
--> 439 return self.compute(data, cache_id=cache_id, **kwargs)

File ~\PycharmProjects\satpy\satpy\resample.py:1087, in NativeResampler.compute(self, data, expand, **kwargs)
   1084 repeats[y_axis] = y_repeats
   1085 repeats[x_axis] = x_repeats
-> 1087 d_arr = self._expand_reduce(data.data, repeats)
   1088 new_data = xr.DataArray(d_arr, dims=data.dims)
   1089 return update_resampled_coords(data, new_data, target_geo_def)

File ~\PycharmProjects\satpy\satpy\resample.py:1053, in NativeResampler._expand_reduce(cls, d_arr, repeats)
   1051     y_size = 1. / repeats[0]
   1052     x_size = 1. / repeats[1]
-> 1053     return cls._aggregate(d_arr, y_size, x_size)
   1054 raise ValueError("Must either expand or reduce in both "
   1055                  "directions")

File ~\PycharmProjects\satpy\satpy\resample.py:997, in NativeResampler._aggregate(d, y_size, x_size)
    994     raise ValueError("Can't aggregrate (reduce) data arrays with "
    995                      "more than 2 dimensions.")
    996 if not (x_size.is_integer() and y_size.is_integer()):
--> 997     raise ValueError("Aggregation factors are not integers")
    998 for agg_size, chunks in zip([y_size, x_size], d.chunks):
    999     for chunk_size in chunks:

ValueError: Aggregation factors are not integers

The error is fixed with reduce_data=False

[djhoese]

All AGRI cases or one specific one? What are the shapes of every area and every DataArray data going into this?

[djhoese]

@simonrp84 in addition to my above questions, does this fail for all AGRI data cases? FY4A and FY4B?

[simonrp84]

While making my FY-4B PR, I noticed that the COFF/LOFF and CFAC/LFAC values provided for AGRI appear to be incorrect - the per-channel area extents are subtly different and I believe that's what's causing this problem.

I've asked the CMA for their opinion and it's probably not worth diagnosing the issue further right now until we hear back from them.

[MengXin001]

Is there any progress now? I've found this problem with FY4B data except for the 1km data

[simonrp84]

@MengXin001 This might now be fixed after PR #2533 was merged, we'd welcome any feedback you have on that! Unfortunately I am away right now and don't have any AGRI data to test with...

[mraspaud]

@simonrp84 @MengXin001 did you have time to check this? can you confirm this is fixed now?

[simonrp84]

I've not had any chance to test this yet, hopefully @MengXin001 has!

[MengXin001]

Apologies for my tardiness, I've just finished testing and the issue has been fixed correctly, it works well!

[mraspaud]

Thank you for confirming @MengXin001 ! we're closing this now then.