"Unexpected number of scanlines!" when reading AVHRR GAC data

[yukaribbba]

Describe the bug
I got this error loading a GAC dataset from AVHRR-2/NOAA-11. Pygac is under my conda env. The dataset comes from NOAA CLASS and it's a part of the whole swath. I've tried several datasets and all failed due to this error.
Here's one for test:
https://1drv.ms/u/s!AjWORUATthfgmHNvtjUxDcqaL7X8?e=HvZdQ0

To Reproduce

#!/usr/bin/env python
from satpy import Scene, find_files_and_readers
from pyresample import get_area_def
files = find_files_and_readers(base_dir="/mnt/hgfs/Downloads/AVHRR",
                               reader='avhrr_l1b_gaclac')
scn = Scene(filenames=files)
scn.load(['1'])

**Expected behavior**
No error

**Actual results**
/home/ll/anaconda3/lib/python3.8/site-packages/satpy/resample.py:150: UserWarning: Gradient search resampler not available, upgrade Pyresample.
  warnings.warn('Gradient search resampler not available, upgrade Pyresample.')
/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py:261: RuntimeWarning: Unexpected number of scanlines!
  warnings.warn("Unexpected number of scanlines!",
Failed to load DataID(name='latitude', resolution=1050, modifiers=()) from <GACLACFile: '/mnt/hgfs/Downloads/AVHRR/NSS.GHRR.NH.D91164.S0423.E0616.B1399091.WI'>
Traceback (most recent call last):
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 687, in _load_dataset
    projectable = fh.get_dataset(dsid, ds_info)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 132, in get_dataset
    self.read_raw_data()
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 126, in read_raw_data
    self.reader.read(self.filename)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/pod_reader.py", line 291, in read
    self.spacecraft_name = self.spacecraft_names[self.spacecraft_id]
KeyError: 108
Could not load dataset 'DataID(name='latitude', resolution=1050, modifiers=())': "Could not load DataID(name='latitude', resolution=1050, modifiers=()) from any provided files"
Traceback (most recent call last):
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 828, in _load_dataset_with_area
    ds = self._load_dataset_data(file_handlers, dsid, **kwargs)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 711, in _load_dataset_data
    proj = self._load_dataset(dsid, ds_info, file_handlers, **kwargs)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 696, in _load_dataset
    raise KeyError(
KeyError: "Could not load DataID(name='latitude', resolution=1050, modifiers=()) from any provided files"
Failed to load DataID(name='longitude', resolution=1050, modifiers=()) from <GACLACFile: '/mnt/hgfs/Downloads/AVHRR/NSS.GHRR.NH.D91164.S0423.E0616.B1399091.WI'>
Traceback (most recent call last):
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 687, in _load_dataset
    projectable = fh.get_dataset(dsid, ds_info)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 138, in get_dataset
    data, _ = self.reader.get_lonlat()
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 612, in get_lonlat
    self.update_meta_data()
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 556, in update_meta_data
    self.meta_data['calib_coeffs_version'] = self.calibration.version
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 149, in calibration
    calibration = Calibrator(
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/calibration.py", line 84, in __new__
    defaults = cls.default_coeffs[spacecraft]
KeyError: None
Could not load dataset 'DataID(name='longitude', resolution=1050, modifiers=())': "Could not load DataID(name='longitude', resolution=1050, modifiers=()) from any provided files"
Traceback (most recent call last):
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 828, in _load_dataset_with_area
    ds = self._load_dataset_data(file_handlers, dsid, **kwargs)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 711, in _load_dataset_data
    proj = self._load_dataset(dsid, ds_info, file_handlers, **kwargs)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 696, in _load_dataset
    raise KeyError(
KeyError: "Could not load DataID(name='longitude', resolution=1050, modifiers=()) from any provided files"
Failed to load coordinates for 'DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=())'
Failed to load DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=()) from <GACLACFile: '/mnt/hgfs/Downloads/AVHRR/NSS.GHRR.NH.D91164.S0423.E0616.B1399091.WI'>
Traceback (most recent call last):
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 687, in _load_dataset
    projectable = fh.get_dataset(dsid, ds_info)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 160, in get_dataset
    data = self._get_channel(key)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 259, in _get_channel
    self.calib_channels = self.reader.get_calibrated_channels()
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 562, in get_calibrated_channels
    self.update_meta_data()
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 556, in update_meta_data
    self.meta_data['calib_coeffs_version'] = self.calibration.version
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 149, in calibration
    calibration = Calibrator(
  File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/calibration.py", line 84, in __new__
    defaults = cls.default_coeffs[spacecraft]
KeyError: None
Could not load dataset 'DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=())': "Could not load DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=()) from any provided files"
Traceback (most recent call last):
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 828, in _load_dataset_with_area
    ds = self._load_dataset_data(file_handlers, dsid, **kwargs)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 711, in _load_dataset_data
    proj = self._load_dataset(dsid, ds_info, file_handlers, **kwargs)
  File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 696, in _load_dataset
    raise KeyError(
KeyError: "Could not load DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=()) from any provided files"
The following datasets were not created and may require resampling to be generated: DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=())

**Environment Info:**
 - OS: Ubuntu 20.04 64bit
 - Satpy Version: 0.23
 - PyResample Version: 1.16
 - Readers and writers dependencies (when relevant): [run `from satpy.config import check_satpy; check_satpy()`]
Readers
=======
abi_l1b:  ok
abi_l1b_scmi:  ok
abi_l2_nc:  ok
acspo:  ok
agri_l1:  ok
ahi_hrit:  ok
ahi_hsd:  ok
ahi_l1b_gridded_bin:  ok
ami_l1b:  ok
amsr2_l1b:  ok
amsr2_l2:  ok
avhrr_l1b_aapp:  ok
avhrr_l1b_eps:  ok
avhrr_l1b_gaclac:  ok
avhrr_l1b_hrpt:  cannot find module 'satpy.readers.hrpt' (cannot import name 'Dataset' from 'satpy.dataset' (/home/ll/anaconda3/lib/python3.8/site-packages/satpy/dataset/__init__.py))
caliop_l2_cloud:  cannot find module 'satpy.readers.caliop_l2_cloud' (No module named 'pyhdf')
clavrx:  cannot find module 'satpy.readers.clavrx' (No module named 'pyhdf')
cmsaf-claas2_l2_nc:  ok
electrol_hrit:  ok
fci_l1c_fdhsi:  ok
fci_l2_nc:  ok
generic_image:  ok
geocat:  ok
ghrsst_l3c_sst:  cannot find module 'satpy.readers.ghrsst_l3c_sst' (cannot import name 'Dataset' from 'satpy.dataset' (/home/ll/anaconda3/lib/python3.8/site-packages/satpy/dataset/__init__.py))
glm_l2:  ok
goes-imager_hrit:  ok
goes-imager_nc:  ok
gpm_imerg:  ok
grib:  cannot find module 'satpy.readers.grib' (No module named 'pygrib')
hsaf_grib:  cannot find module 'satpy.readers.hsaf_grib' (No module named 'pygrib')
hy2_scat_l2b_h5:  ok
iasi_l2:  ok
iasi_l2_so2_bufr:  cannot find module 'satpy.readers.iasi_l2_so2_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n           Error: ', ModuleNotFoundError("No module named 'eccodes'")))
jami_hrit:  ok
li_l2:  cannot find module 'satpy.readers.li_l2' (No module named 'h5netcdf')
maia:  ok
mersi2_l1b:  ok
mimicTPW2_comp:  ok
modis_l1b:  cannot find module 'satpy.readers.modis_l1b' (No module named 'pyhdf')
modis_l2:  cannot find module 'satpy.readers.modis_l2' (No module named 'pyhdf')
msi_safe:  ok
mtsat2-imager_hrit:  ok
nucaps:  ok
nwcsaf-geo:  ok
nwcsaf-msg2013-hdf5:  ok
nwcsaf-pps_nc:  ok
olci_l1b:  ok
olci_l2:  ok
omps_edr:  ok
safe_sar_l2_ocn:  ok
sar-c_safe:  ok
scatsat1_l2b:  cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/home/ll/anaconda3/lib/python3.8/site-packages/satpy/dataset/__init__.py))
seviri_l1b_hrit:  ok
seviri_l1b_icare:  cannot find module 'satpy.readers.seviri_l1b_icare' (No module named 'pyhdf')
seviri_l1b_native:  ok
seviri_l1b_nc:  ok
seviri_l2_bufr:  cannot find module 'satpy.readers.seviri_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
seviri_l2_grib:  cannot find module 'satpy.readers.seviri_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
slstr_l1b:  ok
slstr_l2:  ok
smos_l2_wind:  ok
tropomi_l2:  ok
vaisala_gld360:  ok
vii_l1b_nc:  ok
vii_l2_nc:  ok
viirs_compact:  ok
viirs_edr_active_fires:  ok
viirs_edr_flood:  cannot find module 'satpy.readers.viirs_edr_flood' (No module named 'pyhdf')
viirs_l1b:  ok
viirs_sdr:  ok
virr_l1b:  ok

Writers
=======
cf:  ok
geotiff:  ok
mitiff:  ok
ninjotiff:  cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff')
scmi:  ok
simple_image:  ok

Extras
======
cartopy:  No module named 'cartopy'
geoviews:  No module named 'geoviews'

[mraspaud]

This looks like a lot of errors. Could it be that you didn't remove the archive header from the data when you ordered it? @sfinkens @carloshorn any idea?

[yukaribbba]

@mraspaud I tried two versions (with/without header) of the same dataset. Both failed. But errors are different.
With archive header:
https://drive.google.com/drive/folders/1DzUPfRN1cE0_GYgbytJykZm0Ke84RUJC?usp=sharing

/home/ll/anaconda3/lib/python3.8/site-packages/satpy/resample.py:150: UserWarning: Gradient search resampler not available, upgrade Pyresample.
warnings.warn('Gradient search resampler not available, upgrade Pyresample.')
/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py:261: RuntimeWarning: Unexpected number of scanlines!
warnings.warn("Unexpected number of scanlines!",
Failed to load DataID(name='longitude', resolution=1050, modifiers=()) from <GACLACFile: '/mnt/hgfs/Downloads/AVHRR/NSS.GHRR.NH.D91163.S0434.E0628.B1397677.wi'>
Traceback (most recent call last):
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 687, in _load_dataset
projectable = fh.get_dataset(dsid, ds_info)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 132, in get_dataset
self.read_raw_data()
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 126, in read_raw_data
self.reader.read(self.filename)
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/pod_reader.py", line 291, in read
self.spacecraft_name = self.spacecraft_names[self.spacecraft_id]
KeyError: 108
Could not load dataset 'DataID(name='longitude', resolution=1050, modifiers=())': "Could not load DataID(name='longitude', resolution=1050, modifiers=()) from any provided files"
Traceback (most recent call last):
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 828, in _load_dataset_with_area
ds = self._load_dataset_data(file_handlers, dsid, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 711, in _load_dataset_data
proj = self._load_dataset(dsid, ds_info, file_handlers, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 696, in _load_dataset
raise KeyError(
KeyError: "Could not load DataID(name='longitude', resolution=1050, modifiers=()) from any provided files"
Failed to load DataID(name='latitude', resolution=1050, modifiers=()) from <GACLACFile: '/mnt/hgfs/Downloads/AVHRR/NSS.GHRR.NH.D91163.S0434.E0628.B1397677.wi'>
Traceback (most recent call last):
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 687, in _load_dataset
projectable = fh.get_dataset(dsid, ds_info)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 136, in get_dataset
_, data = self.reader.get_lonlat()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 612, in get_lonlat
self.update_meta_data()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 556, in update_meta_data
self.meta_data['calib_coeffs_version'] = self.calibration.version
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 149, in calibration
calibration = Calibrator(
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/calibration.py", line 84, in new
defaults = cls.default_coeffs[spacecraft]
KeyError: None
Could not load dataset 'DataID(name='latitude', resolution=1050, modifiers=())': "Could not load DataID(name='latitude', resolution=1050, modifiers=()) from any provided files"
Traceback (most recent call last):
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 828, in _load_dataset_with_area
ds = self._load_dataset_data(file_handlers, dsid, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 711, in _load_dataset_data
proj = self._load_dataset(dsid, ds_info, file_handlers, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 696, in _load_dataset
raise KeyError(
KeyError: "Could not load DataID(name='latitude', resolution=1050, modifiers=()) from any provided files"
Failed to load coordinates for 'DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=())'
Failed to load DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=()) from <GACLACFile: '/mnt/hgfs/Downloads/AVHRR/NSS.GHRR.NH.D91163.S0434.E0628.B1397677.wi'>
Traceback (most recent call last):
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 687, in _load_dataset
projectable = fh.get_dataset(dsid, ds_info)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 160, in get_dataset
data = self._get_channel(key)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 259, in _get_channel
self.calib_channels = self.reader.get_calibrated_channels()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 562, in get_calibrated_channels
self.update_meta_data()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 556, in update_meta_data
self.meta_data['calib_coeffs_version'] = self.calibration.version
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 149, in calibration
calibration = Calibrator(
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/calibration.py", line 84, in new
defaults = cls.default_coeffs[spacecraft]
KeyError: None
Could not load dataset 'DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=())': "Could not load DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=()) from any provided files"
Traceback (most recent call last):
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 828, in _load_dataset_with_area
ds = self._load_dataset_data(file_handlers, dsid, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 711, in _load_dataset_data
proj = self._load_dataset(dsid, ds_info, file_handlers, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 696, in _load_dataset
raise KeyError(
KeyError: "Could not load DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=()) from any provided files"
The following datasets were not created and may require resampling to be generated: DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=())

Without archive header:
https://drive.google.com/drive/folders/1GfqhjPyb-OFewVjlaCmGKaNM8TBx0yZO?usp=sharing

/home/ll/anaconda3/lib/python3.8/site-packages/satpy/resample.py:150:` UserWarning: Gradient search resampler not available, upgrade Pyresample.
warnings.warn('Gradient search resampler not available, upgrade Pyresample.')
/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py:261: RuntimeWarning: Unexpected number of scanlines!
warnings.warn("Unexpected number of scanlines!",
/home/ll/anaconda3/lib/python3.8/site-packages/geotiepoints/geointerpolator.py:101: RuntimeWarning: invalid value encountered in arccos
90 - rad2deg(arccos(z__/EARTH_RADIUS)),
/home/ll/anaconda3/lib/python3.8/site-packages/geotiepoints/geointerpolator.py:103: RuntimeWarning: invalid value encountered in arcsin
(90 - rad2deg(arcsin(sqrt(x__ ** 2 + y__ ** 2)
Traceback (most recent call last):
File "/home/ll/Downloads/avhrr.py", line 7, in
scn.load(['1'])
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 982, in load
self._read(**kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 890, in _read
return self._read_datasets(nodes, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 732, in _read_datasets
new_datasets = reader_instance.load(ds_ids, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 918, in load
ds = self._load_dataset_with_area(dsid, coords, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 828, in _load_dataset_with_area
ds = self._load_dataset_data(file_handlers, dsid, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 711, in _load_dataset_data
proj = self._load_dataset(dsid, ds_info, file_handlers, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 687, in _load_dataset
projectable = fh.get_dataset(dsid, ds_info)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 138, in get_dataset
data, _ = self.reader.get_lonlat()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 621, in get_lonlat
self._adjust_clock_drift()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/pod_reader.py", line 460, in _adjust_clock_drift
complete_lons[self.scans["scan_line_number"] - min_idx] = self.lons
IndexError: index 7529 is out of bounds for axis 0 with size 7517

[sfinkens]

I'll try to reproduce this with a fresh copy of NSS.GHRR.NH.D91164.S0423.E0616.B1399091.WI

[sfinkens]

@yukaribbba Sorry I cannot reproduce the error. I downloaded that file from NOAA CLASS (without archive header) and was able to read it using satpy-0.23.0 and pygac-1.4.0. Did you compare the checksum with the CLASS order manifest? Maybe the file was damaged during download.

[yukaribbba]

@sfinkens Okay could you help me to upload your own test data and let me just try it? Thank you.

[sfinkens]

Sure, there you go:

https://public.cmsaf.dwd.de/data/sfinkens/avhrr_gac/NSS.GHRR.NH.D91164.S0423.E0616.B1399091.WI

sha256 checksum is:

29a098497a0b4ad195c2f2efcce9166652875e96c708f2293d55c9fc90ceb1e7  NSS.GHRR.NH.D91164.S0423.E0616.B1399091.WI

[yukaribbba]

@sfinkens Unfortunately it failed again, "TLE directory not specified". And sha256 checksum is no problem.

/home/ll/anaconda3/lib/python3.8/site-packages/satpy/resample.py:150: UserWarning: Gradient search resampler not available, upgrade Pyresample.
warnings.warn('Gradient search resampler not available, upgrade Pyresample.')
/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py:261: RuntimeWarning: Unexpected number of scanlines!
warnings.warn("Unexpected number of scanlines!",
/home/ll/anaconda3/lib/python3.8/site-packages/geotiepoints/geointerpolator.py:103: RuntimeWarning: invalid value encountered in arcsin
(90 - rad2deg(arcsin(sqrt(x__ ** 2 + y__ ** 2)
Traceback (most recent call last):
File "/home/ll/Downloads/avhrr.py", line 7, in
scn.load(['1'])
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 982, in load
self._read(**kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 890, in _read
return self._read_datasets(nodes, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 732, in _read_datasets
new_datasets = reader_instance.load(ds_ids, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 918, in load
ds = self._load_dataset_with_area(dsid, coords, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 828, in _load_dataset_with_area
ds = self._load_dataset_data(file_handlers, dsid, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 711, in _load_dataset_data
proj = self._load_dataset(dsid, ds_info, file_handlers, **kwargs)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 687, in _load_dataset
projectable = fh.get_dataset(dsid, ds_info)
File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/avhrr_l1b_gaclac.py", line 138, in get_dataset
data, _ = self.reader.get_lonlat()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 621, in get_lonlat
self._adjust_clock_drift()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/pod_reader.py", line 466, in _adjust_clock_drift
mlons, mlats = self.compute_lonlat(width=self.lats.shape[1],
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 526, in compute_lonlat
tle1, tle2 = self.get_tle_lines()
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 733, in get_tle_lines
tle_data = self.read_tle_file(self.get_tle_file())
File "/home/ll/anaconda3/lib/python3.8/site-packages/pygac/reader.py", line 709, in get_tle_file
raise RuntimeError("TLE directory not specified!")
RuntimeError: TLE directory not specified!

Does this mean something wrong with my pygac environment settings?
Here's my system env var:

export PYGAC_CONFIG_FILE=/home/ll/Downloads/pygac.cfg

and inside the pygac.cfg:

[tle]
tledir = /home/ll/gapfilled_tles
tlename = TLE_%(satname)s.txt

[output]
output_dir = /tmp
output_file_prefix = ECC_GAC

and the gapfilled_tles folder:

[sfinkens]

Pygac has been refactored lately to act more like a library. That means the config file is only used by the pygac-run command line program. In other cases (such as satpy using pygac as a library), the parameters must be passed directly to the reader instance.

In satpy you can specify these parameters via the reader_kwargs keyword argument upon scene creation. They will be passed to the pygac reader. See the second example in this example notebook. Unfortunately, the available parameters have not been documented properly, yet. I'll file an issue for that. But at least the docstring provides that information.

[yukaribbba]

@sfinkens Thanks! I did it according to your example. But the output geotiff doesn't contain any geolocation info so I can't make a reprojection.

[yukaribbba]

@sfinkens Just ignore it. No problems now. Thank you so much.

[sfinkens]

Did you load the lat/lon coordinates as well? Like so

scene.load(['1', 'latitude', 'longitude'])
scene.save_datasets(writer='geotiff')

I'm not familiar with geotiff, but at least this creates three .tif files, one for the channel data and two for the coordinates. Maybe @djhoese can help here. Alternatively, you could resample first and then save to geotiff:

resampled = scene.resample(my_area)
resampled.save_datasets(writer='geotiff')

[sfinkens]

Ah, ok. Glad you solved it! 🙂