Satpy Writer 'geotiff' exists but could not be loaded #928

anikfal · 2019-10-08T14:24:44Z

Code:
from satpy import available_writers
available_writers()

Description:
Output of the above code:
Could not import writer config from: ['/home/.../satpy/etc/writers/geotiff.yaml']
Could not import writer config from: ['/home/.../satpy/etc/writers/ninjotiff.yaml']
['mitiff', 'simple_image', 'scmi', 'cf']

Environment Info:

OS: Fedora
Satpy Version: 0.16.1
PyResample Version: 1.13.0

The text was updated successfully, but these errors were encountered:

djhoese · 2019-10-08T14:43:23Z

You are most likely missing the rasterio library which is a dependency of the geotiff writer. There is also the check_satpy function mentioned in the Troubleshooting section of the Quickstart page of the satpy documentation that could help determine the problem.

@mraspaud Maybe we should make the "hidden" debug messages that likely exist more obvious in cases like this.

anikfal · 2019-10-09T17:32:53Z

Code:

from satpy.config import check_satpy
check_satpy()

Code Output:

Readers
=======
abi_l1b:  ok
abi_l1b_scmi:  ok
acspo:  ok
ahi_hrit:  ok
ahi_hsd:  ok
amsr2_l1b:  ok
avhrr_l1b_aapp:  ok
avhrr_l1b_eps:  ok
avhrr_l1b_gaclac:  cannot find module 'satpy.readers.avhrr_l1b_gaclac' (No module named 'pygac')
avhrr_l1b_hrpt:  cannot find module 'satpy.readers.hrpt' (No module named 'pygac')
caliop_l2_cloud:  ok
clavrx:  ok
electrol_hrit:  ok
fci_l1c_fdhsi:  ok
generic_image:  ok
geocat:  ok
ghrsst_l3c_sst:  ok
goes-imager_hrit:  ok
goes-imager_nc:  ok
grib:  cannot find module 'satpy.readers.grib' (No module named 'pygrib')
hsaf_grib:  cannot find module 'satpy.readers.hsaf_grib' (No module named 'pygrib')
iasi_l2:  ok
jami_hrit:  ok
li_l2:  cannot find module 'satpy.readers.li_l2' (No module named 'h5netcdf')
maia:  ok
mersi2_l1b:  ok
modis_l1b:  ok
modis_l2:  ok
msi_safe:  cannot find module 'satpy.readers.msi_safe' (No module named 'glymur')
mtsat2-imager_hrit:  ok
nc_seviri_l1b:  cannot find module 'satpy.readers.nc_seviri_l1b' (No module named 'satpy.readers.msg_base')
nucaps:  ok
nwcsaf-geo:  ok
nwcsaf-pps_nc:  ok
olci_l1b:  ok
olci_l2:  ok
omps_edr:  ok
safe_sar_l2_ocn:  ok
sar-c_safe:  ok
scatsat1_l2b:  ok
seviri_l1b_hrit:  ok
seviri_l1b_native:  ok
seviri_l1b_nc:  ok
slstr_l1b:  ok
tropomi_l2:  ok
vaisala_gld360:  ok
viirs_compact:  ok
viirs_edr_active_fires:  cannot find module 'satpy.readers.viirs_edr_active_fires' (fsspec is required to use any file-system functionality. Please install using
conda install -c conda-forge 'fsspec>=0.3.3'
or
pip install 'fsspec>=0.3.3'

Dask dataframe requirements are not installed.

Please either conda or pip install as follows:

  conda install dask                     # either conda install
  pip install dask[dataframe] --upgrade  # or pip install)
viirs_edr_flood:  ok
viirs_l1b:  ok
viirs_sdr:  ok
virr_l1b:  ok

Writers
=======
cf:  ok
geotiff:  ok
mitiff:  ok
ninjotiff:  cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff')
scmi:  ok
simple_image:  ok

Extras
======
cartopy:  ok
geoviews:  No module named 'geoviews'

I don't know what the cause of the missing modules is. As far as I remember, I used conda for installing most of the packages. However, the Writer geotiff seems to be OK, but could not be loaded.

djhoese · 2019-10-09T17:36:55Z

Try running from satpy.writers import geotiff. That should show us an import error directly. Also double check that you are running with the same python environment (no changes to PYTHONPATH, etc) for each thing that we try for this issue.

anikfal · 2019-10-09T18:12:11Z

It's interesting that from satpy.writers import geotiff didn't show any error, and this time Writer geotiff has been loaded.
Code:

from satpy import available_writers
available_writers()

Code output:

Could not import writer config from: ['/home/ah/.conda/envs/mypyngl/lib/python3.7/site-packages/satpy/etc/writers/ninjotiff.yaml']
['scmi', 'cf', 'simple_image', 'mitiff', 'geotiff']

ninjotiff.yaml still has some problems.
Anyway, sorry for the confusion. Maybe I am doing something wrong. I use conda virtual environment for python.

djhoese · 2019-10-09T18:18:16Z

Are you running this code from a notebook or interactive environment like Spyder? Maybe you installed rasterio or updated Satpy and needed to restart python?

anikfal · 2019-10-09T18:25:26Z

I run the codes in Linux Terminal. I will check the issue in another system which is identical to my system.

djhoese · 2020-01-06T17:32:57Z

@anikfal Any luck in getting this to work?

anikfal · 2020-01-14T11:38:18Z

@djhoese After installing the missing rasterio library, geotiff appears in the available libraries:

from satpy import available_writers
available_writers()
Could not import writer config from: ['/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/satpy/etc/writers/ninjotiff.yaml']
['geotiff', 'cf', 'mitiff', 'scmi', 'simple_image']

But using this writer to convert the data to Geotiff seems not much helpful:

from satpy.scene import Scene
from satpy import find_files_and_readers
from datetime import datetime
files = find_files_and_readers(base_dir='/home/ah/MSG10-sdscase/datamsg8',
                               #start_time=datetime(2019,4,1,9,00),
                               #end_time=datetime(2019,4,1,9,15),
                               reader='seviri_l1b_nc')
scn = Scene(filenames=files)
scn.load([10.8])
scn.save_datasets(writer='geotiff')

With the errors which seem to be related to the type of the projection:

/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/trollimage/xrimage.py:311: FutureWarning: This DataArray contains multi-dimensional coordinates. In the future, these coordinates will be transposed as well unless you specify transpose_coords=False.
data = data.transpose('bands', 'y', 'x')
Traceback (most recent call last):
File "1geotiff.py", line 22, in
scn.save_datasets(writer='geotiff')
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/satpy/scene.py", line 1344, in save_datasets
return writer.save_datasets(datasets, compute=compute, **save_kwargs)
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/satpy/writers/init.py", line 676, in save_datasets
results.append(self.save_dataset(ds, compute=False, **kwargs))
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/satpy/writers/init.py", line 807, in save_dataset
return self.save_image(img, filename=filename, compute=compute, fill_value=fill_value, **kwargs)
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/satpy/writers/geotiff.py", line 190, in save_image
**gdal_options)
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/trollimage/xrimage.py", line 283, in save
**format_kwargs)
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/trollimage/xrimage.py", line 370, in rio_save
r_file.open()
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/trollimage/xrimage.py", line 104, in open
self.rfile = rasterio.open(self.path, mode, **self.kwargs)
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/rasterio/env.py", line 445, in wrapper
return f(*args, **kwds)
File "/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/rasterio/init.py", line 226, in open
**kwargs)
File "rasterio/_io.pyx", line 1202, in rasterio._io.DatasetWriterBase.init
File "rasterio/_io.pyx", line 1245, in rasterio._io.DatasetWriterBase._set_crs
File "rasterio/_err.pyx", line 182, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: Only OGC WKT Projections supported for writing to GeoTIFF. PROJCRS["unknown",BASEGEOGCRS["unknown",DATUM["unknown",ELLIPSOID["unknown",6378169,295.488065897014,LENGTHUNIT["metre",1,ID["EPSG",9001]]]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8901]]],CONVERSION["unknown",METHOD["Geostationary Satellite (Sweep Y)"],PARAMETER["Longitude of natural origin",41.5,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8802]],PARAMETER["Satellite Height",35785831,LENGTHUNIT["metre",1,ID["EPSG",9001]]],PARAMETER["False easting",0,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS["Cartesian",2],AXIS["(E)",east,ORDER[1],LENGTHUNIT["metre",1,ID["EPSG",9001]]],AXIS["(N)",north,ORDER[2],LENGTHUNIT["metre",1,ID["EPSG",9001]]]] not supported.

For my case, using gdal (e.g. gdal_translate) directly, could handle the issue more effectively.

pnuu · 2020-01-14T12:31:45Z

@anikfal what is the output of print(scn[10.8].attrs['area'])? Although the log above shows the WKT version, I don't directly understand what it is saying. But my guess is that something in the area definition isn't supported.

djhoese · 2020-01-14T14:17:43Z

@anikfal What version of the GDAL C library do you have installed in your environment? What version of rasterio? pyproj? PROJ?

This error is from rasterio and is stating that the projection is not valid which is odd since it is coming from the reader.

anikfal · 2020-01-15T08:23:58Z

@pnuu Output of print(scn[10.8].attrs['area']):

Area ID: some_area_name
Description: On-the-fly area
Projection ID: geosmsg
Projection: {'a': '6378169', 'b': '6356583.8', 'h': '35785831', 'lon_0': '41.5', 'no_defs': 'None', 'proj': 'geos', 'type': 'crs', 'units': 'm', 'x_0': '0', 'y_0': '0'}
Number of columns: 1084
Number of rows: 1009
Area extent: (-664589.3012, 1648721.5396, 2584847.3274, 4676128.3339)

@djhoese Versions of the requested libraries:

gdal_translate --version
GDAL 2.4.2, released 2019/06/28

conda list | grep -i 'rasterio\|proj\|gdal'
anaconda-project          0.8.3                      py_0  
libgdal                   2.4.2                h04a3d72_6    conda-forge
proj4                     6.1.0                he751ad9_2    conda-forge
pyproj                    2.2.1            py37hc44880f_0    conda-forge
rasterio                  1.0.28           py37hdff7cfa_1    conda-forge

djhoese · 2020-01-15T14:09:16Z

Sorry, one more, what about the version of trollimage?

anikfal · 2020-01-17T08:24:27Z

@djhoese
OS: Fedora Scientific 29

conda list | grep troll
trollimage                1.10.0                    <pip>
trollsift                 0.3.2                     <pip>

It is interesting that on my laptop with the regular Fedora 31, due to the lack of a library, rasterio could not be imported.

djhoese · 2020-01-17T14:03:04Z

@anikfal There is a newer version of trollimage (1.11.0) that should fix this issue or at least handle it better. Could you try updating trollimage and see if your code works?

anikfal · 2020-01-18T10:49:02Z

@djhoese Problem solved. Thank you.
After updating trollimage to the version 1.11.0, the previous code generates the geotiff data with just a warning:

/home/ah/.conda/envs/newpyth/lib/python3.7/site-packages/trollimage/xrimage.py:427: FutureWarning: This DataArray contains multi-dimensional coordinates. In the future, these coordinates will be transposed as well unless you specify transpose_coords=False.
data = data.transpose('bands', 'y', 'x')

djhoese · 2020-01-18T13:35:28Z

Glad it works for you now. Thanks for the update.

djhoese added the question label Oct 8, 2019

djhoese self-assigned this Oct 8, 2019

djhoese closed this as completed Jan 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Satpy Writer 'geotiff' exists but could not be loaded #928

Satpy Writer 'geotiff' exists but could not be loaded #928

anikfal commented Oct 8, 2019

djhoese commented Oct 8, 2019

anikfal commented Oct 9, 2019

djhoese commented Oct 9, 2019

anikfal commented Oct 9, 2019

djhoese commented Oct 9, 2019

anikfal commented Oct 9, 2019

djhoese commented Jan 6, 2020

anikfal commented Jan 14, 2020

pnuu commented Jan 14, 2020

djhoese commented Jan 14, 2020

anikfal commented Jan 15, 2020

djhoese commented Jan 15, 2020

anikfal commented Jan 17, 2020

djhoese commented Jan 17, 2020

anikfal commented Jan 18, 2020

djhoese commented Jan 18, 2020

Satpy Writer 'geotiff' exists but could not be loaded #928

Satpy Writer 'geotiff' exists but could not be loaded #928

Comments

anikfal commented Oct 8, 2019

djhoese commented Oct 8, 2019

anikfal commented Oct 9, 2019

djhoese commented Oct 9, 2019

anikfal commented Oct 9, 2019

djhoese commented Oct 9, 2019

anikfal commented Oct 9, 2019

djhoese commented Jan 6, 2020

anikfal commented Jan 14, 2020

pnuu commented Jan 14, 2020

djhoese commented Jan 14, 2020

anikfal commented Jan 15, 2020

djhoese commented Jan 15, 2020

anikfal commented Jan 17, 2020

djhoese commented Jan 17, 2020

anikfal commented Jan 18, 2020

djhoese commented Jan 18, 2020