GDAL import error

[ThirstyGeo]

Issue:
The following import, detailed below, fails.
This may be a duplicate of https://github.com/conda-forge/gdal-feedstock/issues/219

(GDAL) C:\Users\User>python
Python 3.5.5 | packaged by conda-forge | (default, Jul 24 2018, 01:52:17) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

from osgeo import osr
Traceback (most recent call last):
File "", line 1, in
File "C:\Anaconda3\envs\GDAL\lib\site-packages\osgeo_init_.py", line 21, in
gdal = swig_import_helper()
File "C:\Anaconda3\envs\GDAL\lib\site-packages\osgeo_init.py", line 17, in swig_import_helper
_mod = imp.load_module('_gdal', fp, pathname, description)
File "C:\Anaconda3\envs\GDAL\lib\imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "C:\Anaconda3\envs\GDAL\lib\imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.

Previous fixes for this issue don't apply:

• package versions of gdal and libgdal don't match; problem installing 'anaconda' and 'gdal' ContinuumIO/anaconda-issues#1687
• change from default (mixture of packages from defaults and conda-forge) to packages only from conda-forge; GDAL import error #213

Environment (conda list):

$ conda list
# Name                    Version                   Build  Channel
blas                      1.0                         mkl
boost-cpp                 1.68.0            h6a4c333_1000    conda-forge
bzip2                     1.0.6             hfa6e2cd_1002    conda-forge
ca-certificates           2018.11.29           ha4d7672_0    conda-forge
certifi                   2018.8.24             py35_1001    conda-forge
curl                      7.62.0            h4496350_1000    conda-forge
expat                     2.2.5             he025d50_1002    conda-forge
freexl                    1.0.5             hd288d7e_1002    conda-forge
gdal                      2.3.2            py35h2fc6367_0    conda-forge
geos                      3.6.2             he025d50_1004    conda-forge
geotiff                   1.4.2             h0d924fb_1005    conda-forge
hdf4                      4.2.13            hf8e6fe8_1002    conda-forge
hdf5                      1.10.3            he51fdeb_1001    conda-forge
icc_rt                    2017.0.4             h97af966_0
intel-openmp              2019.1                      144
jpeg                      9c                hfa6e2cd_1001    conda-forge
kealib                    1.4.10            h90db5d1_1000    conda-forge
krb5                      1.16.2            h038dc86_1000    conda-forge
libcurl                   7.62.0            h4496350_1000    conda-forge
libgdal                   2.3.2                h4186697_0    conda-forge
libiconv                  1.15              hfa6e2cd_1003    conda-forge
libkml                    1.3.0             h4fd0f3b_1009    conda-forge
libnetcdf                 4.6.1             h852f1a4_1010    conda-forge
libpng                    1.6.36            h7602738_1000    conda-forge
libpq                     10.5              h4c6b53d_1001    conda-forge
libspatialite             4.3.0a            h582a839_1023    conda-forge
libssh2                   1.8.0             hc4dcbb0_1003    conda-forge
libtiff                   4.0.10            h36446d0_1000    conda-forge
libxml2                   2.9.8             h9ce36c8_1005    conda-forge
m2w64-expat               2.1.1                         2
m2w64-gcc-libgfortran     5.3.0                         6
m2w64-gcc-libs            5.3.0                         7
m2w64-gcc-libs-core       5.3.0                         7
m2w64-gettext             0.19.7                        2
m2w64-gmp                 6.1.0                         2
m2w64-libiconv            1.14                          6
m2w64-libwinpthread-git   5.0.0.4634.697f757               2
m2w64-xz                  5.2.2                         2
mkl                       2018.0.3                      1
mkl_fft                   1.0.9                    py35_0    conda-forge
mkl_random                1.0.1                    py35_0    conda-forge
msys2-conda-epoch         20160418                      1
numpy                     1.15.2           py35ha559c80_0
numpy-base                1.15.2           py35h8128ebf_0
openjpeg                  2.3.0             h25a6d84_1003    conda-forge
openssl                   1.0.2p            hfa6e2cd_1001    conda-forge
pcre                      8.41              h6538335_1003    conda-forge
pip                       18.0                  py35_1001    conda-forge
postgresql                10.5              hdce66b5_1001    conda-forge
proj4                     4.9.3                hfa6e2cd_8    conda-forge
python                    3.5.5                he025d50_2    conda-forge
setuptools                40.4.3                   py35_0    conda-forge
sqlite                    3.26.0            hfa6e2cd_1000    conda-forge
tk                        8.6.9             hfa6e2cd_1000    conda-forge
vc                        14                            0    conda-forge
vs2015_runtime            14.0.25420                    0    conda-forge
wheel                     0.32.0                py35_1000    conda-forge
wincertstore              0.2                   py35_1002    conda-forge
xerces-c                  3.2.0                h6538335_2    conda-forge
xz                        5.2.4             h2fa13f4_1001    conda-forge
zlib                      1.2.11            h2fa13f4_1003    conda-forge

Details about conda and system ( conda info ):

$ conda info

 active environment : GDAL
active env location : C:\Anaconda3\envs\GDAL
        shell level : 2
   user config file : C:\Users\User\.condarc

populated config files : C:\Users\User.condarc
conda version : 4.5.11
conda-build version : 3.15.1
python version : 3.7.1.final.0
base environment : C:\Anaconda3 (writable)
channel URLs : https://conda.anaconda.org/conda-forge/win-64
https://conda.anaconda.org/conda-forge/noarch
https://conda.anaconda.org/anaconda-fusion/win-64
https://conda.anaconda.org/anaconda-fusion/noarch
https://conda.anaconda.org/r/win-64
https://conda.anaconda.org/r/noarch
https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/win-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/pro/win-64
https://repo.anaconda.com/pkgs/pro/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : C:\Anaconda3\pkgs
C:\Users\User\AppData\Local\conda\conda\pkgs
envs directories : C:\Anaconda3\envs
C:\Users\User\AppData\Local\conda\conda\envs
C:\Users\User.conda\envs
platform : win-64
user-agent : conda/4.5.11 requests/2.19.1 CPython/3.7.1 Windows/10 Windows/10.0.17134
administrator : False
netrc file : None
offline mode : False

[gillins]

@ocefpaf are we still supporting Python 3.5?

[ThirstyGeo]

I'm running python 3.5.5 in that test env,
But python 3.7.1 in the original env

Python 3.5.5 | packaged by conda-forge | (default, Jul 24 2018, 01:52:17) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import gdal
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Anaconda3\envs\GDAL\lib\site-packages\gdal.py", line 2, in <module>
    from osgeo.gdal import deprecation_warn
  File "C:\Anaconda3\envs\GDAL\lib\site-packages\osgeo\__init__.py", line 21, in <module>
    _gdal = swig_import_helper()
  File "C:\Anaconda3\envs\GDAL\lib\site-packages\osgeo\__init__.py", line 17, in swig_import_helper
    _mod = imp.load_module('_gdal', fp, pathname, description)
  File "C:\Anaconda3\envs\GDAL\lib\imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "C:\Anaconda3\envs\GDAL\lib\imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: DLL load failed: The specified module could not be found.

Python 3.7.1 | packaged by conda-forge | (default, Nov 13 2018, 19:01:41) [MSC v.1900 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import gdal
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Anaconda3\lib\site-packages\gdal.py", line 2, in <module>
    from osgeo.gdal import deprecation_warn
  File "C:\Anaconda3\lib\site-packages\osgeo\__init__.py", line 21, in <module>
    _gdal = swig_import_helper()
  File "C:\Anaconda3\lib\site-packages\osgeo\__init__.py", line 17, in swig_import_helper
    _mod = imp.load_module('_gdal', fp, pathname, description)
  File "C:\Anaconda3\lib\imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "C:\Anaconda3\lib\imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
>>>

[djhoese]

As mentioned in the related anaconda issue, please use python 3.6 or above. Conda-forge no longer supports python 3.5.

[ThirstyGeo]

I have solved the problem - an incompatibility between GDAL and a runtime library.
But why is this the case?

The solution was found here https://gis.stackexchange.com/questions/294231/installing-gdal-with-anaconda?rq=1

conda create -n GDAL_vs2015 -c conda-forge gdal vs2015_runtime=14

conda activate GDAL_vs2015

python

Python 3.6.6 | packaged by conda-forge | (default, Jul 26 2018, 11:48:23) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import gdal
>>>

[ThirstyGeo]

Hi @djhoese - actually the problem exists in versions of python 3.6 (see my edited post, above)

[djhoese]

When doing these commands what version of gdal gets installed? My guess would be that an older version of gdal is installed (built with an older compiler).

[ThirstyGeo]

@djhoese - first, thanks for your input with this!

In all cases (ie, the three environments I show above) it is gdal 2.3.2 that is installed

Can you suggest what's happening, in term of conda package management?
I've tried this on a few computers, and reinstalling Anaconda3 from scratch. Each time leads to this issue. So it seems that there is a failed dependency somewhere?

[djhoese]

First, you want to make sure that all of the gdal related packages are coming from conda-forge. It should specify this in the install command's output.

Second, the main issue most likely based on the last two weeks is that the libtiff package in anaconda and on conda-forge has stopped copying the tiff.dll library to libtiff.dll. All packages that were dependent on libtiff needed to be rebuilt, but there are still some inconsistencies and poorly built packages sitting in conda-forge. I think with your output in your original issue that one of the conda-forge devs (@ocefpaf has been handling most of this issue) or maybe the @conda-forge/gdal has the ability to disable certain packages that are causing these conflicts. Judging by your list it looks like an old build of gdal is being used for some reason (I know this from the hdf5 1.10.3 version being used, new builds use 1.10.4 I think).

What happens if you take the broken environment and try conda install hdf5=1.10.4?

[ThirstyGeo]

@djhoese I've had a go at conda install hdf5=1.10.4, on two different computers.

Computer 1 had a python env using only default channels. I added conda-forge with highest priority, installed hdf5=1.10.4, and Gdal now works!

Computer 2 already had hdf5 = 1.10.3 installed. Conda wouldn't automagically downgrade relevant packages when trying to update hdf5 to 1.10.4 and resulted in this conda error:

(base) C:\Users\User>conda install hdf5=1.10.4
Solving environment: failed

UnsatisfiableError: The following specifications were found to be in conflict:
  - hdf5=1.10.4
  - pytables
  - python=3.7
Use "conda info <package>" to see the dependencies for each package.

[ocefpaf]

@ocefpaf are we still supporting Python 3.5?

We don't support 3.5 but it seems that the problem exists on 3.6 too.

@ThirstyGeo unfortunately on Windows we cannot afford to remove defaults entirely b/c conda-forge does not build scipy on Windows yet. However, make sure you have conda-forge on top of defaults in your .condarc and try this:

conda update conda --yes
conda create --name TEST python=3.7 gdal --yes
conda activate TEST
python -c "import gdal; print(dir(gdal))"

And if that does not work please send us the results of:

conda info -a

and

conda list  # for the TEST environment.

[ThirstyGeo]

@ocefpaf Yes:
conda create --name TEST python=3.7 gdal --yes
and
python -c "import gdal; print(dir(gdal))"
works fine.

[ocefpaf]

If you try with Python 3.6 please let me know how it goes. (I don't have Windows to test this things.)

[ThirstyGeo]

I have just tried with Python 3.6 — gdal loads properly.

Using the following:
conda create --name TEST python=3.6 gdal --yes
and
python -c "import gdal; print(dir(gdal))"

[ocefpaf]

Thanks for testing it out!