Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libkea.so not found #220

Closed
djhoese opened this issue Aug 10, 2018 · 9 comments
Closed

libkea.so not found #220

djhoese opened this issue Aug 10, 2018 · 9 comments

Comments

@djhoese
Copy link

djhoese commented Aug 10, 2018

My jobs on travis seem to be failing because gdal can't find libkea.so. I feel like this happened a couple weeks ago with rasterio or something.

https://travis-ci.org/pytroll/satpy/jobs/414703708#L4245

Let me know if you need more info.

P.S. Looks like PIL is broken on Windows too (https://ci.appveyor.com/project/pytroll/satpy/build/1.0.456/job/sgxf3x3xpey5k38g#L367).
@ocefpaf
Copy link
Member

ocefpaf commented Aug 11, 2018

P.S. Looks like PIL is broken on Windows too

This s related to conda/conda#7626

My jobs on travis seem to be failing because gdal can't find libkea.so.

The following NEW packages will be INSTALLED:

    affine:                   2.2.1-py_0                           conda-forge
    alabaster:                0.7.11-py_3                          conda-forge
    asn1crypto:               0.24.0-py36_2                        conda-forge
    babel:                    2.6.0-py_1                           conda-forge
    behave:                   1.2.6-py36_0                         conda-forge
    bokeh:                    0.13.0-py36_0                        conda-forge
    boost-cpp:                1.67.0-h3a22d5f_0                    conda-forge
    boto3:                    1.7.74-py_0                          conda-forge
    botocore:                 1.10.74-py_0                         conda-forge
    bottleneck:               1.2.1-py36h7eb728f_1                 conda-forge
    cairo:                    1.14.12-h8948797_3                              
    cartopy:                  0.16.0-py36hfa13621_0                           
    cffi:                     1.11.5-py36h5e8e0c9_1                conda-forge
    cftime:                   1.0.0-py36h7eb728f_1                 conda-forge
    chardet:                  3.0.4-py36_3                         conda-forge
    click:                    6.7-py_1                             conda-forge
    click-plugins:            1.0.3-py_1                           conda-forge
    cligj:                    0.4.0-py36_0                         conda-forge
    cloudpickle:              0.5.3-py_0                           conda-forge
    codecov:                  2.0.15-py_1                          conda-forge
    configobj:                5.0.6-py_0                           conda-forge
    coverage:                 4.5.1-py36h470a237_1                 conda-forge
    coveralls:                1.3.0-py_1                           conda-forge
    cryptography:             2.3-py36hdffb7b8_0                   conda-forge
    cryptography-vectors:     2.3-py36_1                           conda-forge
    curl:                     7.61.0-h84994c4_0                               
    cycler:                   0.10.0-py_1                          conda-forge
    cython:                   0.28.5-py36hfc679d8_0                conda-forge
    cytoolz:                  0.9.0.1-py36h470a237_0               conda-forge
    dask:                     0.18.2-py_0                          conda-forge
    dask-core:                0.18.2-py_0                          conda-forge
    dbus:                     1.13.2-h714fa37_1                               
    distributed:              1.22.1-py36_0                        conda-forge
    docopt:                   0.6.2-py_1                           conda-forge
    docutils:                 0.14-py36_0                          conda-forge
    expat:                    2.2.5-hfc679d8_1                     conda-forge
    fontconfig:               2.13.0-h9420a91_0                               
    freetype:                 2.9.1-h6debe1e_0                     conda-forge
    freexl:                   1.0.5-hf837533_1                     conda-forge
    gdal:                     2.2.4-py36hb00a9d7_1                 conda-forge
    geos:                     3.6.2-hfc679d8_2                     conda-forge
    giflib:                   5.1.4-h470a237_0                     conda-forge
    glib:                     2.56.1-h000015b_0                               
    gst-plugins-base:         1.14.0-hbbd80ab_1                               
    gstreamer:                1.14.0-hb453b48_1                               
    h5netcdf:                 0.6.1-py_0                           conda-forge
    h5py:                     2.8.0-py36hb794570_1                 conda-forge
    hdf4:                     4.2.13-0                             conda-forge
    hdf5:                     1.10.2-hc401514_1                    conda-forge
    heapdict:                 1.0.0-py36_0                         conda-forge
    icu:                      58.2-hfc679d8_0                      conda-forge
    idna:                     2.7-py36_2                           conda-forge
    imageio:                  2.3.0-py_1                           conda-forge
    imagesize:                1.0.0-py_1                           conda-forge
    jinja2:                   2.10-py_1                            conda-forge
    jmespath:                 0.9.3-py_1                           conda-forge
    jpeg:                     9c-h470a237_0                        conda-forge
    json-c:                   0.13.1-h470a237_0                    conda-forge
    kealib:                   1.4.9-h0bee7d0_2                     conda-forge
    kiwisolver:               1.0.1-py36h2d50403_2                 conda-forge
    krb5:                     1.16.1-hbb41f41_0                    conda-forge
    libcurl:                  7.61.0-h1ad7b7a_0                               
    libdap4:                  3.19.1-0                             conda-forge
    libedit:                  3.1.20170329-haf1bffa_1              conda-forge
    libgdal:                  2.2.4-h6f639c0_1                                
    libiconv:                 1.15-h470a237_2                      conda-forge
    libkml:                   1.3.0-hccc92b1_8                     conda-forge
    libnetcdf:                4.6.1-he6cff42_5                     conda-forge
    libpng:                   1.6.35-ha92aebf_0                    conda-forge
    libpq:                    10.4-h1ad7b7a_0                                 
    libspatialite:            4.3.0a-he475c7f_19                              
    libssh2:                  1.8.0-h5b517e9_2                     conda-forge
    libtiff:                  4.0.9-he6b73bb_1                     conda-forge
    libuuid:                  1.0.3-1                              conda-forge
    libxcb:                   1.13-h470a237_2                      conda-forge
    libxml2:                  2.9.8-h422b904_3                     conda-forge
    libxslt:                  1.1.32-h88dbc4e_1                    conda-forge
    locket:                   0.2.0-py_2                           conda-forge
    lxml:                     4.2.4-py36hc9114bc_0                 conda-forge
    markupsafe:               1.0-py36_0                           conda-forge
    matplotlib:               2.2.3-py36h8e2386c_0                 conda-forge
    mock:                     2.0.0-py36_0                         conda-forge
    msgpack-python:           0.5.6-py36h2d50403_2                 conda-forge
    netcdf4:                  1.4.1-py36h62672b6_0                 conda-forge
    olefile:                  0.45.1-py_1                          conda-forge
    openjpeg:                 2.3.0-h0e734dc_3                     conda-forge
    owslib:                   0.16.0-py_0                          conda-forge
    packaging:                17.1-py_0                            conda-forge
    pandas:                   0.23.4-py36hf8a1672_0                conda-forge
    parse:                    1.8.4-py_1                           conda-forge
    parse_type:               0.4.2-py36_0                         conda-forge
    partd:                    0.3.8-py_1                           conda-forge
    pbr:                      4.2.0-py_0                           conda-forge
    pcre:                     8.42-h439df22_0                                 
    pillow:                   5.2.0-py36heded4f4_0                            
    pixman:                   0.34.0-2                             conda-forge
    poppler:                  0.65.0-h581218d_1                               
    poppler-data:             0.4.9-0                              conda-forge
    proj4:                    5.0.1-0                              conda-forge
    psutil:                   5.4.6-py36_0                         conda-forge
    pthread-stubs:            0.4-h470a237_1                       conda-forge
    pycparser:                2.18-py_1                            conda-forge
    pyepsg:                   0.3.2-py_1                           conda-forge
    pygments:                 2.2.0-py_1                           conda-forge
    pyhdf:                    0.9.0-py36_2                         conda-forge
    pykdtree:                 1.3.1-py36h7eb728f_1                 conda-forge
    pyopenssl:                18.0.0-py36_0                        conda-forge
    pyparsing:                2.2.0-py_1                           conda-forge
    pyproj:                   1.9.5.1-py36_0                       conda-forge
    pyqt:                     5.6.0-py36h8210e8a_6                 conda-forge
    pyresample:               1.10.1-py36_0                        conda-forge
    pyshp:                    1.2.12-py_0                          conda-forge
    pysocks:                  1.6.8-py36_1                         conda-forge
    python-dateutil:          2.7.3-py_0                           conda-forge
    pytz:                     2018.5-py_0                          conda-forge
    pyyaml:                   3.13-py36h14c3975_0                             
    qt:                       5.6.3-h39df351_0                                
    rasterio:                 1.0.3-py36h1b5fcde_0                 conda-forge
    requests:                 2.19.1-py36_1                        conda-forge
    s3transfer:               0.1.13-py36_0                        conda-forge
    scipy:                    1.1.0-py36_blas_openblash7943236_201 conda-forge [blas_openblas]
    shapely:                  1.6.4-py36h164cb2d_1                 conda-forge
    sip:                      4.18.1-py36hf484d3e_2                           
    snowballstemmer:          1.2.1-py_1                           conda-forge
    snuggs:                   1.4.2-py36_0                                    
    sortedcontainers:         2.0.4-py_1                           conda-forge
    sphinx:                   1.7.6-py36_0                                    
    sphinxcontrib:            1.0-py36_1                                      
    sphinxcontrib-websupport: 1.1.0-py36_1                                    
    tblib:                    1.3.2-py_1                           conda-forge
    toolz:                    0.9.0-py_0                           conda-forge
    tornado:                  5.1-py36h470a237_1                   conda-forge
    urllib3:                  1.23-py36_1                          conda-forge
    util-linux:               2.21-0                                          
    xarray:                   0.10.8-py36_0                        conda-forge
    xerces-c:                 3.2.1-h5d6a6da_2                     conda-forge
    xorg-libxau:              1.0.8-h470a237_6                     conda-forge
    xorg-libxdmcp:            1.1.2-h470a237_7                     conda-forge
    yaml:                     0.1.7-h470a237_1                     conda-forge
    zict:                     0.1.3-py_0                           conda-forge

There is a bad mix of packages there, you are getting libgdal from defaults.
This is happening so often lately and it is quite hard to debug :-(

Ping @msarahan for awareness. Here is where the "strict" channel preference would really help.

@djhoese
Copy link
Author

djhoese commented Aug 11, 2018

@ocefpaf Is there anything I can do to fix this? This is with a brand new environment on a brand new (technically) virtual machine. If I add the conda-forge channel and install some packages it is pretty messed up that I can't get a working environment and it is all conda's fault (my understanding).

We've talked before about old environments sometimes needing to be reset (deleted and recreated) and that kind of makes sense in a "we're waiting for it to get better" kind of way, but this is a brand new environment. I also understand that you don't develop conda and I'm not trying to blame anyone. Could I maybe force an older version of conda that might make better decisions?

FYI I tried rerunning the jobs and same issue.

@msarahan
Copy link
Member

I don't think an old conda version will help. The problem at the heart of this matter is the state of the index metadata. We're moving towards something better, but we're in a very painful transition state. We're working hard to figure out if there is an intermediate short-term solution. Sorry for the trouble in the meantime.

djhoese added a commit to djhoese/satpy that referenced this issue Aug 13, 2018
@djhoese
Force libkea installation due to conda issues
ac6938a 
conda-forge/gdal-feedstock#220
@jorisvandenbossche jorisvandenbossche mentioned this issue Aug 14, 2018
Closed
@djhoese djhoese mentioned this issue Aug 14, 2018
Closed
djhoese added a commit to pytroll/satpy that referenced this issue Aug 14, 2018
@djhoese
Fix enhancement config loading when yaml file is empty (#387)
4eeb646 
* Fix enhancement config loading when yaml file is empty

* Force libkea installation due to conda issues

conda-forge/gdal-feedstock#220

* Use kealib instead of libkea and drop krb5 fix

* Remove kealib test because it didn't fix travis

* Try new conda hack to get a usable gdal environment
@ocefpaf
Copy link
Member

ocefpaf commented Aug 14, 2018

@djhoese legend has that if we can get 50 👍 on this issue,

conda/conda#7656

we can get it prioritize on the conda's devs list 😉

@leouieda
Copy link
Member

I'm having the same issue on the GMT package. Have you tried adding gdal's dependencies explicitly to your environment?

@djhoese
Copy link
Author

djhoese commented Aug 15, 2018

@leouieda I tried forcing the kealib dependency and I had previously had krb5 from another time gdal was a pain. Final working solution (workaround) was to remove the defaults channel (see #7626).

@ocefpaf I'll spread the word.

@ljwolf ljwolf mentioned this issue Aug 15, 2018
Closed
@ocefpaf
Copy link
Member

ocefpaf commented Aug 15, 2018

Final working solution (workaround) was to remove the defaults channel (see #7626).

Yep. Now that we are shipping the compilers runtimes, the only thing we did not have in conda-forge, that is the preferred solution. (The other way around is OK too, remove conda-forge and use only defaults if you don't need anything from conda-forge.)

@ocefpaf I'll spread the word.

Thanks! The more 👍 we get there higher are the changes of them prioritizing it.

@ocefpaf
Copy link
Member

ocefpaf commented Sep 13, 2018

@djhoese I'm closing this b/c besides the rebuilds and the workaround to use only a single channel we cannot do much here. Hopefully with the new conda and strict channel preference things will be more stable.

@ocefpaf ocefpaf closed this as completed Sep 13, 2018
@djhoese
Copy link
Author

djhoese commented Sep 13, 2018

Sounds good. Thanks for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@msarahan @leouieda @ocefpaf @djhoese