travis CI build failing with KeyError: 'PROJ_LIB'

[jGaboardi]

Current spaghetti build on TravisCI build is failing when installing geopandas via conda and is a known issue: conda-forge/basemap-feedstock#30.

@ocefpaf, might this be similar to the boto3 issue from over the week (#114) or could conda-forge/pyproj-feedstock#18 be the culprit?

---

======================================================================
ERROR: Failure: KeyError ('PROJ_LIB')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/imp.py", line 245, in load_module
    return load_package(name, filename)
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/imp.py", line 217, in load_package
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 697, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/travis/build/pysal/spaghetti/spaghetti/__init__.py", line 7, in <module>
    from .network import Network, PointPattern, SimulatedPointPattern, SortedEdges
  File "/home/travis/build/pysal/spaghetti/spaghetti/network.py", line 7, in <module>
    from . import util
  File "/home/travis/build/pysal/spaghetti/spaghetti/util.py", line 1, in <module>
    from libpysal import cg
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/libpysal/__init__.py", line 28, in <module>
    from . import io
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/libpysal/io/__init__.py", line 3, in <module>
    from .iohandlers import *
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/libpysal/io/iohandlers/__init__.py", line 4, in <module>
    from . import gwt
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/libpysal/io/iohandlers/gwt.py", line 3, in <module>
    from ...weights.weights import W
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/libpysal/weights/__init__.py", line 2, in <module>
    from .distance import *
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/libpysal/weights/distance.py", line 7, in <module>
    from .util import isKDTree, get_ids, get_points_array_from_shapefile,\
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/libpysal/weights/util.py", line 17, in <module>
    import geopandas as gpd
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/geopandas/__init__.py", line 1, in <module>
    from geopandas.geoseries import GeoSeries
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/geopandas/geoseries.py", line 6, in <module>
    import pyproj
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/pyproj/__init__.py", line 52, in <module>
    from pyproj.datadir import pyproj_datadir
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/site-packages/pyproj/datadir.py", line 1, in <module>
    import os; pyproj_datadir = os.environ['PROJ_LIB']
  File "/home/travis/build/pysal/spaghetti/miniconda/envs/test-env/lib/python3.5/os.py", line 725, in __getitem__
    raise KeyError(key) from None
KeyError: 'PROJ_LIB'

[ocefpaf]

You either need to change you CI setup to use envs, where that environment variable will be set for you automatically, or you need to define it yourself.

The story behind that is:

1. pyproj have a bundle proj.4 that is outdated
2. we decided to start building it against a modern external proj.4
3. pyproj writes on install time the "directions" for the proj.4 data directory

b/c of that our choices are limited we need a way to make that directory "relocatable" and only envs have an activation step. BTW, one should never really install things in conda's base env.

See here for the two suggestions on how to fix this:

conda-forge/basemap-feedstock#30 (comment)

PS: I wonder how many packages rely datadir though, it looks like pyproj itself only sets it and so far the only package I found that relies on that is basemap :-/

I'll investigate the possibility of setting that to None if PROJ_LIB is not found. Still, for correctness, I recommend you to try the suggestions above.

[jGaboardi]

@ocefpaf Thanks for the explanation!

Here is the current method in .travis.yml for before_install. I very well may be missing something, but functionally it seems to be the same as what's happening in folium. Do you have any thought about this?

[ocefpaf]

Indeed your CI looks OK. I'll have to take a closer look later.

[jGaboardi]

@ocefpaf thanks so much!

I have a PR running for spaghetti right now that with a minor adjustment to the .travis.yml (#122). I'll see how that goes and let you know.

[jGaboardi]

resolved in #122