installation issues on MacOS

[frederikgeth]

Hi!

Congratulations on this work and the paper. This is a significant contribution, thank you!

Just letting you (and other people interested in this fantastic dataset) know that I ran into a few challenges to get your code running on MacOS 10.13.6 with Python 3.6.5 (Conda). It may be worth mentioning in the readme that a few packages come from conda-forge.

I had to use this workaround to generate the pickle for the NEM zones (nem_zones.py).

Then I tried to run assemble_networks.ipynb, and had package version challenges to solve.

pywinpty and aren't required on mac I think.

UnsatisfiableError: The following specifications were found to be in conflict:

• fiona==1.7.9
• pyomo==5.3
  for now i removed the version requirement for Pyomo.

I couldn't yet figure out how to solve this conflict:

UnsatisfiableError: The following specifications were found to be in conflict:

• fiona==1.7.9 -> gdal=2.1 -> libgdal==2.1.0 -> geos=3.5
• shapely==1.6.4

In any case, I'll continue to figure out what needs to be resolved later this week.

[akxen]

Thanks very much for your feedback, it's much appreciated!

It may be worth mentioning in the readme that a few packages come from conda-forge.

Good advice. I'll update the readme to let people know conda-forge is used for a few packages.

Just letting you (and other people interested in this fantastic dataset) know that I ran into a few challenges to get your code running on MacOS 10.13.6 with Python 3.6.5 (Conda)

Sorry to hear about the difficulties your experiencing getting everything working on MacOS (I also had a few initial problems with dependencies for GeoPandas). I found osmnx to be very helpful. Over the next few days I'll try and sort this out and develop some steps that can be followed to get a virtual environment up and running.

[frederikgeth]

Thanks, awesome! Are you using python 2 or 3 yourself? Do you use pip or conda?

I really appreciate that you're releasing the Pyomo models you implemented as well. I look forward to playing with your optimisation models. In any case, I may eventually give it a go to convert the case study data to something that can be read by PowerModels.

[akxen]

Are you using python 2 or 3 yourself? Do you use pip or conda?

I'm using Python 3, and prefer to use pip. Installing Geopandas via osmnx was the exception.

I really appreciate that you're releasing the Pyomo models you implemented as well. I look forward to playing with your optimisation models.

You're very welcome. Hopefully you find them useful.

In any case, I may eventually give it a go to convert the case study data to something that can be read by PowerModels.

PowerModels looks like a great project! Let me know how it goes if you attempt the conversion.

[akxen]

OK, I tried creating a working conda environment on MacOS 10.13.6 and ran into a few problems with GeoPandas dependencies, in particular Fiona. Here are the steps I've taken to get something up and running:
(After installing Anaconda 3.7)

1. Create conda environment: conda create --name egrimod-nem-env python=3.6.5
2. Activate environment: conda activate egrimod-nem-env
3. Install Fiona using conda: conda install -c anaconda fiona
4. Install GeoPandas and Basemap using conda-forge: conda install -c conda-forge geopandas basemap
5. Install remaining packages via pip: pip install xlrd folium ipykernel matplotlib kml2geojson pyomo
6. Setup Jupyter Notebook kernel: python -m ipykernel install --user --name egrimod-nem-env --display-name "Python (egrimod-nem-env)"
7. Deactivate environment: source deactivate
8. Activate environment: source activate egrimod-nem-env

Note: When in the repo's root directory create the following folder: mkdir ./src/1_network/output/kml_to_geojson. This will create a place for converted kml files to be stored. (.gitignore prevented this folder from being created as its contents were excluded - will fix this in the next commit).

I had to use this workaround to generate the pickle for the NEM zones (nem_zones.py).

You can also try this to set the matplotlib backend.

Here's the explicit spec file that will hopefully create a working environment for MacOS 10.13.6:
spec-file.txt

And the contents of the environment.yml file which might also be useful:

channels:
  - conda-forge
  - anaconda
  - defaults
dependencies:
  - blas=1.0=mkl
  - bzip2=1.0.6=h1de35cc_5
  - cairo=1.14.12=hc4e6be7_4
  - click=7.0=py36_0
  - click-plugins=1.0.4=py36_0
  - cligj=0.5.0=py36_0
  - curl=7.61.1=ha441bb4_0
  - expat=2.2.6=h0a44026_0
  - fiona=1.7.12=py36h0dff353_0
  - fontconfig=2.13.0=h5d5b041_1
  - freetype=2.9.1=hb4e5f40_0
  - freexl=1.0.5=h1de35cc_0
  - gdal=2.2.4=py36h6440ff4_1
  - geos=3.6.2=h5470d99_2
  - gettext=0.19.8.1=h15daf44_3
  - giflib=5.1.4=h1de35cc_1
  - glib=2.56.2=hd9629dc_0
  - hdf4=4.2.13=h39711bb_2
  - hdf5=1.10.2=hfa1e0ec_1
  - icu=58.2=h4b95b61_1
  - intel-openmp=2019.0=118
  - jpeg=9b=he5867d9_2
  - json-c=0.13.1=h3efe00b_0
  - kealib=1.4.7=h40e48e4_6
  - krb5=1.16.1=h24a3359_6
  - libboost=1.67.0=hebc422b_4
  - libcurl=7.61.1=hf30b1f0_0
  - libdap4=3.19.1=h3d3e54a_0
  - libgdal=2.2.4=h7b1ea53_1
  - libgfortran=3.0.1=h93005f0_2
  - libiconv=1.15=hdd342a3_7
  - libkml=1.3.0=hbe12b63_4
  - libnetcdf=4.6.1=h4e6abe9_2
  - libpng=1.6.35=ha441bb4_0
  - libpq=10.5=hf30b1f0_0
  - libspatialite=4.3.0a=ha12ebda_19
  - libssh2=1.8.0=h322a93b_4
  - libtiff=4.0.9=hcb84e12_2
  - libuuid=1.0.3=h6bb4b03_2
  - libxml2=2.9.8=hab757c2_1
  - mkl=2019.0=118
  - mkl_fft=1.0.6=py36hb8a8100_0
  - mkl_random=1.0.1=py36h5d10147_1
  - munch=2.3.2=py36_0
  - numpy=1.15.3=py36h6a91979_0
  - numpy-base=1.15.3=py36h8a80b8c_0
  - openjpeg=2.3.0=hb95cd4c_1
  - pcre=8.42=h378b8a2_0
  - pixman=0.34.0=hca0a616_3
  - poppler=0.65.0=ha097c24_1
  - poppler-data=0.4.9=0
  - proj4=5.0.1=h1de35cc_0
  - shapely=1.6.4=py36h20de77a_0
  - six=1.11.0=py36_1
  - xerces-c=3.2.2=h44e365a_0
  - basemap=1.2.0=py36h50ae964_0
  - ca-certificates=2018.10.15=ha4d7672_0
  - certifi=2018.10.15=py36_1000
  - cycler=0.10.0=py_1
  - descartes=1.1.0=py_2
  - geopandas=0.4.0=py_1
  - kiwisolver=1.0.1=py36h2d50403_2
  - libspatialindex=1.8.5=hfc679d8_3
  - matplotlib=3.0.1=1
  - matplotlib-base=3.0.1=py36h45c993b_1
  - openssl=1.0.2p=h470a237_1
  - pandas=0.23.4=py36hf8a1672_0
  - psycopg2=2.7.6.1=py36hdffb7b8_0
  - pyparsing=2.3.0=py_0
  - pysal=1.14.4.post2=py36_1001
  - pyshp=2.0.0=py_0
  - python-dateutil=2.7.5=py_0
  - pytz=2018.7=py_0
  - rtree=0.8.3=py36_1000
  - sqlalchemy=1.2.14=py36h470a237_0
  - tornado=5.1.1=py36h470a237_0
  - libcxx=4.0.1=hcfea43d_1
  - libcxxabi=4.0.1=hcfea43d_1
  - libedit=3.1.20170329=hb402a30_2
  - libffi=3.2.1=h475c297_4
  - ncurses=6.1=h0a44026_0
  - pip=18.1=py36_0
  - pyproj=1.9.5.1=py36h833a5d7_1
  - python=3.6.5=hc167b69_1
  - readline=7.0=h1de35cc_5
  - scipy=1.1.0=py36h28f7352_1
  - setuptools=40.6.2=py36_0
  - sqlite=3.25.3=ha441bb4_0
  - tk=8.6.8=ha441bb4_0
  - wheel=0.32.2=py36_0
  - xz=5.2.4=h1de35cc_4
  - zlib=1.2.11=hf3cbc9b_2
  - pip:
    - appdirs==1.4.3
    - appnope==0.1.0
    - backcall==0.1.0
    - branca==0.3.1
    - chardet==3.0.4
    - decorator==4.3.0
    - folium==0.6.0
    - idna==2.7
    - ipykernel==5.1.0
    - ipython==7.1.1
    - ipython-genutils==0.2.0
    - jedi==0.13.1
    - jinja2==2.10
    - jupyter-client==5.2.3
    - jupyter-core==4.4.0
    - kml2geojson==4.0.2
    - markupsafe==1.1.0
    - nose==1.3.7
    - parso==0.3.1
    - pexpect==4.6.0
    - pickleshare==0.7.5
    - ply==3.11
    - prompt-toolkit==2.0.7
    - ptyprocess==0.6.0
    - pygments==2.2.0
    - pyomo==5.5.1
    - pyutilib==5.6.3
    - pyzmq==17.1.2
    - requests==2.20.1
    - traitlets==4.3.2
    - urllib3==1.24.1
    - wcwidth==0.1.7
    - xlrd==1.1.0
prefix: /anaconda3/envs/egrimod-nem-env

[akxen]

Closing this now. Please reopen if still having problems.