Skip to content

Commit

Permalink
Merge pull request #85 from flow-project/install_sumo_binaries
Browse files Browse the repository at this point in the history 
Install sumo binaries

- modified the setup instructions to install sumo binaries into flow/bin. The binaries were built on my and Eugene's machines, as well as on AWS instances.
- sumo tools (e.g. traci) are installed in `setup.py` via a wheel. The wheel works for all operating systems (as far as I can tell so far)
- we need many people to test that the binaries work for them before this is merged. There may be some dependencies that won't work, and we can only really find out by testing this multiple times and finding the discrepancies
  • Loading branch information
@AboudyKreidieh
AboudyKreidieh committed Sep 5, 2018
2 parents c0683bb + dde176a commit fe872cc
Show file tree
Hide file tree
Showing 5 changed files with 186 additions and 168 deletions.
176 changes: 107 additions & 69 deletions docs/source/flow_setup.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


1. Local Installation
******************
**********************

To get Flow running, you need three things: Flow,
SUMO, and (optionally) a reinforcement learning library (RLlib/rllab).
Expand All @@ -14,36 +14,41 @@ module bugs from Python. Just install the missing module using
your OS-specific package manager / installation tool. Follow the
shell commands below to get started.

1. Dependencies
============
We begin by installing dependencies needed by the four repositories mentioned
above. **It is highly recommended that users install**
**It is highly recommended that users install**
`Anaconda <https://www.anaconda.com/download>`_ **or**
`Miniconda <https://conda.io/miniconda.html>`_
**for Python and the setup instructions will assume that you are
doing so.**

For Ubuntu 16.04:
::
a. Installing Flow and SUMO
===========================

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install cmake swig libgtest-dev python-pygame python-scipy autoconf libtool pkg-config libgdal-dev libxerces-c-dev libproj-dev libfox-1.6-dev libxml2-dev libxslt1-dev build-essential curl unzip flex bison python python-dev python3-dev
sudo pip3 install cmake cython
In this section we install Flow as well as the binaries and packages needed
to support the traffic simulator used in modeling the dynamics of traffic
networks: SUMO.

We begin by creating a conda environment and installing Flow and its
dependencies within the environment. This can be done by running the below
script.

For OSX (feel free to ignore the rllab dependencies if you don't wish to
install it):
::

# rllab dependencies
brew install swig sdl sdl_image sdl_mixer sdl_ttf portmidi
# sumo dependencies
brew install Caskroom/cask/xquartz autoconf automake pkg-config libtool gdal proj xerces-c fox
# create a conda environment
conda env create -f environment.yml
source activate flow
# install flow within the environment
python setup.py develop

2. Sumo
====
Next, we install SUMO, an open source traffic microsimulator which will be used
the update the states of vehicles, traffic lights, and other RL and
human-driven agents during the simulation process.
Next, we install the necessary SUMO binaries and python tools. The below
commands walk you through installing and building SUMO locally. Note that
if this does not work, you are recommended to point an issue on the flow-dev
message board or refer to SUMO's
`documentation <http://sumo.dlr.de/wiki/Installing/Linux_Build>`_ regarding
installing their software. In additional, if you are interested in trying a
simpler and faster set of experimental setup instructions involving installing
pre-compiled binaries, we refer you to section 1.c.

We begin by downloading's sumo's github directory:

::

Expand Down Expand Up @@ -84,66 +89,80 @@ Finally, test your sumo install and version by running the following commands
which sumo
sumo --version
sumo-gui
If you are a mac user and the above command gives you the error FXApp:openDisplay: unable to open display :0.0 make sure to open the application XQuartz.

3. Flow
====
Once sumo and the various dependencies are in place, we are ready to install a
functional version of Flow. With this, we can begin to simulate traffic within
sumo using OpenAI gym-compatible environments. Note that separate RL algorithms
will be needed to train autonomous agents within the simulation to improve
various traffic flow properties (see the sections on rllab-multiagent and
Ray/RLlib for more).
::
If you are a mac user and the above command gives you the error FXApp:openDisplay: unable to open display :0.0 make sure to open the application XQuartz.

cd ~
git clone https://github.com/flow-project/flow.git
cd flow
conda env create -f environment.yml
source activate flow
python setup.py develop
b. Testing the Installation
===========================

For linux run
::
echo 'export PYTHONPATH="$HOME/flow:$PYTHONPATH"' >> ~/.bashrc
source ~/.bashrc
Once the above modules have been successfully installed, we can test the
installation by running a few examples. Before trying to run any examples, be
sure to enter your conda environment by typing:

For mac run
::
echo 'export PYTHONPATH="$HOME/flow:$PYTHONPATH"' >> ~/.bash_profile
source ~/.bash_profile

4. Testing the Installation
========================

Once the above modules have been successfully installed, we can test the
installation by running a few examples.
source activate flow

Let’s see some traffic action:

::

python examples/sumo/sugiyama.py

Running the following should result in the loading of the SUMO GUI.
Click the run button and you should see unstable traffic form after a
few seconds, a la (Sugiyama et al, 2008). This means that you have Flow
properly configured with SUMO.
properly configured with SUMO and Flow!

c. Easy Install SUMO (optional)
===============================

In this section, we present and faster and simpler method of installing the
necessary SUMO binaries and python tools. These setup instructions are still
experimental, so any and all feedback is greatly appreciated!

In order to install everything you will need from SUMO, run one of the below
scripts from the Flow main directory. Choose the script that matches the
operating system you are running.

For Ubuntu 14.04:

::

scripts/setup_sumo_ubuntu1404.sh

For Ubuntu 16.04:

Optionally, run the unit tests:
::

nose2 -s tests/fast_tests
scripts/setup_sumo_ubuntu1604.sh

Congratulations, you now have successfully set up Flow!
For Ubuntu 18.04:

::

5. Rllab-multiagent (optional)
===========================
scripts/setup_sumo_ubuntu1804.sh

For Mac:

::

scripts/setup_sumo_osx.sh

If you are using an unsupported operating system (e.g. Arch Linux), or the
binaries provided by the above scripts are no compatible with your machine, you
will have to personally build the SUMO binary files. For more, please see
section 1.a or refer to SUMO's
`documentation <http://sumo.dlr.de/wiki/Installing/Linux_Build>`_.


d. Rllab-multiagent (optional)
==============================
Flow has been tested on a variety of RL libraries, the installation of which is
optional but may be of use when trying to execute some of the examples files
located in Flow. rllab-multiagent is one of these such libraries. In order
to install the `rllab-multiagent` library, follow the below instructions

::

cd ~
Expand All @@ -152,23 +171,28 @@ to install the `rllab-multiagent` library, follow the below instructions
python setup.py develop

For linux run

::

echo 'export PYTHONPATH="$HOME/rllab-multiagent:$PYTHONPATH"' >> ~/.bashrc
source ~/.bashrc

For mac run

::

echo 'export PYTHONPATH="$HOME/rllab-multiagent:$PYTHONPATH"' >> ~/.bash_profile
source ~/.bash_profile

6. Ray/RLlib (optional)
====================
e. Ray/RLlib (optional)
=======================
RLlib is another RL library that has been extensively tested on the Flow
repository.
First visit <https://github.com/eugenevinitsky/ray/blob/master/doc/source/installation.rst> and
install the required packages. Do NOT `pip install ray`.

The installation process for this library is as follows:

::

cd ~
Expand All @@ -180,16 +204,18 @@ required libraries as specified at
<http://ray.readthedocs.io/en/latest/installation.html> and
then follow the setup instructions.

7. Getting started (rllab-multiagent)
==================================
f. Getting started (rllab-multiagent)
=====================================

To run any of the RL examples, make sure to run

::

source activate flow
In order to test run an Flow experiment in rllab-multiagent, try the following
command:

::

python examples/rllab/stabilizing_the_ring.py
Expand All @@ -198,17 +224,19 @@ If it does not fail, this means that you have Flow properly configured with
rllab-multiagent.


8. Getting started (Ray/RLlib)
===========================
g. Getting started (Ray/RLlib)
==============================

See `getting started with RLlib <http://ray.readthedocs.io/en/latest/rllib.html#getting-started>`_ for sample commands.

To run any of the RL examples, make sure to run

::

source activate flow

In order to test run an Flow experiment in RLlib, try the following command:

::

python examples/rllib/stabilizing_the_ring.py
Expand All @@ -217,6 +245,7 @@ If it does not fail, this means that you have Flow properly configured with
RLlib.

To visualize the training progress:

::

tensorboard --logdir=~/ray_results
Expand All @@ -231,48 +260,57 @@ jobs from there.
ray teardown scripts/ray_autoscale.yaml


9. Custom configuration
====================
h. Custom configuration
=======================

You may define user-specific config parameters as follows

::

cp flow/core/config.template.py flow/core/config.py # Create template for users using pycharm


2. Remote desktop using Docker for simple utilisation of flow (recommended for testing purpose)
******************
***********************************************************************************************

1. Installation
====================
a. Installation
===============

Installation of a remote desktop and docker to get access to flow quickly

First install docker on https://www.docker.com/

In terminal

::

1° docker pull lucasfischerberkeley/flowdesktop
2° docker run -d -p 5901:5901 -p 6901:6901 -p 8888:8888 lucasfischerberkeley/flowdesktop
Go into your browser ( Firefox, Chrome, Safari)

::

1° Go to http://localhost:6901/?password=vncpassword
2° Go to Applications and open Terminal Emulator
3° For sumo: Write python flow/examples/sumo/sugiyama.py and run it
4° For rllib : Write python flow/examples/rllib/stabilizing_the_ring.py and run it
5° For rllab : source activate flow-rllab and python flow/examples/rllab/figure_eight.py ( first time, run it twice)

2. Notebooks and tutorial
====================
b. Notebooks and tutorial
=========================

In the docker desktop

::

1° Go into Terminal Emulator
2° Run jupyter notebook --NotebookApp.token=admin --ip 0.0.0.0 --allow-root

Go into your browser ( Firefox, Chrome, Safari)

::

1° go to localhost:8888/tree
2° the password is 'admin' and you can run all your notebook and tutorial
50 changes: 16 additions & 34 deletions scripts/setup_sumo_osx.sh
View file
Original file line number Diff line number Diff line change
@@ -1,40 +1,22 @@
#!/bin/bash
echo "Installing system dependencies for SUMO"
# Read in desired path
# rllab dependencies
brew install swig sdl sdl_image sdl_mixer sdl_ttf portmidi
# sumo dependencies
brew install Caskroom/cask/xquartz autoconf automake pkg-config libtool gdal proj xerces-c fox

BASH_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

echo "Downloading SUMO to $1. This may take some time."
echo "You may be prompted (twice) to authorize downloading from the repository (press (t) to temporarily accept)."
mkdir -p $1
echo "Temporarily changing directories"
pushd $1
svn checkout https://svn.code.sf.net/p/sumo/code/trunk/sumo@26784 > /dev/null
pushd sumo

echo "Patching SUMO for flow compatibility"
patch -p1 < $BASH_DIR/departure_time_issue.patch


echo "Building SUMO"
export CPPFLAGS="$CPPFLAGS -I/opt/X11/include/"
export LDFLAGS="-L/opt/X11/lib"

autoreconf -i
./configure CXX=clang++ CXXFLAGS="-stdlib=libc++ -std=gnu++11" --with-xerces=/usr/local --with-proj-gdal=/usr/local

make -j`sysctl -n hw.ncpu`
make install

echo "\n#############################\n"
echo "add $1/sumo/tools to your PYTHONPATH and set SUMO_HOME to complete the installation!\n"
echo "Installing sumo binaries"
mkdir -p $HOME/sumo_binaries/bin
pushd $HOME/sumo_binaries/bin
wget https://akreidieh.s3.amazonaws.com/sumo/flow-0.2.0/binaries-mac.tar.xz
tar -xf binaries-mac.tar.xz
rm binaries-mac.tar.xz
chmod +x *
popd
echo 'export PATH=$PATH:$HOME/sumo_binaries/bin' >> ~/.bash_profile
echo 'export SUMO_HOME=$HOME/sumo_binaries/bin' >> ~/.bash_profile

echo "This can be done by appending the following to your bash_profile:\n "
echo "export PYTHONPATH=$1/sumo/tools\n"
echo "export SUMO_HOME=\"$1/sumo\""
echo "#############################\n"
echo "Returning to flow directory"
source ~/.bash_profile

popd
popd
echo "Installing sumo tools"
pip install https://akreidieh.s3.amazonaws.com/sumo/flow-0.2.0/sumotools-0.1.0-py3-none-any.whl

0 comments on commit fe872cc

Please sign in to comment.