Integr learn plan (#67)

* minor fix docs * adding a few comments * first commit integr_learn_and_plan * 1st version integr learn and plan * fixing non trainable disc in full models * fix lr and minor modif * first prototype * adding plot + disentgling t and a + pred R + first draft planning best action * add R visualisation and fix qValues_planning * add visualisation of planning * clean * avoiding memory leaks * minor modif plots * new way of training the transition model * minor fixes * clean * fix for stability * working version * some modifs * working * fixing NN for all types of inputs and switching to an entropy + L2 reg for abstract state space * fix * working example simplest_test_PLI * some partial cleaning * test_env2 * improve plot visibility * improve figs * improve plots * modif exploration * modif PLE (with PLI) * improve plot * implementation planning d * removing useless trick roll * uniformize env so that test22 and PLE are similar * working * agent gathering_data option and first draft sticky_action * a few modifs * adding files * fix * nstep learning (work on maze with size5 with low int dim) and a few minor changes * working +- distrib laby * working explo simple laby * introduce force_features and beginning transfer * modifs * modifs * modifs * fix lr and ALE * fix parser * some clean up * changing names simple envs * ALE_env_gym * Modif summarizePerformance function and mark terminal=True when end of episode because max number steps reached * remove ALE_gym * removing theano * rename qnetworks to learning_algo * q_network renamed to more general learning_algo + necessary modifications * fix * removing unnecessary examples * some rename * remove some unnecessary comments * some cleaning * fixing doc * additional fixes * improve doc and fixes * improve docs and folder learning_algo becomes learning_algos * towards 0.4 in the docs * clean simple_maze_env * clean CRAR_keras and NN_CRAR_keras * modify shape actions in train and Dataset.randomBatch, improve doc method train in learning_algo, clean CRAR_keras, fix summarize_performance in environment and a few other doc improvements * modif doc * Pass the dataset to policies * clean comments+code, introduce clip_norm instead of unused clip_delta in all learning algos * cleaning+swith to version 0.4 * cleaning and some doc improvement * adding maze * cleaning maze + compatibility python 3 * additional cleaning * cleaning and adding comments * Cleaning and updating comments in CRAR_keras * cleaning CRAR_keras * improve doc agent * improve doc environment * fixes ALE * modifs CRAR agents (minor fixes, adaptations to ALE and cleaning) * modifs ALE * maze * update requirements * minor modifs catcher * simple maze * fix q_net_keras.py * fix travis * fixes * fixes * remove PLE * update doc
VinF · Sep 10, 2018 · b7ad427 · b7ad427
1 parent 2ec59ba
commit b7ad427
Show file tree

Hide file tree

Showing 60 changed files with 3,676 additions and 1,984 deletions.
diff --git a/README.rst b/README.rst
@@ -25,22 +25,14 @@ DeeR is a python library for Deep Reinforcement. It is build with modularity in
 Dependencies
 ============
 
-This framework is tested to work under Python 2.7, and Python 3.5. It should also work with Python 3.3 and 3.4.
+This framework is tested to work under Python 3.6.
 
-The required dependencies are NumPy >= 1.10, joblib >= 0.9. You also need theano >= 0.8 or tensorflow >= 0.9 along with the keras library.
+The required dependencies are NumPy >= 1.10, joblib >= 0.9. You also need Keras>=2.1.
 
 For running the examples, Matplotlib >= 1.1.1 is required.
 For running the atari games environment, you need to install ALE >= 0.4.
 
 Full Documentation
 ==================
 
-The documentation is available at : http://deer.readthedocs.io/
-
-Here are a few examples :
-
-.. image:: http://vincent.francois-l.be/img_GeneralDeepQRL/seaquest.gif
-   :width: 200 px
-
-.. image:: http://vincent.francois-l.be/img_GeneralDeepQRL/output7.gif
-   :width: 200 px
+The documentation is available at : http://deer.readthedocs.io/
diff --git a/ci_scripts/install.sh b/ci_scripts/install.sh
@@ -15,7 +15,7 @@ ls -l
 echo
 if [[ ! -f miniconda.sh ]]
    then
-   wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh \
+   wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh \ #Miniconda3-4.5.4-Linux-x86_64.sh \
        -O miniconda.sh
    fi
 chmod +x miniconda.sh && ./miniconda.sh -b
@@ -30,11 +30,23 @@ popd
 # Configure the conda environment and put it in the path using the
 # provided versions
 conda create -n testenv --yes python=$PYTHON_VERSION pip nose \
-      numpy=$NUMPY_VERSION scipy=$SCIPY_VERSION \
-      matplotlib theano=$THEANO_VERSION joblib
+      numpy
 
+conda install libgcc -y
 source activate testenv
-
+pip install --upgrade pip
+pip install scipy
+pip install keras
+pip install tensorflow
+pip install matplotlib
+pip install joblib
+pip install cython
+
+#if [[ "$PYTHON_VERSION" == "2.7" ]]; then
+#    pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.10.1-cp27-none-linux_x86_64.whl # tensorflow
+#elif [[ "$PYTHON_VERSION" == "3.5" ]]; then
+#    pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.10.1-cp35-cp35m-linux_x86_64.whl 
+#fi
 
 if [[ "$COVERAGE" == "true" ]]; then
     pip install coverage coveralls
@@ -44,5 +56,6 @@ python --version
 python -c "import numpy; print('numpy %s' % numpy.__version__)"
 python -c "import scipy; print('scipy %s' % scipy.__version__)"
 python -c "import theano; print('theano %s' % theano.__version__)"
+python -c "import tensorflow; print('tensorflow %s' % tensorflow.__version__)"
 
 python setup.py develop
diff --git a/ci_scripts/test.sh b/ci_scripts/test.sh
@@ -13,15 +13,7 @@ elif [[ "$EXAMPLE" == "mountain_car" ]]; then
     python run_mountain_car.py  --epochs 5
 
     pip -V pip
-    pip install --upgrade pip
-    conda install libgcc -y
-    if [[ "$PYTHON_VERSION" == "2.7" ]]; then
-      pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0-cp27-none-linux_x86_64.whl;
-    elif [[ "$PYTHON_VERSION" == "3.5" ]]; then
-      pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0-cp35-cp35m-linux_x86_64.whl;
-    fi
 
-#    pip install keras
 #    python run_mountain_car_continuous.py  --epochs 5
 
 else