Skip to content

Create local testing environment for keras

Jump to bottom
Ramón Casero edited this page Nov 5, 2018 · 10 revisions

Introduction

Keras uses Travis to automatically run test units on pull requests. You should create a keras docker to test locally, but if for some reason that's not an option, here are my notes to create a simple local testing environment for keras. (I have made a note at the end of this page for some issues I found to run the keras docker in Ubuntu).

Note: This local environment doesn't pass all the keras tests. For proper testing, you need the keras docker. This is just a hack that can be useful if you want to run a specific new test, e.g. for a pull request you are debugging.

Travis testing is configured via the file .travis.yml in the keras github repository.

Basically, we are going to read the instructions in that file and replicate the configuration in a local conda environment.

We are going to use the commit version of .travis.yml that corresponds to a pull request we wrote for Keras 2.2.0, "Enable element-wise weighing of outputs in model.fit(..., sample_weight=weights)". You will need to adapt the instructions below to the version of .travis.yml that corresponds to your own pull request.

Install conda

The first step is to install conda in your system if you don't have it yet. This will allow you to create and manage python local environments.

Keras uses Miniconda to install conda. If you are going to test for python 2.7, install Miniconda 2. For python 3.6, install Miniconda 3.

I wrote a script called install_miniconda.sh to expedite this.

Create local environment to test keras

You need a different local environment for each version of python (2.7 or 3.6), and for each backend (Tensorflow, CNTK, etc) that you want to test.

Here, I'm going to create one for python 3.6 and Tensorflow, but you can modify the steps accordingly for other combinations.

First, from a shell create a local environment keras-test, adding the pytest and pandas packages

conda create -q -n keras-test python=3.6 pytest pandas

Activate the environment

source activate keras-test

Select the backend you want to run tests for

export KERAS_BACKEND=tensorflow

Install some more python packages

pip install --only-binary=numpy,scipy numpy nose scipy matplotlib h5py theano
pip install keras_applications keras_preprocessing
conda install mkl mkl-service

Install PIL for preprocessing tests

conda install Pillow

Go to your local keras source directory, presumably with the changes you have made for the pull request, and that you want to test, e.g.

cd ~/Software/keras_patch

Try to install your patched keras.

pip install -e .[tests]

If you get an error such as

Exception:
Traceback (most recent call last):
...
pip._vendor.pkg_resources.ContextualVersionConflict: (Keras-Preprocessing 1.0.5 (/home/rcasero/.conda/envs/keras-test/lib/python3.6/site-packages), Requirement.parse('keras_preprocessing==1.0.1'), {'Keras'})

your keras fork is out of sync with the current keras dependencies, which you installed a bit earlier. You need to install the correct version. For example, in this case, you need keras_preprocessing=1.0.1.

pip install keras_preprocessing==1.0.1

After that, you can try to install your patch again

pip install -e .[tests]

Keep an eye on warnings or error messages. For instance, I get the message mkl-random 1.0.1 requires cython, which is not installed. Thus, I install cython too

pip install cython

Install backends. Note that CNTK needs a different URL depending on the python version. You can find the right URL in the .travis.yml. The one I use here is for python 3.6

pip install tensorflow==1.7
pip install https://cntk.ai/PythonWheel/CPU-Only/cntk-2.5.1-cp36-cp36m-linux_x86_64.whl

Install pydot for visualization tests

conda install pydot graphviz

Exclude the backends you are not going to test for, e.g.

if [[ "$KERAS_BACKEND" != "tensorflow" ]]; then
  echo '    keras/backend/tensorflow_backend.py' >> .coveragerc;
fi
if [[ "$KERAS_BACKEND" != "theano" ]]; then
  echo '    keras/backend/theano_backend.py' >> .coveragerc;
fi
if [[ "$KERAS_BACKEND" != "cntk" ]]; then
  echo '    keras/backend/cntk_backend.py' >> .coveragerc;
fi

Detect whether core files are changed or not

export CORE_CHANGED=False
for entry in `git diff --name-only HEAD~1`; do if [[ "$entry" == "keras/backend/"* ]] || [[ "$entry" == "keras/engine/"* ]] || [[ "$entry" == "keras/layers/"* ]]; then export CORE_CHANGED=True; fi; done
export APP_CHANGED=False
for entry in `git diff --name-only HEAD~1`; do if [[ "$entry" == "keras/applications/"* ]]; then export APP_CHANGED=True; fi; done

Install open mpi

rm -rf ~/Software/mpi
mkdir ~/Software/mpi
pushd ~/Software/mpi
wget http://cntk.ai/PythonWheel/ForKeras/depends/openmpi_1.10-3.zip
unzip ./openmpi_1.10-3.zip
sudo dpkg -i openmpi_1.10-3.deb
popd

Run tests

Make sure you have selected the backend

export KERAS_BACKEND=tensorflow

Run the keras tests (we assume that you are in the local environment with source activate keras-test)

export MKL_THREADING_LAYER="GNU"
# run keras backend init to initialize backend config
python -c "import keras.backend"
# create dataset directory to avoid concurrent directory creation at runtime
mkdir ~/.keras/datasets
# set up keras backend
sed -i -e 's/"backend":[[:space:]]*"[^"]*/"backend":\ "'$KERAS_BACKEND'/g' ~/.keras/keras.json
echo -e "Running tests with the following config:\n$(cat ~/.keras/keras.json)"
if [[ "$TEST_MODE" == "INTEGRATION_TESTS" ]]; then
  PYTHONPATH=$PWD:$PYTHONPATH py.test tests/integration_tests;
elif [[ "$TEST_MODE" == "PEP8" ]]; then
  PYTHONPATH=$PWD:$PYTHONPATH py.test --pep8 -m pep8 -n0;
elif [[ "$TEST_MODE" == "DOC" ]]; then
  PYTHONPATH=$PWD:$PYTHONPATH py.test tests/test_documentation.py;
else
  PYTHONPATH=$PWD:$PYTHONPATH py.test tests/ --ignore=tests/integration_tests --ignore=tests/test_documentation.py --ignore=tests/keras/legacy/layers_test.py --cov-config .coveragerc --cov=keras tests/;
fi > test.log

Some issues with the keras docker in Ubuntu

  • Edit keras/docker/Makefile, adding --network host to the build, because otherwise my computer cannot reach ubuntu servers
docker build -t keras --build-arg python_version=$(PYTHON_VERSION) ...
  • Edit keras/docker/Dockerfile, replacing
git clone git://github.com/

with

git clone http://github.com/

and replacing

pip install git+git://github.com/keras-team/keras.git

with

pip install git+http://github.com/keras-team/keras.git

because otherwise the git and pip commands time out without completing.

Clone this wiki locally