Wrapper integrating Mri and caffe-cnn
Python Rebol
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Incooperate Mri and Caffe-cnn to perform hyper-parameter search.


Quick run

We provide some toy data on which you can quickly perform hyper-parameter searching, training and testing.You need to run it under the repository directory.

docker pull haoyangz/mri-wrapper
docker run -v $(pwd)/example:/data -v $(pwd)/example/model:/model -i --rm \
--device /dev/nvidiactl \
--device /dev/nvidia-uvm \
--device /dev/nvidia0 \
haoyangz/mri-wrapper python main.py /model/runparam.list 0

This is what happens if it runs:

  • Search for the best hyper-parameters for a toy Caffe model on some toy training and validation data.

  • Pick the set of parameters with best performance, and try a few times to train on the training data. The trial and iteration where the model reaches the best performance on the validation set will be picked for testing and prediction.

  • Make prediction with the best model on the toy testing data. The output will be under $REPO_HOME$/example/16_G/mri-best/best_trial/

  • Make prediction with a pre-trained model (specified by example/newdata/deploy.txt and example/newdata/bestiter.caffemodel) to predict on new data specified in example/newdata/test.txt. The output will be saved under example/newdata_output.

Data preparation

Follow the instruction in Caffe-cnn to prepare data under $DATADIR$/data.

Model preparation

All of the files in this part should be saved under $MODELDIR$.

Caffe model files

Follow the instruction in Caffe-cnn to prepare caffe model files:

  • trainval.prototxt : training architecture
  • solver.prototxt: solver parameter
  • deploy.prototxt: testing architecture

The only differences:

  • In trainval.prototxt, the "source" in the input layer should be /data/data/train.txt and /data/data/valid.txt
Modify model files for hyper-parameter search

For each hyperparameter to tune, replace the value with %{param}% instead. For example:

In solver.prototxt:

delta: %{delta}%

In trainval.prototxt / deploy.prototxt:

layer {
  name: "drop1"
  type: "Dropout"
  bottom: "fc1"
  top: "fc1"
    dropout_ratio: %{dropout_ratio}%

Note that every parameter changed in trainval.prototxt should also be modified accordingly in deploy.prototxt

Specify value choices of hyper-parmeters

In "hyperparams.txt" file.

Specify model name

In "modelname" file.

Prepare runparam.list

General argument:

  • ORDER: The following orders are accepted. Multiple orders should be concatenated by comma, and the relative order should be maintained.
    • trainMRI: hyper-parameter tuning
    • update: pick the best hyper-parameter
    • trainCaffe: train on the training set with the best hyper-parameter
    • testCaffe: predict on the test set with the trained model
    • testEvalCaffe: evaluate the performance on test set
    • predictCaffe: predict with a pre-trained model to predict on new data

If you want to perform hyper-parameter tuning, training and testing:

  • MRI_MAXITER: The number of hyper-parameter setting to try.
  • search_max_iter: Maximum number of iteration in hyper-parameter search.
  • search_test_interval: The iteration interval to test on validation set in hyper-parameter search.
  • search_snapshot: The iteration iterval to save model in hyper-parameter search.
  • search_display: The iteration interval to display the training loss in hyper-parameter search.
  • train_trial: After the best hyper-params are chosen, we will train this number of times and pick the best trial.
  • debugmode: The verbosity of log ('NONE'<'INFO'<'DEBUG'). We recommend 'INFO' in most case.
  • optimwrt: Choose the best param and training trial with respect to either accuracy or loss
  • outputlayer: The name of the output layer that will be used as prediction in test phase.

If you want to predict on new data with a trained model:

  • outputlayer: Same as above.
  • deploy2predictW: path to the deploy.prototxt file of the trained model, which specifies the network architecture.
  • caffemodel2predictW: path to the snapshot of the trained model, which ends with '.caffemodel' and during training will be saved every snapshot number of iteration as specified in solver.prototxt.
  • data2predict: path to the manifest file specifying all HDF5 format data to predict on.
  • predict_outdir: the output directory, under which 'bestiter.pred' and 'bestiter.pred.params.pkl' will be saved. See description for outputlayer.

As Mri-wrapper lives in the docker container, it can only access the directory mounted to the container. For instance, in the toy example above, $(pwd)/example is mounted as /data and $(pwd)/example/model is mounted as /model. Therefore all the above path parameters should be relative to one of the mounted directory. But you can mount as many directory as you want with '-v' option as in the toy example.

Ready to go!

docker pull haoyangz/mri-wrapper
docker run -v DATADIR:/data -v MODELDIR:/model -i --rm \
	--device /dev/nvidiactl --device /dev/nvidia-uvm MOREDEVICE \
	haoyangz/mri-wrapper python main.py /model/runparam.list GPUNUM
  • DATADIR: The topfolder of the output. Your data should be under $DATADIR$/data.

  • MODELDIR: The folder containing the model files, hyperparam file and modelname file

  • MOREDEVICE: For each of the GPU device available on your machine,append one "--device /dev/nvidiaNUM" where NUM is the number of that device. For hsf1/hsf2 in Gifford Lab, since there are three GPU, it should be :

--device /dev/nvidia0 --device /dev/nvidia1 --device /dev/nvidia2

+ `GPUNUM`: The GPU device number to run

For hyper-parameter tuning, training and testing, all the output will be generated under *$DATADIR$/$thenameofthemodel$*, where `$thenameofthemodel$` is the name specified in the file [*$MODELDIR$/modelname*](https://github.com/gifford-lab/mri-wrapper/blob/master/example/model/modelname) .

After training, the information of the best model will be saved as *bestiter.caffemodel* and *deploy.prototxt* under  *$DATA_DIR$/$thenameofthemodel$/mri-best/best_trial/*.

For testing and prediction, the output of the first layer specified in `outputlayer` (see above section on params.list) will be saved to a text file named *bestiter.pred* and the output of all layers will be saved in a Pickle object named *bestiter.pred.params.pkl*. For testing, they will be both under *$DATA_DIR$/$thenameofthemodel$/mri-best/best_trial/*. For prediction, they will be under `predict_outdir` as specified in [runparams.list](https://github.com/gifford-lab/mri-wrapper/blob/master/example/model/runparam.list).