Skip to content
Theano-based Alexnet
Branch: master
Clone or download
Latest commit 47523e6 Mar 28, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
lib fix the T.neq not giving int8 type issue #29 Mar 28, 2017
preprocessing release version Dec 7, 2014
pretrained/alexnet
.gitignore release version Dec 7, 2014
LICENSE.txt
README.md Update README.md Mar 28, 2017
alex_net.py added top5 error and script for validating performance Apr 6, 2015
config.yaml
proc_load.py
spec_1gpu.yaml
spec_2gpu.yaml
train.py
train_2gpu.py fix a typo Aug 28, 2015
train_funcs.py remove some comments and fix a drv init bug in train.py when Para_loa… Jun 24, 2015
validate_performance.py fix some bugs in validate_performance.py Jun 19, 2015

README.md

AlexNet Implementation with Theano

Demonstration of training an AlexNet in Python with Theano. Please see this technical report for a high level description. theano_multi_gpu provides a toy example on how to use 2 GPUs to train a MLP on the mnist data.

If you use this in your research, we kindly ask that you cite the above report:

@article{ding2014theano,
  title={Theano-based Large-Scale Visual Recognition with Multiple GPUs},
  author={Ding, Weiguang and Wang, Ruoyan and Mao, Fei and Taylor, Graham},
  journal={arXiv preprint arXiv:1412.2302},
  year={2014}
}

Dependencies

How to run

Prepare raw ImageNet data

Download ImageNet dataset and unzip image files.

Preprocess the data

This involves shuffling training images, generating data batches, computing the mean image and generating label files.

Steps

  • Set paths in the preprocessing/paths.yaml. Each path is described in this file.
  • Run preprocessing/generate_data.sh, which will call 3 python scripts and do all the mentioned steps. It runs for about 1~2 days. For a quick trial of the code, run preprocessing/generate_toy_data.sh, which takes ~10 minutes and proceed.

preprocessing/lists.txt is a static file that lists what files should be created by running generate_data.sh.

Train AlexNet

Set configurations

config.yaml contains common configurations for both the 1-GPU and 2-GPU version.

spec_1gpu.yaml and spec_2gpu.yaml contains different configurations for the 1-GPU and 2-GPU version respectively.

If you changed preprocessing/paths.yaml, make sure you change corresponding paths in config.yaml, spec_1gpu.yaml and spec_2gpu.yaml accordingly.

Start training

1-GPU version, run:

THEANO_FLAGS=mode=FAST_RUN,floatX=float32 python train.py

2-GPU version, run:

THEANO_FLAGS=mode=FAST_RUN,floatX=float32 python train_2gpu.py

Validation error and loss values are stored as weights_dir/val_record.npy

Here we do not set device to gpu in THEANO_FLAGS. Instead, users should control which GPU(s) to use in spec_1gpu.yaml and spec_2gpu.yaml.

Pretrained AlexNet

Pretrained AlexNet weights and configurations can be found at pretrained/alexnet

Acknowledgement

Frédéric Bastien, for providing the example of Using Multiple GPUs

Lev Givon, for helping on inter process communication between 2 gpus with PyCUDA, Lev's original script https://gist.github.com/lebedov/6408165

Guangyu Sun, for help on debugging the code

You can’t perform that action at this time.