Skip to content

Commit

Permalink
updated README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
williamFalcon committed Feb 14, 2018
1 parent 3d7aae5 commit 3f307f4
Show file tree
Hide file tree
Showing 2 changed files with 81 additions and 48 deletions.
121 changes: 74 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@
</h3>

<p align="center">
Log, organize and optimize Deep Learning experiments
Log, organize and optimize Deep Learning experiments
</p>

<p align="center">
<a href="https://badge.fury.io/py/test_tube"><img src="https://badge.fury.io/py/test_tube.svg"></a>
<a href="https://williamfalcon.github.io/test-tube/"><img src="https://readthedocs.org/projects/test-tube/badge/?version=latest"></a>
<a href="https://github.com/williamFalcon/test-tube/blob/master/LICENSE.txt"><img src="https://img.shields.io/github/license/mashape/apistatus.svg?maxAge=2592000"></a>
<a href="https://github.com/williamFalcon/test-tube/blob/master/LICENSE.txt"><img src="https://img.shields.io/github/license/mashape/apistatus.svg?maxAge=2592000"></a>
</p>


Expand All @@ -39,24 +39,24 @@ Use Test Tube if you need to:
- [Optimize your hyperparameters](hyperparameter_optimization/HyperOptArgumentParser/) using grid_search or random_search.
- Visualize experiments without uploading anywhere, logs store as csv files.
- Automatically track ALL parameters for a particular training run.
- Automatically snapshot your code for an experiment using git tags.
- Save progress images inline with training metrics.
- Automatically snapshot your code for an experiment using git tags.
- Save progress images inline with training metrics.

Compatible with:
Compatible with:
- Python 2, 3
- Tensorflow
- Keras
- Pytorch
- Caffe, Caffe2
- Chainer
- MXNet
- Theano
- Scikit-learn
- Any python based ML or DL library
- Runs seamlessly on CPU and GPU.

### Why Test Tube
If you're a researcher, test-tube is highly encouraged as a way to post your paper's training logs to help add transparency and show others what you've tried that didn't work.
- Tensorflow
- Keras
- Pytorch
- Caffe, Caffe2
- Chainer
- MXNet
- Theano
- Scikit-learn
- Any python based ML or DL library
- Runs seamlessly on CPU and GPU.

### Why Test Tube
If you're a researcher, test-tube is highly encouraged as a way to post your paper's training logs to help add transparency and show others what you've tried that didn't work.

## Examples

Expand All @@ -71,18 +71,45 @@ exp.add_meta_tags({'learning_rate': 0.002, 'nb_layers': 2})
for step in range(1, 10):
tng_err = 1.0 / step
exp.add_metric_row({'tng_err': tng_err})
```
```

### Visualize experiments
```python
import pandas as pd
import matplotlib
### Visualize experiments
```python
import pandas as pd
import matplotlib

# each experiment is saved to a metrics.csv file which can be imported anywhere
# images save to exp/version/images
# images save to exp/version/images
df = pd.read_csv('../some/dir/test_tube_data/dense_model/version_0/metrics.csv')
df.tng_err.plot()
```
df.tng_err.plot()
```

### Optimize hyperparameters across gpus
```python
from test_tube import HyperOptArgumentParser

# subclass of argparse
parser = HyperOptArgumentParser(strategy='random_search')
parser.add_argument('--learning_rate', default=0.002, type=float, help='the learning rate')

# let's enable optimizing over the number of layers in the network
parser.add_opt_argument_list('--nb_layers', default=2, type=int, tunnable=True, options=[2, 4, 8])

# and tune the number of units in each layer
parser.add_opt_argument_range('--neurons', default=50, type=int, tunnable=True, start=100, end=800, nb_samples=10)

# compile (because it's argparse underneath)
hparams = parser.parse_args()

# optimize across 4 gpus
# use 2 gpus together and the other two separately
hparams.optimize_parallel_gpu_cuda(MyModel.fit, gpu_ids=['1', '2,3', '0'], nb_trials=192, nb_workers=4)
```

Or... across CPUs
```python
hparams.optimize_parallel_cpu(MyModel.fit, nb_trials=192, nb_workers=12)
```

### Optimize hyperparameters
```python
Expand All @@ -104,43 +131,43 @@ hparams = parser.parse_args()
# run 20 trials of random search over the hyperparams
for hparam_trial in hparams.trials(20):
train_network(hparam_trial)
```
```

### Convert your argparse params into searchable params by changing 1 line
```python
import argparse
### Convert your argparse params into searchable params by changing 1 line
```python
import argparse
from test_tube import HyperOptArgumentParser

# these lines are equivalent
# these lines are equivalent
parser = argparse.ArgumentParser(description='Process some integers.')
parser = HyperOptArgumentParser(description='Process some integers.', strategy='grid_search')
parser = HyperOptArgumentParser(description='Process some integers.', strategy='grid_search')

# do normal argparse stuff
# do normal argparse stuff
...
```
```

### Log images inline with metrics
### Log images inline with metrics
```python
# name must have either jpg, png or jpeg in it
img = np.imread('a.jpg')
exp.add_metric_row('test_jpg': img, 'val_err': 0.2)

# saves image to ../exp/version/media/test_0.jpg
# csv has file path to that image in that cell
# saves image to ../exp/version/media/test_0.jpg
# csv has file path to that image in that cell
```


## Demos
- [Experiments and hyperparameter optimization for tensorflow across 4 GPUs simultaneously](https://github.com/williamFalcon/test-tube/blob/master/examples/tensorflow_example.py)
- [Experiments and hyperparameter optimization for tensorflow across 4 GPUs simultaneously](https://github.com/williamFalcon/test-tube/blob/master/examples/tensorflow_example.py)

## How to contribute
Feel free to fix bugs and make improvements!
1. Check out the [current bugs here](https://github.com/williamFalcon/test-tube/issues).
2. To work on a bug, head over to our [project page](https://github.com/williamFalcon/test-tube/projects/1) and assign yourself the bug.
3. We'll add contributor names periodically as people improve the library!
## How to contribute
Feel free to fix bugs and make improvements!
1. Check out the [current bugs here](https://github.com/williamFalcon/test-tube/issues).
2. To work on a bug, head over to our [project page](https://github.com/williamFalcon/test-tube/projects/1) and assign yourself the bug.
3. We'll add contributor names periodically as people improve the library!

## Bibtex
To cite the framework use:
## Bibtex
To cite the framework use:
```
@misc{Falcon2017,
author = {Falcon, W.A.},
Expand All @@ -152,5 +179,5 @@ To cite the framework use:
}
```

## Contributors:
1. [William Falcon](https://williamfalcon.com)
## Contributors:
1. [William Falcon](https://williamfalcon.com)
8 changes: 7 additions & 1 deletion update.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,10 @@ git commit -am "release v$version"
git tag $version -m "test_tube v$version"
git push --tags origin master

python setup.py sdist upload -r pypi
python setup.py sdist upload -r pypi



# to update docs
# cd to root dir
# mkdocs gh-deploy

0 comments on commit 3f307f4

Please sign in to comment.