Skip to content

Commit

Permalink
Update documentation for PytorchExperiment and Config
Browse files Browse the repository at this point in the history
  • Loading branch information
Jens Petersen committed Nov 23, 2018
1 parent a173390 commit 8642117
Show file tree
Hide file tree
Showing 9 changed files with 232 additions and 101 deletions.
3 changes: 1 addition & 2 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The MIT License (MIT)

Copyright (c) 2017 Medical Image Computing Group, DKFZ
Copyright (c) 2018 Medical Image Computing Group, DKFZ

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand All @@ -19,4 +19,3 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

5 changes: 0 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
init:
pip install numpy ; \
pip install -f http://www.simpleitk.org/SimpleITK/resources/software.html --trusted-host www.simpleitk.org -r requirements.txt


tests:
python -m unittest discover

Expand Down
9 changes: 6 additions & 3 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
</p>

Finally get some structure into your machine learning experiments.
**trixi** is a tool that helps you configure, log and visualize your experiments in a reproducible fashion.
**trixi** (Training & Retrospective Insights eXperiment Infrastructure) is a tool that helps you configure, log and visualize your experiments in a reproducible fashion.

* [Features](#features)
* [Installation](#installation)
Expand All @@ -28,7 +28,7 @@ Finally get some structure into your machine learning experiments.
* Experiment Browser <br>
*Compare, combine and visually inspect the results of your experiments*.

An detailed implementation overview is given [here](https://trixi.readthedocs.io/en/latest/class_diagram.html).
An implementation diagram is given [here](https://trixi.readthedocs.io/en/latest/class_diagram.html).

### Logging API

Expand Down Expand Up @@ -64,7 +64,7 @@ Here are some examples:
### Experiment Infrastructure

The [Experiment Infrastructure](https://trixi.readthedocs.io/en/latest/_api/trixi.experiment.html) provides a unified way to configure, run, store and evaluate your results.
It gives you an Experiment interface, for which you can implement the training, validation and testing.
It gives you an experiment interface, for which you can implement the training, validation and testing.
Furthermore it automatically provides you with easy access to the Logging API and stores your config as well as the
results for easy evaluation and reproduction. There is an abstract [Experiment](https://trixi.readthedocs.io/en/latest/_api/trixi.experiment.html#trixi.experiment.experiment.Experiment) class and a [PytorchExperiment](https://trixi.readthedocs.io/en/latest/_api/trixi.experiment.html#trixi.experiment.pytorchexperiment.PytorchExperiment) with many convenience features.

Expand All @@ -73,6 +73,9 @@ results for easy evaluation and reproduction. There is an abstract [Experiment](
For more info, visit the [Documentation](https://trixi.readthedocs.io/en/latest/_api/trixi.experiment.html).

### Experiment Browser

**(We're currently remaking this from scratch, expect major improvements :))**

The Experiment Browser offers a complete overview of experiments along with all config parameters and results.
It also allows to combine and/or compare different experiments, giving you an interactive comparison highlighting differences in the configs and a detailed view of all images,
plots, results and logs of each experiment, with live plots and more.
Expand Down
3 changes: 0 additions & 3 deletions doc/changes.rst

This file was deleted.

2 changes: 1 addition & 1 deletion doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ You can jump right into the package by looking into our :ref:`quick-start-label`
quickstart
license
authors
changes
Github <https://github.com/MIC-DKFZ/trixi>

.. toctree::
:maxdepth: 1
Expand Down
2 changes: 1 addition & 1 deletion requirements_full.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
-r requirements.txt
python-telegram-bot==10.1.0
umap>=0.1.1
umap-learn>=0.3.6
scikit-learn>=0.19.1
44 changes: 24 additions & 20 deletions trixi/experiment/experiment.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,27 +16,31 @@ class Experiment(object):
end()
The reason there is both :meth:`.setup` and :meth:`.prepare` is that internally there is also
a :meth:`._setup_internal` method for hidden magic in classes that inherit from this. For
example, the :class:`trixi.experiment.pytorchexperiment.PytorchExperiment` uses this to restore checkpoints. Think
if :meth:`.setup` as an :meth:`.__init__` that is only called when the Experiment is actually
asked to do anything. Then use :meth:`.prepare` to modify the fully instantiated Experiment if
you need to.
To write a new Experiment simply inherit the Experiment class and overwrite the methods.
You can then start your experiment calling :meth:`.run`
You can then start your Experiment calling :meth:`.run`
In Addition the Experiment also has a test function. If you call the :meth:`.run_test` method is will call the
:meth:`.test` and :meth:`.end_test` method internally (and if you give the parameter setup = True
in run_test is will again call :meth:`.setup` and :meth:`.prepare` ).
In Addition the Experiment also has a test function. If you call the :meth:`.run_test` method it
will call the :meth:`.test` and :meth:`.end_test` method internally (and if you give the
parameter setup = True in run_test is will again call :meth:`.setup` and :meth:`.prepare` ).
Each experiment also has its current state in :attr:`_exp_state`, its start time in :attr:`_time_start`,
its end time in :attr:`_time_end` and the current epoch index in :attr:`_epoch_idx`
Each Experiment also has its current state in :attr:`_exp_state`, its start time in
:attr:`_time_start`, its end time in :attr:`_time_end` and the current epoch index in
:attr:`_epoch_idx`
"""
Args:
n_epochs (int): The number of epochs in the Experiment (how often the train and validate
method will be called)
"""

def __init__(self, n_epochs=0):
"""
Initializes a new Experiment with a given number of epochs
Args:
n_epochs (int): The number of epochs in the experiment (how often the train and validate method
will be called)
"""

self.n_epochs = n_epochs
self._exp_state = "Preparing"
Expand All @@ -46,7 +50,7 @@ def __init__(self, n_epochs=0):

def run(self):
"""
This method runs the experiment. It runs through the basic lifecycle of an experiment::
This method runs the Experiment. It runs through the basic lifecycle of an Experiment::
setup()
prepare()
Expand Down Expand Up @@ -99,7 +103,7 @@ def run(self):

def run_test(self, setup=True):
"""
This method runs the experiment.
This method runs the Experiment.
The test consist of an optional setup and then calls the :meth:`.test` and :meth:`.end_test`.
Expand Down Expand Up @@ -137,12 +141,12 @@ def run_test(self, setup=True):
raise e

def setup(self):
"""Is called at the beginning of each experiment run to setup the basic components needed for a run"""
"""Is called at the beginning of each Experiment run to setup the basic components needed for a run"""
pass

def train(self, epoch):
"""
The training part of the experiment, it is called once for each epoch
The training part of the Experiment, it is called once for each epoch
Args:
epoch (int): The current epoch the train method is called in
Expand All @@ -152,7 +156,7 @@ def train(self, epoch):

def validate(self, epoch):
"""
The evaluation/validation part of the experiment, it is called once for each epoch (after the training
The evaluation/validation part of the Experiment, it is called once for each epoch (after the training
part)
Args:
Expand All @@ -162,7 +166,7 @@ def validate(self, epoch):
pass

def test(self):
"""The testing part of the experiment"""
"""The testing part of the Experiment"""
pass

def process_err(self, e):
Expand Down
Loading

0 comments on commit 8642117

Please sign in to comment.