Note: The latest PyTorch version (
0.4) merged Tensors and Variables.
The current convis release was adapted to this change, but it is still possible
that something broke in the change, so please report any Exceptions thrown by
PyTorch in the issue tracker.
This python package provides an implementation of the Virtual Retina developed by Adrien Wohrer. It uses
PyTorch to simulate spike trains of retinal ganglion cells by directing the input through a number of computation layers. Each layer might do linear or nonlinear computations, eg. convolve the inpute with a spatio-temporal kernel or apply gain control.
We are supporting Python 3.5 and 2.7 right now, but are aiming to support Python 3.6 as well.
Convis is under development and some features might not work in the current master branch or the PyPi releases. If you discover unexpected behaviour, please leave an Issue on github.
Also there are two mailing lists for people interested in Convis:
- To recieve announcements of changes, please subsribe to: email@example.com
- If you want to participate in the development, please subscribe to: firstname.lastname@example.org
Find the documentation at: https://jahuth.github.io/convis/
import numpy as np import matplotlib.pylab as plt import convis c = convis.retina.RetinaConfiguration() ret = convis.retina.Retina(c) inp = np.zeros((2000,50,50)) inp[:,20:30,20:30] = 255.0*(rand(*inp[:,20:30,20:30].shape)<0.2) out = ret.run(inp) plt.plot(np.mean(out,(0,1,3,4))) plt.plot(np.mean(out,(0,1,3,4)))
An earlier version using
theano has been put on hold, but is still available here. If you are interested in continued development of the
theano version, please let me know!
An even older version was published as the retina package
convis for your scientific publications, please cite:
- Huth J, Masquelier T and Arleo A (2018) Convis: A Toolbox to Fit and Simulate Filter-Based Models of Early Visual Processing. Front. Neuroinform. 12:9. doi: 10.3389/fninf.2018.00009
PyTorch itself is not complicated.
Requirements for the base installation are: Python 2.7 or Python 3.5, Numpy, SciPy.
The current PyTorch version built for CUDA 8 can be installed via conda or pip:
conda install pytorch -c pytorch # or pip install torch torchvision
For other CUDA versions, installing via pip on Windows etc., see the PyTorch website.
Once the requirements are installed,
convis can be installed via pip for the latest release:
pip install convis
or from the most up-to-date version from github:
pip install git+https://github.com/jahuth/convis.git
Alternatively, you can clone the repository and install it locally:
git clone https://github.com/jahuth/convis.git cd convis # change something in the source code pip install -e convis
I recommend installing opencv, and jupyter notebook, if you do not already have it installed:
pip install convis notebook # eg. for ubuntu: sudo apt-get install python-opencv
convis provides a retina model which is identical to
VirtualRetina and tools
to create either simpler or more complex models.
The paper about
convis is available in frontiers in Neuroinformatics:
"Convis: A Toolbox To Fit and Simulate Filter-based Models of Early Visual Processing".
The Retina model
A description of all parameters for the retina model can be obtained directly from an instantiated model. This description contains doc strings for each parameter.
import convis retina = convis.retina.Retina() print(retina)
To use the model, supply a numpy array as an argument to the
Retina (for short input) or to the
run function with a
dt keyword to split the input in smaller chunks (and automatically reassemble the output):
import numpy as np import matplotlib.pylab as plt inp = np.ones((100,20,20)) output = retina(inp) inp = np.ones((2000,20,20)) output = retina.run(inp,dt=100)
It will return an
Output object containing all outputs of the model (the default for retina is two outputs: spikes of On and Off cells).
convis.plot_5d_time(output) title('On Cells (1 line = 1 pixel)') figure() convis.plot_5d_matshow(output[:,:,::50,:,:]) title('Every 50th frame of activity') figure() # dimension 2 is time, so we mean over all others # to get the average activity convis.plot_5d_time(output.mean((0,1,3,4))) convis.plot_5d_time(output.mean((0,1,3,4))) title('Mean Activitiy of On and Off Cells')
The output object holds the ouput of the computation (in most cases
torch.autograd.Variables) but can be converted to a
numpy array with
output.array(...). Outputs can be addressed with numbers or names.
>>> type(output) <class 'torch.autograd.variable.Variable'> >>> type(output.data.cpu().numpy()) <class 'numpy.ndarray'> >>> type(output.array(0)) <class 'numpy.ndarray'> >>> output is output.ganglion_spikes_ON True >>> output is output['ganglion_spikes_ON'] True >>> np.sum(abs(output.array('ganglion_spikes_ON') - output.array(0))) 0
If instead of spikes, only the firing rate should be returned, the retina can be initialized without a spiking mechanism:
retina_without_spikes = convis.retina.Retina(spikes = False)
Found a bug or want to contribute?
Bug reports and feature requests are always welcome! The best place to put them is the github issue tracker. If you have questions about usage of functions and classes and can not find an answer in the documentation and docstrings, this is considered a bug and I appreciate it if you open an issue for that!
If you want, you can flag your issue already with one of the labels:
If you have fixed a bug or added a feature and you would like to see the change included in the main repository, the preferred method is for you to commit the change to a fork on your own github account and submit a pull request.
General discussion is encouraged on the two mailing lists: