Home
The Image System Engineering Toolbox for Biology (ISETBio) is a Matlab toolbox for vision science calculations. It lets you build spectral radiance scenes and use them as input to model human optics, eye movements, cone absorptions and photocurrent, and retinal cell responses.
New to ISETBio? Start with Getting Started.
Vision scientists have discovered many facts and developed many precise computations that characterize critical steps in vision. Some of these computations are complex and require access to numerical tables (e.g., color-matching, optical wavefront aberrations). ISETBio provides many of these computations, and the numerical data they need, in a convenient and extensible format so that investigators can both use and extend vision science. We hope this code helps develop new biological insights and theoretical ideas.
We hope that ISETBio will be a computational framework that helps specialists in different branches of vision science and engineering explain and share their work. Specialists working on one part of the system - say optics - can contribute code that people expert in a different part of the system - say the retina - can use. By placing the knowledge of the vision science fields in a unified software framework, we believe vision scientists will be able to better see the big picture.
David Brainard has recorded a series of short videos introducing ISETBio's core concepts and the Matlab tools that implement them:
- Representing a visual stimulus - the scene (spectral radiance) representation for planar stimuli
- Computing the cone excitations
- Eye movements and photocurrents - fixational eye movements and cone photocurrent
- Contrast sensitivity - predicting contrast sensitivity of a spatially localized harmonic (Gabor patch)
- Intro to tools for contrast threshold calculations
See ISETBio Videos for the full collection, including the 2024 OSA tutorial series, color-science lectures, and ISET3d integration talks.
This project is based on the idea that computational models that are approximately right and exist are preferable to models that are perfect and do not exist. We hope that our colleagues and students accept that a specific computation implementing our best current understanding is better than no computation. To support this approach, it is important to specify the domain over which the calculations have been checked and to give some sense of the accuracy of the model over that domain. We try to indicate the stimulus compliance range and precision of computations, as best we can, throughout the code and in the documentation. If we identify a problem, we try to follow this advice.
The ISETBio code base is evolving and being checked by different individuals. We do our best to write validation tests for each component as well as system and regression tests. We have confidence in many of the critical routines, which have been independently implemented and produce the same results; other routines are still being tested. We have tried to make these differences clear in code comments and on this wiki. We invite you to perform your own tests and to share the results with us.
This project includes specifications of scene radiance, optics and retinal irradiance, cone mosaic absorptions and photocurrent, retinal encoding, eye movements, and computational classifiers. The compliance range and precision of the scene, optics, and absorptions computations are the most advanced; our confidence declines as we go deeper into the nervous system.
Our ambitions go beyond the current coverage. With more experience and feedback, we will decide how much further we can extend the scope of these computations. That's right, LGN and V1, we're talking about you.
If you use ISETBio, please cite it using the following publication:
Nicolas P. Cottaris, Haomiao Jiang, Xiaomao Ding, Brian A. Wandell, David H. Brainard; A computational-observer model of spatial contrast sensitivity: Effects of wave-front-based optics, cone-mosaic structure, and inference engine. Journal of Vision 2019;19(4):8. doi:10.1167/19.4.8.
Related publications describing ISETBio's goals, architecture, and models:
David H. Brainard, Haomiao Jiang, Nicolas P. Cottaris, Fred Rieke, E.J. Chichilnisky, Joyce E. Farrell, and Brian A. Wandell (2015). ISETBIO: Computational tools for modeling early human vision. Imaging Systems and Applications, Optical Society of America.
Wandell, Brian A., David H. Brainard, and Nicolas P. Cottaris (2022). Visual encoding: principles and software. Progress in Brain Research 273.1: 199-229. Elsevier Press, Ed. Manuel Spitschan.
Cottaris, N. P., Wandell, B. A., & Brainard, D. H. (2026). An image-computable spatio-chromatic receptive field model of the midget retinal ganglion cell mosaic across the retina. Journal of Computational Neuroscience, 1-36.
Related repositories, satellite projects, and external toolboxes are listed on the Resources page.