-
Notifications
You must be signed in to change notification settings - Fork 45
Architecture
ISETBIO is organized conceptually in terms of several critical components of biological vision. For example, representations encapsulate the quantitative properties of the input scene, the physiological optics, the photoreceptor absorptions, and so forth. These elements are implemented as either a Matlab structure (with corresponding set/get/create/plot routines; this is how we started) or as a Matlab class (this is how we're doing new code). Computations are implemented as transformations between these objects.
ISETBio builds on ISETCam, which supplies the general scene (spectral radiance), display, and optics/oi (optical image) representations - Lambertian and depth/light-field scenes, display calibration data, and the diffraction- limited and shift-invariant optics models used to compute a generic spectral irradiance image. That machinery, and its object/structure APIs, are documented in the ISETCam wiki (see Scene radiance and Optics and optical images) rather than duplicated here.
ISETBio adds the parts of the optics pipeline that are specific to biological vision. The wavefront aberrations model computes the human eye's optical blur from measured or population wavefront data (see WavefrontOptics), which the diffraction-limited and shift-invariant models in ISETCam do not attempt. The transmittance of the human lens and macular pigment are also implemented here, since they matter specifically for computing cone fundamentals and cone absorptions. sceneEye's 3D ray-traced human-eye optics (see Human optics 3D) is a further extension along the same lines.
The cone mosaic Matlab object combines the spectral irradiance image arriving at the retina with the specification of the cone mosaic and computes a spatial array of cone absorptions. The cone absorption calculations account for the macular pigment and photopigment optical density. The properties of the individual cones as well as the distribution of cones in the mosaic can be controlled. For example, it is possible to create models of dichromacy and anomalous trichromacy. It is also possible to set parameters that match the fovea and the periphery. Note that because the lens transmittance is implemented in the optics, the isetbio representation of the cone fundamentals requires combining information held in the optics and information in the come mosaic object itself.
The cone mosaic object implements spatial sampling on a rectangular grid. This is computationally efficient, but not biologically realistic. A subclass of the cone mosaic object, the cone mosaic hex object, implements space varying hexagonal sampling, as well as jittered versions of hexagonal sampling.
We note that in addition to the optics and cone mosaic other critical entities, such as the photopigment, macular pigment, and lens are implemented as Matlab objects.
In addition to the pigment absorptions and isomerizations, the cone outer segment current is calculated. There are several models for calculating the current, ranging from simple and fast linear approximations to a more sophisticated biophysical difference-equation-based model.
ISETBio simulates fixational eye movements with the fixationalEM object, which generates biologically plausible eye-movement paths (tremor, drift, and microsaccades) rather than a fixed static gaze. A path is attached to a cone mosaic and shifts the retinal image between absorption samples, so the same optical image can generate different cone-absorption sequences depending on the simulated fixation behavior. See Eye movements.
ISETBio has an initial set of code that models retinal ganglion cell (RGC) responses, centered on the mRGCMosaic object: an image-computable, spatio-chromatic receptive field model of the midget RGC mosaic, with pre-baked ON-midget circuits available at a range of retinal locations. See Retinal ganglion cell (RGC) mosaics and the Cottaris, Wandell & Brainard (2026) paper cited on the Home page. This part of the toolbox is newer and less complete than the optics and cone mosaic code above; earlier RGC/bipolar-cell modeling approaches in the repository are historical and are not the current supported path.
Finally, there are routines that are used mainly to predict nearly optimal discrimination between different stimulus classes. For example, we use support vector machines and other classifiers to assess how much information is present in the simulated biological signals at different points in the visual pathway.
- Where we might go in the near term
- An initial stab at data structure documentation