Skip to content

Home

Jump to bottom
Brian Wandell edited this page Sep 9, 2024 · 52 revisions

The Image Systems Engineering Toolbox for Cameras (ISETCam) is a Matlab toolbox used to design and evaluate imaging systems. ISETCam is implemented as an integrated suite of Matlab scripts, functions, and data structures that focus on modeling scene spectral radiance, optics, and image sensors. The software is designed to:

  • Represent quantitative radiometric properties of scenes and illuminants
  • Model image formation through optical elements such as microlens arrays and camera lenses
  • Characterize the sensitivity and noise of sensors, including spatial and spectral sampling of color filter arrays (CFAs)
  • Implement image processing algorithms
  • Calculate a calibrated (radiometric) representation of the rendered image
  • Throughout the simulations real units matter

We originally sold ISETCam through ImageVal Consulting, and also shared it freely with many academic colleagues -- including labs around the world and students who took our Stanford course on Image Systems Engineering. Many of our colleagues are using the software in innovative ways, integrating ISETCam with quantitative computer graphics (PBRT, ISET3d), AR/VR design, machine learning applications for rendering (isethdrsensor), modeling human vision ISETBio, and automotive sensor design (ISETAuto). There are many repositories describing research papers in the ISET and ISETBio organizations on GitHub.

At the time we open sourced the code, ISET had over 500 registered users who are working in more than 80 companies, 9 research institutes, and 65 universities in 24 different countries. As the usage increased, we decided that keeping ISET commercial would slow progress in developing innovative imaging systems for new applications in consumer, medical and industrial imaging. This perception was reinforced by our experience with the ISETBIO project, an effort that built on ISETCam to model the biology of the human visual system.

By creating this open ISETCam repository, which includes the code that pertains to sensor and display models, we hope to support engineering reproducibility, provide educational tools for people entering the field, and speed progress in the exciting field of image systems engineering.

Joyce Farrell and Brian Wandell

Download notes

  • In May, 2024, we completed the refactoring of ISETCam/ISETBio. As of this time, the main branch of ISETBio depends on having the main branch of ISETCam on your Matlab path. (We also renamed the default branch to 'main' from 'master' to be compliant with GitHub's new defaults.) As part of this merge, we validated the tutorials, numerical validations, scripts, and and examples in ISETCam. Locally, we have been running with this approach for more than six months; the system appears stable and accurate.

  • In July 2023, we started to put scripts for quantitative validation of ISETCam into a separate repository (isetvalidate). There are many such validations, and we consider them very important. Updates to the validations, however, should not interfere with the core ISETCam library itself. Also, moving the validation into a separate repository allows us to develop additional validation tools that can apply to related repositories (e.g., ISETBio, ISET3D, ISETLens ...). There will be a short time where the world is a bit shaky, but by September we expect that anyone wanting the validation scripts for ISETCam (or ISETBio) will need to download ISETValidate.

  • As of September 2020 and the Matlab 2020a release, we are dividing the ISETCam downloads into distinct branches. The reason for this is largely because recent Mathworks developments required that we re-write the different windows with their 'appdesigner' framework, replacing the deprecated 'guide' framework. Mathworks has urged us to shift away from 'guide' and we are doing so. However, the 'appdesigner' windows are not supported in 3-year old Matlab versions.

  • The 'master' branch is the current stable release that is tested using v_ISET, ieRunValidateAll, and ieRunScriptsAll. The GUI in this branch was updated to appdesigner in September, 2020.

  • The 'dev' branch, and various offshoots of that, is where we do our development.

The core computations are the same in these branches and the differences are mainly confined to the CS framework and new features.

How to cite this work

If you use this toolbox in your published work, please cite this repository and at least one of the articles below. We list them in order of preference:

ISETcam development is led by Brian Wandell's Vistalab group at Stanford University and supported by contributors from other research institutions and industry.

Clone this wiki locally