Spectral Scene Data

Spectral Scene Data

ISETCam calculates how light travels through lenses and excites sensors by modeling scenes as calibrated spectral radiance distributions. While synthetic targets (like the Macbeth ColorChecker or resolution charts created by sceneCreate) are useful for testing, evaluating real camera systems requires spectral radiance measurements of natural and man-made objects, human faces, and complex lighting.

ISETCam provides built-in tools (ieWebGet and sceneSDR) to discover, browse, download, and load calibrated spectral, multispectral, and high dynamic range (HDR) scene collections preserved in the Stanford Digital Repository (SDR).

---

Accessing SDR Data with ieWebGet and sceneSDR

The Stanford Digital Repository provides permanent, citable digital object identifiers (PURLs) for ISET research datasets. ISETCam provides MATLAB functions to interact with these repositories directly.

Discover and browse with ieWebGet

ieWebGet manages discovery and downloading of remote data assets:

% List all known SDR deposits and collections
ieWebGet('list');

% Open the Stanford Digital Repository landing page in your browser
ieWebGet('browse', 'fruits-charts');
ieWebGet('browse', 'faces-1m');
ieWebGet('browse', 'hdr-images');
ieWebGet('browse', 'iset-multispectral-collection');

To download a specific deposit or asset file:

% Download and unzip a specific dataset into the default download directory
localDir = ieWebGet('deposit name', 'fruits-charts', 'unzip', true);

% Or download without interactive confirmation
localFile = ieWebGet('deposit name', 'misc-multispectral1', 'confirm', false);

Loading rendered scenes with sceneSDR

For 3D scenes rendered using PBRT and converted to ISETCam spectral format, use sceneSDR. It automatically checks your local cache in isetcam/data/scenes/web/, downloads the file from SDR if needed, and loads the scene into MATLAB:

% Load a rendered Cornell Box scene directly
scene = sceneSDR('isetcam bitterli', 'cornell-box.mat');

% Open the interactive scene window
sceneWindow(scene);

Supported rendered deposit names for sceneSDR:

• 'isetcam bitterli' — Interior and exterior test scenes from Benedict Bitterli's rendering resources.
• 'isetcam pharr' — Complex scenes from Matt Pharr's PBRT collection (e.g., San Miguel).
• 'isetcam iset3d' — Standard ISET3D test geometries (e.g., SimpleScene, chessSet, Macbeth chart).

---

Hyperspectral Scene Collections (400–950 nm)

The hyperspectral datasets capture spectral radiance across ~100 narrowband channels spanning 400 nm to 950 nm. They were measured in collaboration with Dr. Torbjørn Skauli (Norwegian Defence Research Establishment, FFI) using a custom HySpex hyperspectral imaging camera.

• ISET Hyperspectral Scenes with Fruits and Calibration Charts (fruits-charts): Fruit, vegetables, Macbeth ColorChecker charts, and calibrated targets under controlled illuminants.
• ISET Hyperspectral Scene Data for Landscapes (landscape-hyperspectral): Outdoor natural vegetation, foliage, trees, and sky.
• ISET Hyperspectral Scenes of Human Faces at 1m (High Resolution) (faces-1m): High-resolution spectral measurements of human skin, eyes, and hair under studio illumination.
• ISET Hyperspectral Scenes of Faces at 3m (Lower Resolution) (faces-3m): Hyperspectral portraits captured at a 3-meter distance.

---

Multispectral Scene Collections (Visible and NIR)

The multispectral datasets were captured using customized imaging setups that combine multiple spectral color filters or multi-channel LED illumination arrays with multiple exposure brackets.

1. Visible and NIR filter acquisitions

Captured with a 12-bit scientific CCD/CMOS camera and calibrated optical filters across multiple exposure durations. Multiple exposures expand the dynamic range while the filter sequence enables accurate spectral radiance reconstruction.

• ISET Multispectral Scenes in Visible Range (400–700 nm) (misc-multispectral1): Faces, fruit, vegetables, toys, flowers, and Macbeth ColorCheckers.
• ISET Multispectral Scenes in Visible and Near-Infrared (380–1080 nm) (misc-multispectral2): Fruit, vegetables, books, and calibration charts spanning visible through NIR.
• Reference: Manu Parmar, Francisco Imai, Sung Ho Park, and Joyce Farrell, “A Database of High Dynamic Range Visible and Near-infrared Multispectral Images”, Proc. SPIE 6817, Digital Photography IV (2008).

2. LED-based multispectral lighting acquisitions

Captured using a custom-built LED Multispectral Lighting System (Max Klein & Steven Lansel) featuring 216 LEDs arranged in 8 narrowband clusters between 400 nm and 700 nm. Multiple acquisitions during synchronized LED firings capture spectral reflectance and radiance variations.

• ISET Multispectral Scenes of People (400–700 nm) (people-multispectral): Multi-channel spectral captures of people under narrowband LED illumination.
• Reference: Manu Parmar, Steven Lansel, and Joyce Farrell, “An LED-based lighting system for acquiring multispectral scenes”, Proc. SPIE 8299 (2012).

---

High Dynamic Range (HDR) Scene Collections

Natural scenes frequently span dynamic ranges of $10^4:1$ or higher (e.g., bright sunlight alongside deep shadows). These datasets were acquired with multiple exposure times using linear, calibrated grayscale and color cameras to capture the full dynamic range without sensor saturation or clipping.

• HDR Images of Natural Scenes (hdr-images): High dynamic range scenes of indoor and outdoor environments, shadows within shadows, and bright light sources.
• ISET HDR Auto Lightgroup (isethdr-lightgroup): High dynamic range automotive lighting and road scene captures.
• Reference: F. Xiao, J. DiCarlo, P. Catrysse, and B. Wandell, “High Dynamic Range Imaging of Natural Scenes”, Tenth Color Imaging Conference: Color Science, Systems, and Applications, Scottsdale, AZ, pp. 267–272 (2002).

---

Specialized Research Collections

Additional SDR deposits contain data generated for specific peer-reviewed publications and biological models:

• Deriving the Cone Fundamentals (cone-fundamentals-paper): Spectral data and analysis assets for physiological cone fundamental calculations.
• ISET HDR Sensor Paper (isethdrsensor-paper): Sensor validation and HDR pixel modeling assets.
• ISETBio Retinal Mosaic Data (isetbio-mosaics): Retinal cone mosaic arrays and spatial sampling representations used in ISETBio.
• Stanford Vista Lab Collection (vistalab-collection): Comprehensive collection of datasets deposited by the Stanford Vista Lab.

---

Data Representations and Physical Units

In ISETCam, every spectral scene is a MATLAB structure containing:

1. Spectral Radiance Matrix (scene.data.photons): A 3D array of size $[rows \times cols \times nWavelengths]$. The values represent spectral radiance in physical units: $$\text{photons} \cdot \text{s}^{-1} \cdot \text{m}^{-2} \cdot \text{sr}^{-1} \cdot \text{nm}^{-1}$$ You can also retrieve radiance in energy units ($\text{Watts} \cdot \text{m}^{-2} \cdot \text{sr}^{-1} \cdot \text{nm}^{-1}$) via sceneGet(scene, 'energy').
2. Wavelength Samples (scene.spectrum.wave): A vector specifying the center wavelength of each spectral band in nanometers (e.g., 400:10:700).
3. Illuminant SPD (scene.illuminant): The spectral power distribution of the scene illuminant, enabling illuminant replacement and color constancy experiments (sceneAdjustIlluminant).
4. Spatial Geometry: Scene distance, horizontal field of view, and sample spacing in meters per pixel.

Example: Loading and inspecting a spectral scene

ieInit;

% Load a calibrated spectral scene file
scene = sceneFromFile('MacbethColorChecker.mat', 'multispectral');

% Inspect physical properties
fov     = sceneGet(scene, 'fov');            % Field of view (degrees)
wave    = sceneGet(scene, 'wave');           % Wavelength samples (nm)
photons = sceneGet(scene, 'photons');        % Radiance (photons/s/m^2/sr/nm)
meanLum = sceneGet(scene, 'mean luminance'); % Mean luminance (cd/m^2)

% Change the scene illuminant to CIE D65
sceneD65 = sceneAdjustIlluminant(scene, 'D65.mat');

% Display in the interactive App Designer scene window
sceneWindow(sceneD65);

---

Synthetic 3D Scenes and ISET3D

If your simulation requires full 3D geometry, object motion, depth maps, or custom material and surface textures, use ISET3D. ISET3D uses PBRT (Physically Based Ray Tracing) to render full 3D computer graphics models and outputs native ISETCam scene or opticalimage structures that feed directly into ISETCam sensors and image processing pipelines.