IlluminationSpectraDataset

Code for the paper Variation of outdoor illumination as a function of solar elevation and light pollution by Spitschan, Aguirre, Brainard & Sweeney (2016). The code provided here performs calibration of the spectrometers, pre-processing of the raw, uncalibrated spectra and figure generation for the figures in the paper.

Authors

The code provided here was written by Manuel Spitschan. The scientific work was performed by Manuel Spitschan, Geoffrey K. Aguirre, David H. Brainard and Alison Sweeney, University of Pennsylvania.

Citation

If you use this code or data set, please cite it as

Spitschan M, Aguirre GK, Brainard DH & Sweeney AM (2016). Variation of outdoor illumination as a function of solar elevation and light pollution. Scientific Reports 6, 26756. doi:10.1038/srep26756.

Clone the repository

The URL for the repository is https://github.com/spitschan/IlluminationSpectraDataset.git. Use your favorite git client to clone the repository or type the following in the command line:

git clone https://github.com/spitschan/IlluminationSpectraDataset.git

Dependencies, third party software and requirements

The code provided here relies on functions from the Psychtoolbox. A small number of functions in IlluminationSpectraDataset/code/helpers were obtained from third party sources and have been attributed in IlluminationSpectraDataset/code/Contents.m. The software provided here was developed and tested on MATLAB 2015a (8.5.0.197613, 64-bit, maci64) on Mac OS X 10.11.3 (El Capitan).

The raw data are available on the FigShare repository (doi:10.6084/m9.figshare.2009070.v1).

Setting up paths

1. In MATLAB, go to the folder where you have cloned in the repository.
2. Add it recursively to the path:

>> addpath(genpath(pwd))

Spectrometer calibration

The spectrometer give us uncalibrated spectra but we have produced a pipeline to calibrate these to convert the spectra to absolute downwelling vector irradiance.

1. Download the spectrometer calibration data (300.55 MB) from the FigShare repository (doi:10.6084/m9.figshare.2009070.v1). This tarball contains measurements of the spectrometer properties needed reproduce the calibration steps described in the paper.
2. Unpack the repository into IlluminationSpectraDataset/calibration. The resulting folder should be called IlluminationSpectraDataset/calibration/calibrationdata. Note that this folder is not subject to git versioning as it is included in .gitignore.
3. Run the calibration for the spectrometers:

>> IlluminationSpectraDataset_SpecCal_InitCalFile('a');
>> IlluminationSpectraDataset_SpecCal_InitCalFile('b');

4. Note that the low-sensitivity spectrometer is called 'a' and the high-sensitivity spectrometer is called 'b'.
5. The calibration will produce two calibration files in IlluminationSpectraDataset/calibration/cal called OO_USB2000+a.mat (for the 'a' spectrometer) and OO_USB2000+b.mat (for the 'b' spectrometer).
6. There will also be plots produced inIlluminationSpectraDataset/calibration/cal/plots which form the basis for supplementary figures S5-S7 of Spitschan et al. (2016).

Data preprocessing

1. Download the raw data (669.38 MB) from the FigShare repository (doi:10.6084/m9.figshare.2009070.v1). This tarball contains the raw, uncalibrated spectra from the paper.
2. Unpack the repository into directly into the root repository IlluminationSpectraDataset/. The resulting folder should be called IlluminationSpectraDataset/dataraw. Note that this folder is not subject to git versioning as it is included in .gitignore.
3. Run IlluminationSpectraDataset_Analysis_Preprocess to pre-process the raw, uncalibrated spectra.
4. This will produce two CSV files with the calibrated data in IlluminationSpectraDataset/data: CSSP_spectra.csv (Rural data) and DRL_spectra.csv (City data).
5. CSSP_quality.csv and DRL_quality.csv are also produced by this function and contain information about the included spectra.

Data analysis

1. To run the full analysis, run IlluminationSpectraDataset_Analysis_FullAnalysis. This script will generate the Figures 1-6 and S1-S4 from the paper. Note that figures S5-S7 are generated by the calibration step above.

Figures and tables

The code provided here reproduces all elements from Figures 1-6, Supplementary Figures S1-S7 and Tables S1-S7. Note that the figures produced are raw versions of the ones produced in the paper, which have been edited in Adobe Illustrator. The correspondences of the figures in the paper and supplementary material and the figures produced here are as follows.

Figure/table in paper	Path to figure/table produced here
Figure 1	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_Figure1A.pdf IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_Figure1B.pdf Color maps in IlluminationSpectraDataset/analysis/results/ColorMaps
Figure 2	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_Figure2.pdf
Figure 3	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_Figure3.pdf
Figure 4	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_Figure4a.png IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_Figure4b.png
Figure 5	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_Figure5.pdf
Figure 6	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_Figure6.pdf
Figure S1	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_FigureS1A.pdf IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_FigureS1B.pdf
Figure S2	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_FigureS2.pdf
Figure S3	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_FigureS3.pdf
Figure S4	IlluminationSpectraDataset/analysis/results/IlluminationSpectraDataset_Analysis_FigureS4.pdf
Figure S5	IlluminationSpectraDataset/calibration/cals/plots/SpectralStatsAnalysis_FigureS5_DarkNoiseSpectra_a.pdf IlluminationSpectraDataset/calibration/cals/plots/SpectralStatsAnalysis_FigureS5_DarkNoiseSpectra_b.pdf
Figure S6	IlluminationSpectraDataset/calibration/cals/plots/IlluminationSpectraDataset_Calibration_FigureS1_WlCorrection_a.pdf IlluminationSpectraDataset/calibration/cals/plots/IlluminationSpectraDataset_Calibration_FigureS1_WlCorrection_b.pdf IlluminationSpectraDataset/calibration/cals/plots/IlluminationSpectraDataset_Calibration_FigureS1_WlCorrectionInset_a.pdf IlluminationSpectraDataset/calibration/cals/plots/IlluminationSpectraDataset_Calibration_FigureS1_WlCorrectionInset_b.pdf
Figure S7	IlluminationSpectraDataset/calibration/cals/plots/IlluminationSpectraDataset_Calibration_FigureS3_RelativeSensitivityCalibration_a.pdf IlluminationSpectraDataset/calibration/cals/plots/IlluminationSpectraDataset_Calibration_FigureS3_RelativeSensitivityCalibration_b.pdf
Table S1	(Column vector 280-840 nm with 1 nm spacing.)
Table S2	IlluminationSpectraDataset/data/CSSP_spectra.csv
Table S3	IlluminationSpectraDataset/data/DRL_spectra.csv
Table S4	IlluminationSpectraDataset/analysis/data/B_CIE3x/B_CIE3R.csv
Table S5	IlluminationSpectraDataset/analysis/data/B_CIE3x/B_CIE3C.csv
Table S6	IlluminationSpectraDataset/analysis/data/B_CIE3x/w_CIE3R.csv
Table S7	IlluminationSpectraDataset/analysis/data/B_CIE3x/w_CIE3C.csv

Sources of data

In IlluminationSpectraDataset/analysis/data/, we provide the following basis functions and data sets:

• B_cie - CIE basis functions from Wyszecki & Stiles (1982), page 762.
• B_Granada - Basis functions from the Granada model, obtained from the authors' website (Hernández-Andrés et al., 2001).
• spd_DiCarloWandell - DiCarlo & Wandell (2000) data set, obtained from the authors.
• spd_Granada - Granada spectral data set, obtained from the authors' website (Hernández-Andrés et al., 2001).
• spd_LightPollution - Light pollution spectrum, digitized from Cronin et al. (2014), Figure 2.12, page 22.
• spd_Zabriskie - Light pollution-free night spectrum, digitized from Cronin et al. (2014), Figure 2.14, page 24.

Full references

• Cronin, T.W., Johnsen, S., Marshall, J., and Warrant, E.J. (2014). Visual ecology, (Princeton: Princeton University Press).
• DiCarlo, J.M., and Wandell, B.A. (2000). Illuminant estimation: beyond the bases. In IS&T/SID Eighth Color Imaging Conference. (Scottsdale, AZ), pp. 91-96.
• Hernández-Andrés, J., Romero, J., Nieves, J.L., and Lee, R.L. (2001). Color and spectral analysis of daylight in southern Europe. Journal of the Optical Society of America A 18, 1325. doi:10.1364/josaa.18.001325.
• Wyszecki, G., and Stiles, W.S. (1982). Color science: concepts and methods, quantitative data and formulae, 2nd Edition, (New York: Wiley).

Contact

For any questions, bug reports, and comments please contact Manuel Spitschan (spitschan@stanford.edu).