Validation

The validation code is used to make sure

• the physical calculations are correct, and
• additions or changes to the code do not introduce errors or (even worse) inaccuracies in existing calculations.

To achieve these goals, the ISETBIO validation functions (ieValid*) check certain calculations against first principles (e.g., diffraction) and against independent code that performs the same calculations (e.g. Psychophysics Toolbox). We also perform regression testing to compare current and previous outputs that are stored on the Remote Data site.

Configuration

Remote Data Toolbox

• Clone the RemoteDataToolbox repository, and add it to your Matlab path.

The full validation dataset is too large to include with the distribution in GitHub. So, we store the dataset on the Remote Data site. Test stimuli for validation are stored on an Amazon machine that we address with the Remote Data Toolbox. To read about the validation stimuli and how they were computed visit the ISETBIO data page.

You can test that the RemoteDataToolbox is properly installed by running

rd = RdtClient('isetbio');
rd.openBrowser

Unit Test Toolbox

• Clone the UnitTestToolbox repository, and add it to your Matlab path
• Modify and execute isetbioLocalHookTemplate.m in the configuration sub-directory.
  • The settings (paths) in isetbioLocalHookTemplate may need to be adjusted; but often they will just work for you.
  • Typically, isetbioLocalHookTemplate only needs to be executed once as it uses Matlab's setpref/getpref mechanism to store the required information on your system.

Validation tests

To run the core full validation suite you can invoke

ieValidateFullAll

This takes a few minutes (5-10 min) to run. Perhaps you would enjoy a cup of coffee or some time to plan your next vacation while it completes.

If one of the scripts fails and you would like to explore why, you can run

ieValidateFullOne

This routine prints a list of the validation scripts and asks you to select one. You can set a break point within that script to debug the problem.

Note, however, that ieValidateFullAll and ieValidateFullOne trap errors, so that if the script is actually failing to execute (a rare circumstance, we hope), you may want to simply run the problematic script directly from the command line. This will not check against the stored validation data, but will show you the typical Matlab printout when an error occurs.