Digital cameras are ubiquitus and are slowly finding their way in niche application areas, such as imaging below the surface of the sea. Underwater imaging is becoming popular among consumers (GoPro action cameras) as well as professionals (underwater exploration).
Underwater imaging faces many more challenges than imaging on the ground, for example excessive absorption of the red light component or sever backscattering. Realistic simulations enable to explore some of those limitations without the need for troublesome experiments.
This project provides a set of tools to realisticly simualte the appearance of underwater scenes in a variety of water conditions. Simulated images faciliate evaluation, and comparisons across different underwater imaing systems and correction algorithms.
If you use these tools please cite
@inproceedings{blasinski2017uiss,
title={Underwater Image Systems Simulation},
author={Blasinski, Henryk and Lian, Trisha and Farrell, Joyce},
booktitle={Imaging and Applied Optics Congress},
year={2017},
organization={OSA}
}
To succesffuly run the scripts in this project the following dependencies need to be installed.
- Homebrew - a package manager for Mac OSX, it will make installing many components much easier.
- RenderToolbox4 - a set of Matlab tools to create, manipulate and render complex 3D scene files.
- Docker - RenderToolbox4 uses either Mitsuba or PBRT renderers to produce final images. Rather than install these programs we provide their dockerized implementations. A cross-platform Docker engine is required to run these.
- ISET - Image Systems Engineering Toolbox for camera sensor simulations. A light version of ISET is provided with this code repository.
- DCRaw - a simple program for reading raw camera images.
- CVX - a Matlab toolbox for convex optimization. As an alternative MATLAB Optimization Toolbox can also be used. As of 06/2017 CVX is only supported on MATLAB 2016b and older.
We have successfully installed and run the simulator on OSX and Linux (Debian) systems on an number of MATLAB releases prior to 2017a. If you are interested in using the code on a Windows machine the biggest limitation is RenderToolbox4, which is not officially supported on Windows (but we have heard of some successful installations).
In order to reproduce the results from our publications please download the input data (underwater images, 3D models) from Stanford Digital Repository.
The Stanford Digital Repository contains a single, ~3GB .zip file. Once you extract it's contents you will see two directories, please move them to the github directory for this project.
- Images contains two sets of real images captured with a Canon G7X camera. The first set are underwater images of a Macbeth chart captured in a few locations in the West Indies. The second set are images of a white Spectralon target illuminated with monochromatic light. This set was used to derive the camera responsivity curves.
- Results are outputs of some of the scripts we provide. You should be able to re-generate this data. We only provide this directory for your reference and so you can run the scripts that use the data in different analyses.
- Code contains all the Matlab source code.
- Code/Utilities a number of 'unrelated' functions we use thourghout the project (for example XML file parsing)
- Code/Remodellers functions used by RenderToolbox4 to change the scene geometry or other properties. Please refer to RenderToolbox4 docummentation for details.
- Code/VideosAndFigures functions to generate sample images, figures and videos used in presentations and publications.
- Figures images and movies that were generated from the simulation results. For example the two figures on top of this page.
- Parameters contains calibration data for different cameras and light sources. For example the spectral responsivities of the Canon G7X used in the experiments.
- Scenes 3D scene files used in simulations. We usually generate these scenes in Blender
To install Homebrew run the following in your terminal window.
>> /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Update apt-get
>> sudo apt-get update
>> sudo apt-get upgrade
The RenderToolbox code is contained in three github repositories: RenderToolbox4, mexximp and mPbrt, you will need to clone all of them.
>> git clone https://github.com/RenderToolbox/RenderToolbox4.git
>> git clone https://github.com/RenderToolbox/mexximp.git
>> git clone https://github.com/RenderToolbox/mPbrt.git
We have found that the official installation instructions may not always work. Since this toolbox requires a few dependencies, we also describe the required steps below.
Install this library to read high dynamic range, multispectral images.
MacOS:
>> berw install openexr
Linux:
>> sudo apt-get install libopenexr-dev openexr
Assimp is a library designed to read different 3D object file formats (such as .obj, .fbx etc) into a common representation. It can be easily installed on MacOS, but on Linux it needs to be complied from source.
MacOS:
>> brew install assimp
Linux:
>> sudo apt-get install zlib1g-dev cmake
>> git clone https://github.com/assimp/assimp.git
>> cd ./assimp
>> cmake CMakeLists.txt -G 'Unix Makefiles'
>> make
Mexximp is a Matlab wrapper around the Assimp library. It requires MEX compiliation of a few files. To install, open MATLAB, go to the Mexximp root directory mexximp and type
makeMexximp
This script will compile all the necessary files.
The RenderToolbox uses a Matlab wrapper around the OpenEXR library. These wrappers also need to be MEX compiled. Open MATLAB, go to RenderToolbox4/Utilities/ReadMultispectralEXR/ReadMultichannelEXR directory and run
rtbMakeReadMultichannelEXR
Finally, you need to configure RenderToolbox preferences. Still in MATLAB, go to the RenderToolbox4 root directory and run
rtbLocalConfigTemplate
You may want to edit this script if you would like to change RenderToolbox preferences.
Don't foret to add all RenderToolbox4, mexximp and mPbrt directories (and all subdirectories) to MATLAB PATH. Now you should be able to run a simple demo script rtbMakeChessSet.
Go to Docker website and search for the installation package. We use Docker Community Edition for Mac. Follow the installation instructions. Once Docker is set up, the software will automatically download the necessary image rendering components from the cloud.
Run the following command in your terminal window.
MacOS:
>> brew install dcraw
Linux:
>> sudo apt-get install dcraw
** Apt-get installs a version of DCRaw (9.17) with a small bug where the raw camera data is read in without subtracting the dark pixel level. Version 9.27 works correctly, but has to be installed manually. Please follow the instructions posted on DCRaw website.**
Visit the CVX site to download the standard CVX bundle for Mac OSX. Unzip the archive in your target directory (for example ~/Documents/MATLAB). After you launch MATLAB, set the CVX directory as the working folder and run the cvx_setup.m script, which will set up CVX on your machine.
Clone this repository to your local drive
>> git clone https://github.com/hblasins/uwSim.git
** Each time you start MATLAB run Code/install.m function to correctly set up MATLAB PATH in your environment. **
To get you started please have a look (and run) at the Code/s_uwRenderUnderwaterChart.m script. This is a simple script that renders an image of a Macbeth chart submertged in water. You can specify a number of parameters that affect the final image appearance :
- depth
- camera to chart distance
- chlorophyll concentraion
- colored dissolved organic matter (CDOM) concentraion
- small particle concentration
- large particle concentration
Once the computation is completed you should see the following image.
Many other scripts in the Code directory are a variation on the renderUnderwaterChart.m where we simply vary or co-vary different parameters and produce a large number of output images.