AU LightStage Experimentation Framework
Computer Science, Aberystwyth University, Wales, UK
Displays, evaluates and tunes LED light positions and intensities for a simulated object inside a Light Stage geodesic dome.
The key feature of the tool is an Experimentation Framework to evaluate different lighting and capture configurations for their effects on data capture in
3d-reconstruction pipelines via
structure from motion,
photogrammetry and other
multi-viewpoint capture approaches.
Take a read below for details of those features, and how to use them.
Feel free to check out the YouTube videos including this one showing different Light Stage frames under the simulation:
INSTALLATION & PREREQUISITES
Python2.7.x (**Python 3 compatibility migration underway**). PyOpenGL numpy scipy Docker (for hardware in loop feature)
Downloading the Anaconda distribution should be sufficient, except for
Python wrapper for OpenGL. The following methods have been successful for earlier users:
To install PyOpenGL on Windows:
Download: PyOpenGL version '3.1.1b1-cp27' as x64 or x86 from http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyopengl Install: pip install path/to/download/PyOpenGL-3.1.1b1-cp27-none-win_XXXXX.whls
To install PyOpenGL on Linux (Ubuntu/ Debian):
sudo apt install git python2.7 python-opengl python-pip sudo pip install -r requirements.txt
Clone the repo:
git clone https://github.com/LightStage-Aber/LightStage-Repo.git cd src/
Usage Examples below and take a read of
options switches and
configuration file, to make use of the full framework.
BASIC MODES OF USE:
python run.py -h. Displays options on command line. ./properties/default.properties (config file) Contains a number of comfigurations for the application, evaluations, tuning and display modes. python run.py -m1 -e3 Display mode with LightIndexPositions mode, see the visualiation of the configured setup. Effected by ./properties/default.properties file, and command line options switches. Use default low-res target sphere (180 faces), default camera layout and LEDs defined with properties file. python run.py -m2 -e3 Evalation mode python run.py -m3 -e3 Tuning mode
Simply put, the tool is to evaluate light positions or search for positions that are optimised for data capture for 3d reconstruction.
- Current evaluations measure physical effects of light.
- Future evaluations may evaluate effects of positions in calibrated photometry.
A typical experiment might use a spherical lightstage frame on which lights are positioned to illuminate a typically spherical target object. The light received (illuminance) on each surface of the target object is calculated. The balance (i.e. a relative or normalised standard deviation) of the light received across the target, provides a measure of evaluation; this key metric, we call
evenness. Other statistical measures are also reported. Variations include differently structured frames, different targets, light positions, quantities and light power output levels.
Debevec's LS3/LS5 at USC, US 156 LED icosahedron 2v, Dutta & Smith in 2010 at UoYork, UK 41 LED icosahedron 2v, and Kampouris & Ghosh at ICL, UK 168 sphere of rings design and our stage at Aber Uni, UK which is uses a icosahedron 3v structure.The state-of-art lightstages already simulated and evaluated include
video demonstrating Dutta, Debevec and Ghosh's (left-to-right) designs.Here's a
illuminance-based experiments have taken priority over
luminance-based experimentation, as this removes camera positions and surface materials (BRDFs) from the evaluations. Luminance-based approaches do permit measuring (simulated) photometry data to evaluate different set-ups, etc. so they stil exist, but are under re-enable at your own risk status.
Prior experiments have evaluated different approaches and their results remain in
/results/. These results are mostly from searches that outputed results suitable for input as
[LightIndexPositions] (described under Illuminance-based section). They can be visualised/loaded using
-m1 and their corresponding
options switches and
configuration file. These include evaluating the light reflected into a set of cameras (single view or multi-view).
For the illuminance tooling, light positions are defined (in
/properties/default.properties configuration file) through two methods:
- vertex index number of the frame (
[LightIndexPositions], -e3/4) and
- directly by the vertex coordinates (x,y,z) (
These modes are supported and available for Illuminance evaluations (e.g.
For illumance evaluation, light received (illuminance) on the target object surfaces is calculated by the angle of incidence into the Lambertian (diffuse) distribution function. This differs from the luminance-based approach in that it measures absorption, rather than reflectance to a capture point.
Luminance-based evaluations are not currently supported. They were measured by bidirectional reflectance distribution functions (BRDF) such as Blinn-Phong (specular and diffuse) and Lambertian (diffuse only) reflectance models.
Search heuristics to guide luminance-based positioning have been disabled in favour of unguided illuminance evalutions. (**)
- selecting individual LEDs that maximised the lumens value received at the cameras (
maximised intensity), and
- selecting LEDs that individually maximised lumens to the cameras, and also, as a set of LEDs, hit all (
coverage) surfaces of the target object.
** Developers wanting to reinstate this approach should follow the
run.py -m2 -e0 call graph to test and re-establish, and see
Example of Luminance-based results:
Historically, the luminance-based option switches used
-m2 -e1 and
-m2 -e2 to evaluate
maximised intensity and
coverage scores, as shown in the video above.
maximised intensity configuration demo, shown in the video above, as:
python run.py -m1 -p "../models/wheat/wheat1.obj" -s 1 -t '(0,-0.1,0)' -r '../results/blinn-phong+lambert_shading_scores/wheat1_realistic_cam/led_scores_2016-01-12-17-11-28.csv'
Key experiment statistical metrics are printed to STDOUT.
All experiment data is recorded to a CSV file (filename specified in STDOUT) and written to the directory specifed in
light.results_output_file_path_prefix= in the
./properties/default.properties configuration file. Default is a subdirectory of
The output filename corresponds to the experiment type (for example,
*_VertexIndexPositionEvaluator.csv, corresponds to
-e3 experiments (Joint or "Vertex" Positions from Index).
Below is a list of the CSV column header titles (correct at time of writing):
[Qty_Selected_LED_Indexes, Qty_Available_LED_Indexes, total_surface_lambertian_score, normalised_stdev_n, normalised_stdev_n_intensity, MEAN_INTENSITY, UNIFORM_LED_INTENSITIES, coefficient_of_stdev, coefficient_of_iqr_median, surface_mean_score, surface_stdev, surface_median, surface_iqrange, surface_min, surface_max, Evaluator_Shortname, source_filename, Evaluator_ClassName, frame_objfilename, frame_scale, obj.TARGET_SHAPE, obj.TARGET_SCALE, obj.TARGET_TRANSLATION, surfaces_raw_data, timestamp, light_indexes, light_vertices, light_intensities]
Details of the output data format are specified in the functions of
Details of Experiment Configurations
The different types of configuration variations can be configured and executed as follows:
run.py options switches.
- specify target (.obj) -p '/path/to/tri-surface-WaveFront-Object-file.obj' - adjust target scale -s scale (relative to frame, default is 1) - adjust target position -t translate position
See configuration under
default.properties [FrameModel] section.
- specify target (.obj) frame.objfilename=path/to/Poly-Face-WaveFront-Object-file.obj - adjust frame scale frame.scale (relative to target object, default is 8) - remove lower mount points frame.withsupportaccess=True - specify n mount points to remove frame.support_access_vertices_to_remove=1 (remove 1 bottom-most y-axis vertex from frame) - specify frame EdgeN mounting points frame.number_of_vertices_per_edge=1, Edge1, Edge2,.. Edge10.
Lights by Vertex Positions:
See configuration under
default.properties [LightPositions] section.
- specify light positions from file light.objfilename=path/to/Poly-Face-WaveFront-Object-file.obj - This specifies quantity and vertex positions, in WaveFront OBJ format. - specify results output file prefix light.results_output_file_path_prefix=/path/ - adjust vertex position scale light.scale= (relative to target object, should correspond to frame.scale, default is 8) Usage includes Euclidean k-nearest neighbour mapping of floating light vertex positions to frame vertex positions.
Lights by Frame Index Positions:
See configuration under
default.properties [LightIndexPositions] section.
- specify light index positions from file results_file.csvfilename=path/to/filename.csv - This contains a binary column (0/1). Row number corresponds to vertex index number from the frame's OBJ file. - specify the column binary number results_file.column_number= (historical default from the luminance-based results files is, 3) - specify quantity of LEDs to load and display results_file.number_of_leds= (read in n rows with value '1', a count ignoring '0's). - specify results output file prefix results_file.results_output_file_path_prefix=/path/
Light Intensity Configuration (Output per Indexed Light):
See configuration under
default.properties [LightOutput] section.
This section of configuration let's us define a set of light intensities, either by (1) an
enforced_default or by (2) a
column_number specified within a CSV file defined by
filename_path. The baseline value is 1.0.
light.output_intensity_from_index.enforce_default=True light.output_intensity_from_index.default_value=1.0 light.output_intensity_from_index.allow_default=False light.output_intensity_from_index.column_number=4 light.output_intensity_from_index.filename_path=../results/Control_91-92_March2017/edges_l3926.csv
Non-uniform light intensities can improve target lighting balance, see the Brightness Control Tuning section for the automated improvement procedure. However, non-uniform light intensities will effect photometric calucations.
Uniform light intensities have no effect on balance peformance.
Readme Firstrecommendation is to modify and compare only between variations of a single dependent variable (e.g. positions or intensities or target or frame, but not multiple). Drawing statistical comparisons, while modifying the position sets and these intensity values, can be unreliable.
Further explanation of these configuration options and their statistical analysis are released on the accompanying website.
Experiment Configurations to Mount LED Lights to Frame Edges
In real world Light Stages, the LED lights can be mounted anywhere on the frame. The
frame.number_of_vertices_per_edge parameter in the
default.properties config file defines the number of discrete mounting points along each frame edge.
Edge2 is used by setting the value to
2. This will add an extra central point along each edge, such that the vertex mount points include joints and 1 point on each edge. For
Edge2,.. to Edge10, the quantity of mount points is equal to
|V| is the quantity of joints and
|E| is the quantity of edges.
Edge1 is a special case, used by setting the value to
1. This will add a central point along each edge; however, no joints will remain available.
Edge1 delivers a quantity of mount points equal to
This video shows examples of
Edge2 features. On right, Ghosh's design uses
Edge1. In the middle, Debevec's LS3/5 design uses
Edge2. On left, Dutta's design does not use edge mounting.
|Edge 1||Edge 2||Edge 4|
Display Experiment Environment
The tool will visually display the experiment configuration (without running the evaluation). This helps with:
- visual verification of an experiment set-up.
- demonstrations of experiment set-ups.
Together with the associated configuration file and options switches, run with:
python run.py -m1 -e<3,4,7,8,9>
Switch back to experiment mode with:
python run.py -m2 -e<3,4,7,8,9>
|Display example showing an Edge 3 dome visualisation|
- Created an with
Edge 3configuration set-up in
- Run as
python run.py -m1 -e4
Optimise Evenness of any Light Position Set:
Given a set of light positions, the illumination balance can often be improved by fine tuning the balance of intensity (lumens output or power) emitted by each LED light. In other words, this feature is to find a way to lower the standard deviation of the distribution (evaluation data) by effecting the inputs into the scoring function.
In the current release you can enable this feature with mode
-m3. Two strategies to improve the
evenness balance of a set of LEDs are provided.
iterative regressionreduces the target object's surface illumination normalised standard deviation (
evennessmetric) by iteratively modifying the light output intensity value of the LED with an illuminance score furthest from the mean. This is done to lower the standard deviation of the target's surface illumination scores.
L-BFGS-Bsearch method to optimise the
evennessmetric, by modifying the array of light intensity values within a bounded (min/max) range.
Both methods have a set of parameters to configure in
./properties/default.properties under section
[BrightnessControlTuner]. The optimisation search stopping criteria include iterations and thresholds.
How to Run:
The brightness control tuning can be applied to all forms of light positions sets (i.e. Index
-e3/4 and Vertex
Within the configuration file and options switches the optimisation stopping criteria and frame lighting design conditions should be set-up.
Then, run as:
python run.py -m3 -e<3,4,7,8,9>
The key data output from the tuning is an array of new light output intensities.
These are light output values that correspond to the ordered (by vertex order or by index order, depending on the trial) light positions.
The intensity values can inform the power output per light in a physical light stage control system. It is advisable to scaling these values into a [0.0 to 1.0] range, then translate into physical light's control's power (i.e. wattage) range (e.g. [0 to 256]).
The improved light output intensities set (
intensitiescolumn) and corresponding LED index positions (column
led_indexes) are written to a unique output file with identifying suffix information:
- Filename format:
- This file path can be used directly as input to the
- Filename format:
The rest of the result statistics data (including the improved light output intensities set in the
light_intensitiescolumn) is written to the
Recommendations and Reading:
Recommend to start with
iterative regression, which has been beta tested. For sub-optimal position sets, this algorithm can rapidly improve scores. For already optimised positions, this can perform poorly.
Read the algorithms (
./src/modes/brightness_control_tuning/), documentations (e.g. scipy / pydocs) and configuration file settings for more details.
Read and run the unit tests and their configuration changes in
./test/test_BrightnessControl.pywith nose2 or by executing the py file.
OTHER USEFUL DETAILS
Press F1, see the console print outs. Use mouse to drag and zoom view of object. Use spacebar to stop/start display rotation. Press Esc to quit.
/src - Code. /test - Unit tests for src/ code. /models - Sample 3D obj model files. /results - Example result data files for various experiment types and example input .csv/.obj files. /properties - Configuration file to simplify running of experiments, visualisations, etc. (-f 'path/file.properties') /exp - Experiment scripts for batch and single trials. Includes charged particle repulsion positioning (Lettvin) and range testing for evaluating metrics. /vendor - Third party content, including docker container scripts and dependency support scripts.
FORMER LUMINANCE-BASED EXAMPLES:
-m1 -e0) executes the code associated with luminance-based tools for evaluating and viewing lightstage models.
-m1 -e0) viewing code is still operable, such that first example below will display an LED position result set (defined in
python run.py -m1 -p "../models/Flower/plants3.obj" -s 0.01 -t '(0,-4,0)' -r '../results/blinn-phong+lambert_shading_scores/wheat1_realistic_cam/led_scores_2016-01-12-17-11-28.csv' Run in display mode, with plants3.obj target model loaded (-p), with a scaling factor (-s) of 0.01 and translated (-t) -4 on the Y-axis Loads result data file (-r) for the wheat1 model: ../blinn-phong+lambert_shading_scores/wheat1_realistic_cam/led_scores_2016-01-12-17-11-28.csv
Please note: the earlier luminance-based evaluation methods have been disabled in favour of illuminance-based evaluations. The code is still there, but needs testing. Follow the
run.py -m2 -e0 call graph to test and re-establish that evaluation tooling.
python run.py -m 2 Run in evaluation mode (-m), with default mini-dome, default camera layout (-c) and default required LEDs (-l). Creates information file: ../led_meta_data_YYYY-MM-DD-hh-mm-ss.txt Creates result data file: ../led_scores_YYYY-MM-DD-hh-mm-ss.csv
Issues: If you find any issues, please file it.
Slack: We're fairly responsive on slack and you can find us in the #lightstage channel.