VR Photo Training System

In this repository, we provide information that contributed to the creation of the VR Photo Training System. This includes the machine learning models we used, the datasets we created, and the annotation system we developed.

System diagram

We utilize the following four machine learning models for evaluating and recommending photographs.

1. NIMA model ("NIMA: Neural Image Assessment")
2. Gated CNN model ("Gated CNN for visual quality assessment based on color perception")
3. VEN model ("Good View Hunting: Learning Photo Composition from Dense View Pairs")
4. VPN model ("Good View Hunting: Learning Photo Composition from Dense View Pairs")

Datasets

In this study, we created three datasets to realize the evaluation of photographs in a VR environment:

1. VR Photo Aesthetic Dataset is used to fine-tune the NIMA model (Download Data from Google Drive)
2. VR Photo Composition Dataset is used to fine-tune the VEN and VPN models (Download Data from Google Drive)
3. VR Photo Color Dataset is used to train the Gated CNN models (Download Data from Google Drive)

Also, we use three existing datasets:

1. AVA Dataset is used to pre-train the NIMA model (Data)
2. CPC Dataset is used to pre-train the VEN and VPN models (Data)
3. FLMS Dataset is used to evaluate the VEN and VPN models (Data, Annotation)

How to utilize this system

Illustrations of using the VR Photo Training System

Firstly, use the VR controller to select camera parameters such as aperture value and lens focal length from a panel within the VR interface (left image). After parameter selection, manipulate the virtual camera using the VR controller to take a photo of the subject (center image). Select the captured photo from the virtual album for it to be automatically evaluated (right image). If required, a sample image demonstrating a more optimal composition will be presented.