Project Description

This project is centered around the Python script parse_camm.py, which extracts and processes motion data from videos recorded by 360-degree cameras, specifically the Ricoh Theta V. The main type of data extracted is the 'camm' track data, which includes Angular Velocity and Acceleration for each video frame. These are then used to compute the Pitch and Roll of the camera.

CAMM Data Description

The 'camm' track in a video file is a specific data track containing motion-related information that the camera's built-in sensors have captured. This motion data comprises readings from the gyroscope (Angular Velocity) and accelerometer (Acceleration), both provided in the Inertial Measurement Unit (IMU) coordinate system.

The gyroscope data represents the rotational speed around each of the XYZ axes in radians/second. On the other hand, the accelerometer data signifies the linear acceleration along each of the XYZ axes in meters/second^2. The relationship between the IMU coordinate system and the camera coordinate system is defined by the applications and usually recommended to be aligned by the camera manufacturers.

It's important to note that the initial readings from the gyroscope are in the IMU coordinate system defined by the driver, and a transformation might be needed to convert it to the camera coordinate system. For further details, refer to Android Sensor.TYPE_GYROSCOPE and Android Sensor.TYPE_ACCELEROMETER.

Data Extraction Process

The extraction process follows two main steps:

1. Using ExifTool: This powerful command-line utility is employed to extract the 'camm' track data from the video file and put it into a text file. The command syntax is as follows: exiftool -ee -V3 video_file.MP4 > output.txt.

2. Parsing with parse_camm.py: This Python script reads the output text file generated by ExifTool and transforms the data into a structured JSON format. It also performs additional computations to calculate the Pitch and Roll of the camera based on the Acceleration data.

Detailed Explanation of parse_camm.py

The parse_camm.py script is the heart of the project, providing several functions for parsing and analyzing the 'camm' data. The script is divided into several functions:

parse_camm_data()

This function serves the purpose of reading the 'camm' data from the output text file produced by ExifTool. The parsing process extracts Angular Velocity and Acceleration values and the associated Sample Time and Sample Duration for each sample. An additional computation is performed here to calculate the Pitch and Roll using the Acceleration data. A UUID is also generated for each sample for identification purposes.

get_video_data()

This function uses ExifTool to extract critical video information, specifically the frame rate and duration. This information is crucial in the subsequent processing of 'camm' data.

get_frame_samples()

This function accepts the parsed 'camm' data and the video frame rate as inputs and aims to identify the closest data sample for each video frame. This is particularly useful when the data samples and video frames are not 1:1, which is often the case.

calculate_pitch_roll()

This function calculates the pitch and roll given the acceleration data. It uses the mathematical formulas for pitch and roll calculation based on acceleration, taking into account the direction of gravity. The pitch and roll are then converted to degrees for easier interpretation.

Output JSON File Documentation

The end product of the parse_camm.py script is a structured JSON file, where each element in the list corresponds to one video frame. Each element is a dictionary with the following keys:

• "angularvelocity": A list of three floating-point numbers indicating the Angular Velocity (gyroscope data) in radians/second.
• "acceleration": A list of three floating-point numbers denoting the Acceleration in meters/second^2.
• "pitch": A floating-point number representing the calculated Pitch of the camera in degrees.
• "roll": A floating-point number indicating the calculated Roll of the camera in degrees.
• "uuid": A unique identifier for each sample.
• "sampletime": A floating-point number showing the time of the sample in seconds.
• "sampleduration": A floating-point number presenting the duration of the sample in seconds.

Usage

The script is designed for command-line usage. You can run the script using the following command:

python parse_camm.py video_file.MP4

Replace video_file.MP4 with the path to your video file. The script will first generate a text file with the 'camm' data extracted by ExifTool. It will then parse this data and output a JSON file with the parsed and calculated data.

Dependencies

The script relies on the following Python packages:

• numpy
• uuid

In addition, the ExifTool command-line utility must be installed and available in the system's PATH.

Note

The script assumes that the camera and IMU coordinate systems are aligned and that the 'camm' data is in the IMU coordinate system. If this is not the case for your camera, additional transformations may be required to convert the data to the camera coordinate system. It's recommended to check the camera's documentation or reach out to the manufacturer for more information.