Analyser Overview

A list of Analysers implemented in vitrivr-engine. This document is both, an overview for users of vitrivr-engine, but also a good starting point for potential contributors.

All descriptors, regardless of analyser do have two sub-fields (in this order): id (DescriptorId) and retrievableId (RetrievableId). The former is the unique id per descriptor, the latter the foreign key of the retrievable.

Core Module

Analysers from the core module.

Exif Metadata

Analyser Name: ExifMetadata

The Exif Metadata analyser is capable of reading exif metadata.

Sub-Fields:

The subfields of this analyser are configurable

For exif metadata, the ExifMetadata Analyser can be used. For each exif tag that should be included, a parameter with the name "{EXIF_DIRECTORY_NAME}_{TAG_NAME}" must be set to a type. Keys that do not match an exif tag via the aforementioned pattern are interpreted to be custom metadata tags that are stored in the exif UserComment tag in JSON format. Each parameter corresponds to a sub-field. Here is an example with custom "time_zone" metadata:

        {
          "name": "exif",
          "factory": "ExifMetadata",
          "parameters": {
            "ExifSubIFD_FocalLength": "INT",
            "ExifSubIFD_ApertureValue": "FLOAT",
            "ExifSubIFD_DateTimeOriginal": "DATETIME",
            "ExifSubIFD_MeteringMode": "STRING",
            "time_zone": "STRING"
          }
        }

For extraction, the exif UserComment of images might look like this:

{"time_zone": "Europe/Berlin", "hours_awake": 12}

File Source Metadata

Analyser Name: FileSourceMetadata

Provides the path of the source file, as well as its size.

Sub-Fields:

Sub-Field	Description	Type
path	The absolut path of the file	String
size	The file's size in bytes	Long

Video Source Metadata

Analyser Name: VideoSourceMetadata

Provides basic metadata of the video file, such as width and height (in pixels), frames per seconds and some.

Sub-Fields:

Sub-Field	Description	Type
width	The width in pixels	Int
height	The height in pixels	Int
fps	The video's frames per second	Double
channels	The video's channels	Int
sampleRate	The sample rate	Int
sampleSize	The sample size	Int

Temporal Metadata

Analyser Name: TemporalMetadata

Provides temporal information (e.g. of the segment)

Sub-Fields:

Sub-Field	Description	Type
startNs	The start point, relative to the beginning and in nanoseconds	Long
endNs	The end point, relative to the beginning and in nanoseconds	Long

Features Module

Analysers from the features module.

Feature Extraction Server Module

M3D Module