Analysis of spatiotemporal consistency

[pattacini]

We need to design an experiment to evaluate how well we can identify/track a skeleton based on spatiotemporal persistence.

Ideally, we should be able to re-identify as the same skeleton that one disappearing and reappearing due to a temporary occlusion.

Use only the RealSense for this study and log all the relevant information to then analyze data consistency offline.

To begin with, let's focus on skeleton data only. Later on, we might consider other visual sources (e.g. visual cues attached to skeleton's segments).

[vvasco]

The scenario considered for the analysis is the following: a person walks towards the camera and a first distractor appears, occluding the person. After a while, a second distractor also appears, occluding the person for longer. The person finally re-appears in a different position.
The following image shows the trajectories traced from each skeleton id:

The skeleton 28 appears, after a while the first distractor 29 occludes it and 28 becomes 30. Then a new distractor 31 appears and 30 becomes 32.

Data look consistent to apply constraints on the spatio-temporal consistency of the current skeleton, for example an emi-sphere can be associated to the current skeleton, with a radius decreasing over time. A skeleton falling in the emi-sphere would then be associated to the current skeleton.
Additional features can be used to make the strategy more robust, such as mean colors of the skeleton segments to discriminate clothes or mean length of the skeleton segments to discriminate adults and children.

Data and script can be found here:
opc+script.zip

[pattacini]

Awesome analysis!