surfWaveTrack – Data processing pipeline for analysis of surface wave video recordings

Github

General Notes

This code was developed by the Romanczuk-lab for the analysis of surface waves collectively generated by the fish species ''Poecilia Sulphuraria'' endemic in sulphuric springs in Mexico. The social transmission of the diving behavior causes waves which cover up to 60 squaremeter, whereby each squaremeters can contain roughly between 1000 and 3000 individual fish (bodylength of about 2.5cm).

perspective transformed videos

The code is meant to be run on calibrated videos (with for example a square-shaped calibration frame) with a camera perspective transformation (homography) to obtain top view of the waving activity. Additionally it provides the pixel to meter conversion factor.

wave-front localization

First, video are transformed to gray scale. Second, the wave activity is quantified through the application of a standard deviation filter on the grayscale videos: For each pixel in a video frame, for a small windows of M x M pixels around the focal pixel the std. deviations of gray scale values is computed. This yields a standard deviation array Sframe of the same dimensions as the input image, where each element of Sframe represents the standard deviation around the corresponding pixel in the video frame. In general, large standard deviations indicate the presence of an edge (wave front) in the corresponding window around the focal pixel. In order to obtain the background B for each video, the standard deviation array is averaged either over the time before the bird attack (or the projectiles impact) or, if the recorded time before attack/shot is too short, over the undisturbed part of the subsequent video (maximally seperated by 2 minutes). If the background is created also from the time after the attack, the background standard deviation array would be larger in areas with more waves which would make it harder to detect waves in these regions. Then for each frame the difference array Dframe is computed by subtracting the background from the standard deviation array: Dframe=Sframe-B. Finally, pixels in a single frame are classified as active, i.e. containing a wave front, whenever Dframe is larger than a threshold T. Through this binarization the two-dimensional activity map versus time A(t) is obtained. Changing light conditions can cause different contrasts of the videos. Therefore the parameter of the standard deviation filter can be adjusted.

single out individual waves

Individual waves correspond to spatio-temporally connected active regions. They are extracted by applying a standard blob-detection algorithm to the three-dimensional data (two spatial dimensions + time). The number of unique pixels belonging to such a 3d blob corresponds to the total area covered by the wave object. The blobs can be filtered to i) exclude floating dirt unrelated to the waving activity and ii) to exclude small waves which might also be caused by either false positive activity detections or small water surface perturbations (e,g. due to small non-propagating waves). To filter out dirt, all 2d blobs (only spatial dimensions) below a certain (dirt typical) size can be neglected. Then, to exclude small waves, only 3d blob objects with a total unique area covered of above a certain are considered.

calculating speed of the wave front

The average speed of a wave is calculated from the displacement of the leading edge of the wave-front between two subsequent frames. It is the average over all spatial front positions and all frames.

Required python packages:

The code runs in the anaconda environment specified in environment.yml which can be created from the projects root directory via

conda env create -f environment.yml

which creates the environment surfWave (name specified in environment.yml) which you activate by

conda activate surfWave

As alternative we documented from this environment the python packages in use in requirements.txt (created via pip freeze > requirements.txt). In case you want to update your python packages to enable the code to run the most important packages and their versions are listed in environment.yml.

Docker alternative

We will soon provide a Docker file.

Example code

This package contains 2 short videos in data which can be processed by changing the directory to cd surfWaveTrack and run python3 main.py. It will analyze the video as described above (in General Notes).

Open tasks

• "We consider connected cubes only in the time-forward direction as belonging to one cluster in order to avoid counting two colliding clusters with uncorrelated origins as one coherent structure. When two waves collide to form one new wave, the coherent cluster corresponding to one of the incoming waves is terminated at the collision." - Noise driven avalanche behavior in subexcitable media (1999)

License

Copyright (C) 2016-2020 Pawel Romanczuk, Pascal Klamser, Carsten Roesner

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.