Python implementation of Typhoon algorithm: dense estimation of 2D-3D optical flow on wavelet bases.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
demo
doc
LICENSE
README.md
pytyphoon.py

README.md

PyTyphoon

Python implementation of Typhoon motion estimator: dense estimation of 2D/3D optical flow on wavelet bases, primarily aimed at fluid motion estimation.

Important remarks

At the moment, the wavelet-based data DFD term (Dérian et al., 2013) only is provided: the high-order regularizers (Kadri-Harouna et al., 2013) are not included in this implementation.

The reference implementation used in (Dérian et al., 2015) and (Dérian et al., 2017) is written in C++ and GPU-accelerated with CUDA, and contains the high-order regularizers. It is the property of Inria (FR) and the CSU Chico Research Foundation (Ca, USA), and can be licensed from these institutions. This Python implementation is not the same as the reference for many reasons, and it is obviously much slower.

Requirements

Tested with Anaconda Python 3.6.1, Numpy 1.12.1, Scipy 0.19.1, PyWavelet 0.5.2.

Usage

The Typhoon class can be imported from other modules/scripts to perform estimations as needed.

The script can also work as a standalone estimator in simple cases, e.g.:

python pytyphoon.py -i0 path/to/im0.jpg -i1 path/to/im1.jpg -wav 'db3' --display

will solve the problem for image pair (im0.jpg, im1.jpg) and wavelet Daubechies-3. See python pytyphoon.py -h for the complete list of parameters.

How does it work?

Typhoon solves a dense variational optical flow problem, that is to say: (i) it provides one motion vector at every pixel of input images and (ii) it estimates the entire vector field altogether.

To do so, it looks for the motion field which minimizes the displaced frame difference (DFD):

DFD

This is achieved by minimizing the following functional:

DFD functional

where the integral is taken over the image. The functional above is non-linear with respect to the motion field. This has the advantage of better handling large displacement, but complicates the minimization process.

For the non-linear minimization to succeed, the solution should lie reasonable "close" to the first guess. This is where wavelets bases come into play: by providing a multiscale representation of the motion field, they enable to estimate the motion iteratively from its coarsest scales to the finests.

The minimization is handled by L-BFGS, which is efficient memory-wise and only requires the functional value and its gradient.

Demos

These demos are shipped with the project.

(2D) Synthetic particle images

Simple 2d estimation using synthetic particle images (256x256 pixels) originally created for the FLUID project (image database #1). Run:

python pytyphoon.py --demo particles

Particle results

(3D) Homogeneous shift

Simple 3d estimation using synthetic images (64x64x64 pixels) obtained by filtering random normal noise at various scales. The displacements are integer shifts along each of the 3 dimensions. Run:

python pytyphoon.py --demo 3dshift

3dshift results

(3D) Column vortex

Simple 3d estimation using synthetic images (96x96x96 pixels) obtained by filtering random normal noise at various scales. The displacement field is a column vortex (first two axes) with an updraft increasing linearly along the third axis. Run:

python pytyphoon.py --demo 3dvortex

3dshift results

References

  • (Dérian et al., 2017) Dérian, P. & Almar, R. "Wavelet-based Optical Flow Estimation of Instant Surface Currents from Shore-based and UAV Video". IEEE Transactions on Geoscience and Remote Sensing, Vol. 55, pp. 5790-5797, 2017.
  • (Dérian et al., 2015) Dérian, P.; Mauzey, C. F. and Mayor, S. D. "Wavelet-based optical flow for two-component wind field estimation from single aerosol lidar data". Journal of Atmospheric and Oceanic Technology, Vol. 32, pp. 1759-1778, 2015.
  • (Dérian et al., 2013) Dérian, P.; Héas, P.; Herzet, C. & Mémin, E. "Wavelets and Optical Flow Motion Estimation". Numerical Mathematics: Theory, Method and Applications, Vol. 6, pp. 116-137, 2013.
  • (Kadri-Harouna et al., 2013) Kadri Harouna, S.; Dérian, P.; Héas, P. and Mémin, E. "Divergence-free Wavelets and High Order Regularization". International Journal of Computer Vision, Vol. 103, pp. 80-99, 2013.

Todo

  • NetCDF for output results? in standalone mode;
  • support of masks;
  • some regularizers;
  • alternative penalization functions;
  • divergence-free wavelets;
  • ...