Python implementation of Typhoon motion estimator: dense estimation of 2D/3D optical flow on wavelet bases, primarily aimed at fluid motion estimation.
At the moment, the wavelet-based data DFD term (Dérian et al., 2013) only is provided: the high-order regularizers (Kadri-Harouna et al., 2013) are not included in this implementation.
The reference implementation used in (Dérian et al., 2015) and (Dérian et al., 2017) is written in C++ and GPU-accelerated with CUDA, and contains the high-order regularizers. It is the property of Inria (FR) and the CSU Chico Research Foundation (Ca, USA), and can be licensed from these institutions. This Python implementation is not the same as the reference for many reasons, and it is obviously much slower.
- Numpy, Scipy;
- PyWavelets;
- Pillow and Matplotlib for the demos.
Tested with Anaconda Python 3.6.1, Numpy 1.12.1, Scipy 0.19.1, PyWavelet 0.5.2.
The Typhoon class can be imported from other modules/scripts to perform estimations as needed.
The script can also work as a standalone estimator in simple cases, e.g.:
python pytyphoon.py -i0 path/to/im0.jpg -i1 path/to/im1.jpg -wav 'db3' --display
will solve the problem for image pair (im0.jpg, im1.jpg) and wavelet Daubechies-3.
See python pytyphoon.py -h for the complete list of parameters.
Typhoon solves a dense variational optical flow problem, that is to say: (i) it provides one motion vector at every pixel of input images and (ii) it estimates the entire vector field altogether.
To do so, it looks for the motion field which minimizes the displaced frame difference (DFD):
This is achieved by minimizing the following functional:
where the integral is taken over the image. The functional above is non-linear with respect to the motion field. This has the advantage of better handling large displacement, but complicates the minimization process.
For the non-linear minimization to succeed, the solution should lie reasonable "close" to the first guess. This is where wavelets bases come into play: by providing a multiscale representation of the motion field, they enable to estimate the motion iteratively from its coarsest scales to the finests.
The minimization is handled by L-BFGS, which is efficient memory-wise and only requires the functional value and its gradient.
These demos are shipped with the project.
Simple 2d estimation using synthetic particle images (256x256 pixels) originally created for the FLUID project (image database #1). Run:
python pytyphoon.py --demo particles
Simple 3d estimation using synthetic images (64x64x64 pixels) obtained by filtering random normal noise at various scales. The displacements are integer shifts along each of the 3 dimensions. Run:
python pytyphoon.py --demo 3dshift
Simple 3d estimation using synthetic images (96x96x96 pixels) obtained by filtering random normal noise at various scales. The displacement field is a column vortex (first two axes) with an updraft increasing linearly along the third axis. Run:
python pytyphoon.py --demo 3dvortex
- (Dérian et al., 2017) Dérian, P. & Almar, R. "Wavelet-based Optical Flow Estimation of Instant Surface Currents from Shore-based and UAV Video". IEEE Transactions on Geoscience and Remote Sensing, Vol. 55, pp. 5790-5797, 2017.
- (Dérian et al., 2015) Dérian, P.; Mauzey, C. F. and Mayor, S. D. "Wavelet-based optical flow for two-component wind field estimation from single aerosol lidar data". Journal of Atmospheric and Oceanic Technology, Vol. 32, pp. 1759-1778, 2015.
- (Dérian et al., 2013) Dérian, P.; Héas, P.; Herzet, C. & Mémin, E. "Wavelets and Optical Flow Motion Estimation". Numerical Mathematics: Theory, Method and Applications, Vol. 6, pp. 116-137, 2013.
- (Kadri-Harouna et al., 2013) Kadri Harouna, S.; Dérian, P.; Héas, P. and Mémin, E. "Divergence-free Wavelets and High Order Regularization". International Journal of Computer Vision, Vol. 103, pp. 80-99, 2013.
- NetCDF for output results? in standalone mode;
- support of masks;
- some regularizers;
- alternative penalization functions;
- divergence-free wavelets;
- ...