Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maybe tidy up optical flow implementations #15394

Open
chacha21 opened this issue Aug 26, 2019 · 1 comment
Open

Maybe tidy up optical flow implementations #15394

chacha21 opened this issue Aug 26, 2019 · 1 comment

Comments

@chacha21
Copy link
Contributor

chacha21 commented Aug 26, 2019

Currently, there are plenty of optical flow implementations in OpenCV, in different modules.
It's currently unclear if some of them are superseding others, if they are equivalent, if they are to be deprecated. They do not share either the same rules to create instances (sometimes through a factory, sometimes not).

Here is a summary of the current state (OpenCV 4.1.1)

-while the implementations are subclasses of cv::Algorithm, none of them is using the read/write parameters set abstraction

-two different implementation of the same opticalflow may not have the same parameters names or even the same parameters set (e.g. Farneback getIterations() vs getNumIters(), or getFastPyramids())

================== module VIDEO ==================
There are a few implementations here and there.

opencv/modules/video/tracking.hpp
	class cv::DenseOpticalFlow : public cv::Algorithm
	class cv::SparseOpticalFlow : public cv::Algorithm 
	class cv::FarnebackOpticalFlow : public cv::DenseOpticalFlow
		factory : cv::Ptr<cv::FarnebackOpticalFlow> cv::FarnebackOpticalFlow::create()
	class cv::VariationalRefinement : public cv::DenseOpticalFlow
		factory : cv::Ptr<cv::VariationalRefinement> cv::VariationalRefinement::create()
	class cv::DISOpticalFlow : public cv::DenseOpticalFlow
		factory : cv::Ptr<cv::DISOpticalFlow> cv::DISOpticalFlow::create()
	class cv::SparsePyrLKOpticalFlow: public cv::SparseOpticalFlow
		factory : cv::Ptr<cv::SparsePyrLKOpticalFlow> cv::SparsePyrLKOpticalFlow::create()

There are only two shortcuts to avoid explicit instanciation. Should it be deprecated or extended to other implementations ?

	void cv::calcOpticalFlowPyrLK() //{cv::SparsePyrLKOpticalFlow}
	void cv::calcOpticalFlowFarneback() //{cv::FarnebackOpticalFlow}

-the cpp files where the implementations are dispatched do not really share the same name logic

opencv/modules/video/optflowgf.cpp:
	cv::Ptr<cv::FarnebackOpticalFlow> cv::FarnebackOpticalFlow::create()
opencv/modules/video/variational_Refinement.cpp:
	cv::Ptr<cv::VariationalRefinement> cv::VariationalRefinement::create()
opencv/modules/video/dis_flow.cpp:
	cv::Ptr<cv::DISOpticalFlow> cv::DISOpticalFlow::create()
opencv/modules/video/lkpyramid.cpp:
	cv::calcOpticalFlowPyrLK() {cv::SparsePyrLKOpticalFlow}
	cv::Ptr<cv::SparsePyrLKOpticalFlow> cv::SparsePyrLKOpticalFlow::create()

================== module CUDAOPTFLOW ==================
the cudaoptflow module treats all the implementations the same way, it is very convenient.

opencv_contrib/modules/cudaoptflow/cudaoptflow.hpp
	class cv::cuda::DenseOpticalFLow
	class cv::cuda::SparseOpticalFlow
	class cv::cuda::NvidiaHWOpticalFlow : public cv::Algorithm
	class cv::cuda::NvidiaOpticalFlow_1_0 : public cv::cuda::NvidiaHWOpticalFlow
		factory ; cv::cuda::NvidiaOpticalFlow_1_0::create()
	class cv::cuda::BroxOpticalFlow: public cv::cuda::DenseOpticalFLow
		factory ; cv::Ptr<cv::cuda::BroxOpticalFlow> cv::cuda::BroxOpticalFlow::create()
	class cv::cuda::SparsePyrLKOpticalFlow: public cv::cuda:: SparseOpticalFlow
		factory ; cv::Ptr<cv::cuda::SparsePyrLKOpticalFlow> cv::cuda::SparsePyrLKOpticalFlow::create()
	class cv::cuda::DensePyrLKOpticalFlow: public cv::cuda:: DenseOpticalFlow
		factory ; cv::Ptr<cv::cuda::DensePyrLKOpticalFlow> cv::cuda::DensePyrLKOpticalFlow::create()
	class cv::cuda::FarnebackOpticalFlow: public cv::cuda:: DenseOpticalFlow
		factory ; cv::Ptr<cv::cuda::FarnebackOpticalFlow> cv::cuda::FarnebackOpticalFlow::create()
	class cv::cuda::OpticalFlowDual_TVL1: public cv::cuda:: DenseOpticalFlow
		factory ; cv::Ptr<cv::cuda::FarnebackOpticalFlow> cv::cuda::FarnebackOpticalFlow::create()

================== module OPTFLOW ==================
Contrary to other modules, the implementations are most of the time private, under a cv::DenseOpticalFlow type.

opencv_contrib/modules/optflow/optflow.hpp
        cv::Ptr<cv::DenseOpticalFlow> cv::optflow::createOptFlow_DeepFlow();
        cv::Ptr<cv::DenseOpticalFlow> cv::optflow::createOptFlow_SimpleFlow();
        cv::Ptr<cv::DenseOpticalFlow> cv::optflow::createOptFlow_FarnebackFlow();
        cv::Ptr<cv::DenseOpticalFlow> cv::optflow::createOptFlow_SparseToDenseFlow();

For some reason, the DualTVL1OpticalFlow is public, with a static factory, in this header file :

	class cv::optflow::DualTVL1OpticalFlow : public cv::DenseOpticalFlow
		factory : cv::Ptr<cv::optflow::DualTVL1OpticalFlow> create()

There are only two shorcuts to avoid explicit instanciation

	void cv::optflow::calcOpticalFlowSF()
	void cv::optflow::calcOpticalSparseToDense()

There is a public header for OpticalFlowPCAFlow , but the creation does not use a static create() factory function :

opencv_contrib/modules/optflow/pcaflow.hpp
	public cv::optflow::OpticalFlowPCAFlow : public cv::DenseOpticalFlow
		no create()
	cv::Ptr<cv::DenseOpticalFlow> cv::optflow::createOptFlow_PCAFlow()

There is a public header for RLOFOpticalFlow , and there is a static create() factory function :

opencv_contrib/modules/optflow/rlofflow.hpp
	cv::optflow::DenseRLOFOpticalFlow : public cv::DenseOpticalFlow
	cv::Ptr<cv::optflow::DenseRLOFOpticalFlow> cv::optflow::DenseRLOFOpticalFlow::create()

	cv::optflow::SparseRLOFOpticalFlow : public cv::SparseOpticalFlow
	cv::Ptr<cv::optflow::SparseRLOFOpticalFlow> cv::optflow::SparseRLOFOpticalFlow::create()

For some reason, there is an alias factory function for those ones :

cv::ptr<cv::DenseOpticalFlow> cv::optflow::createOptFlow_DenseRLOF() {cv::optflow::DenseRLOFOpticalFlow::create();}
cv::ptr<cv::SparseOpticalFlow> cv::optflow::createOptFlow_SparseRLOF() {cv::optflow::SparseRLOFOpticalFlow::create();}

And there are shorcuts for those ones :

cv::optflow::calcOpticalFlowDenseRLOF()
cv::optflow::calcOpticalFlowSparseRLOF()

Deepflow, simpleflow, Farneback are private with no public way to instanciate them :

opencv_contrib/modules/optflow/deepflow.cpp
	class cv::optflow::OpticalFlowDeepFlow: public cv::DenseOpticalFlow

opencv_contrib/modules/optflow/simpleflow.cpp
	no class

opencv_contrib/modules/optflow/interfaces.cpp
	class cv::optflow::OpticalFlowSimpleFlow : public cv::DenseOpticalFlow
		no create()
	class cv::optflow::OpticalFlowFarneback : public cv::DenseOpticalFlow
		no create()
	class cv::optflow::OpticalFlowSparseToDense : public cv::DenseOpticalFlow
		no create()

DualTVL1 is also private, but there is a also private factory :

opencv_contrib/modules/optflow/tvl1flow.cpp
	class cv::optflow::OpticalFlowDual_TVL1 : public cv::DualTVL1OpticalFlow

	cv::Ptr<DualTVL1OpticalFlow> cv::DualTVL1OpticalFlow::create() 

================== module SUPERRES ==================
In this module, there is a public interface, which can hide specific implemenations.
Like in the the cudaoptflow module, it treats all the implementations the same way, it is very convenient. But it is not the same logic as in cudaoptflow. Should it be made uniform ?

opencv_contrib/modules/superres/optical_flow.hpp
class cv::superres::DenseOpticalFlowExt : public cv::Algorithm
class cv::superres::FarnebackOpticalFlow : public cv::superres::DenseOpticalFlowExt
cv::Ptrcv::superres::FarnebackOpticalFlow cv::superres::createOptFlow_Farneback();
cv::Ptrcv::superres::FarnebackOpticalFlow cv::superres::createOptFlow_Farneback_CUDA();

class cv::superres::DualTVL1OpticalFlow : public cv::superres::DenseOpticalFlowExt
cv::Ptr<cv::superres::DualTVL1OpticalFlow> cv::superres::createOptFlow_DualTVL1();
    cv::Ptr<cv::superres::DualTVL1OpticalFlow> cv::superres::createOptFlow_DualTVL1_CUDA();

class cv::superres::BroxOpticalFlow : public cv::superres::DenseOpticalFlowExt
    cv::Ptr<cv::superres::BroxOpticalFlow> cv::superres::createOptFlow_Brox_CUDA();

class cv::superres::PyrLKOpticalFlow : public cv::superres::DenseOpticalFlowExt
    cv::Ptr<cv::superres::PyrLKOpticalFlow> cv::superres::createOptFlow_PyrLK_CUDA();
@mshabunin
Copy link
Contributor

@chacha21 , we have related section in How to contribute wiki page. Private implementation can use different approaches as long as public interface is aligned with the guidelines.

Functions like calcOpticalFlowPyrLK are there for historical reasons and we don't plan to remove them or add similar shortcuts for other algorithms.

Classes in superres module are wrapping algorithms from other modules, maybe we could hide them all in implementation.

opencv_contrib is meant to be a place for experimental modules, so we don't keep source-level compatibility and don't enforce strict rules to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants