This implements the SRDCF tracker

[GustavHager]

Was the winner of the opencv challenge in tracking.

[vpisarev]

@GustavHager, thanks for the code submission!
Could you please fix the build problems reported by our buildbot:
pullrequest.opencv.org ?

Actually, as I see, Eigen is used in the module. This dependency should be removed.

[joelgallant]

Hi @vpisarev thanks for self-assigning! Do you have a time estimate to work on this, just curious when I can start testing this guy out. Thanks!

[GustavHager]

I will submit a new pull request in a few weeks without eigen, until then the matlab implementation exists at http://www.cvl.isy.liu.se/research/objrec/visualtracking/regvistrack/index.html

[y22ma]

real.size() is actually 0 in this case, since you haven't initialized the real vector. This leads to exception thrown when you try to declare a SparseMat of 0 size in line 481.

[GustavHager]

No it is not, i am working on a new pull request with less bugs and less Eigen at the moment on my own fork.

[bhack]

@vpisarev Any update on this? Can we expand on this for a GSoC? @GustavHager I've not read the poster again but is there space for OpenCL acceleration? /cc @lenlen

[GustavHager]

The main problem is that the current formulation requires sparse matrices and sparse solvers to compute the model updates in a timely manner. Hence the use of eigen in the pull request. I have implemented a variant using the OpenCV sparse matrices, but this proved significantly slower than the eigen or matlab code we prototyped with. Unfortunately i ran out of time to try to speed it up, and i am not sure if it is due to the sparse representation in opencv or the solver itself.

Since there seems to be some interest i could try to speed up the current opencv sparse implementation over the weekend, hopefully that should be enough to get acceptable performance and this pull request could be closed

More generally, most of the algorithm should be trivial to parallelize and it is something me and Martin have thought of but never had time to implement.

[bhack]

@GustavHager If you can push an updated version I'll try to put a student to profile it.

[kurnianggoro]

hello all, last year I implemented KCF and CN algorithm which are similar to SRDCF in some aspects, how about the status of this pull request?
@bhack if there still a room for improvement (or re-implementation) I would like to apply this project for the this year GSOC, what do you think?

[GustavHager]

I have more or less ironed out all the outright bugs in the implementation now, but there is still quite a bit of optimization to be done for it to be properly useful, i will submit a proper pull request on Monday with these fixes. It might be a decent summer project to optimize the implementation, or make some components such a the feature extraction more general though.

Alternatively there is a number of extra features that could be added to the current implementation.

[berkerlogoglu]

@GustavHager Have you give up for the code? I really want to try out your tracker even it is not part of opencv? any suggestions?

[kurnianggoro]

@berkerlogoglu they provide the Matlab version, check here https://www.cvl.isy.liu.se/research/objrec/visualtracking/regvistrack/

[berkerlogoglu]

@kurnianggoro thanks but I need c/c++ codes.

[Dikay900]

@berkerlogoglu I think you can use the PR as is since the problems stated by the build bot (checked linux x64 and docs) are trivial
use the main opencv_contrib branch then do:

git checkout -b SRDCFTracker
git remote add GustavHager https://github.com/GustavHager/opencv_contrib.git
git fetch GustavHager
git merge GustavHager/master

Then you have a current master with this PR included

[berkerlogoglu]

@Dikay900 Thanks! it actually built without any problems. I could also make it work. BUT!, it does not update the bounding box! :(

[OmarHamdoun]

I have the same problem, it does not update the bounding box!
TrackerSRDCF::Params params;
params.reg_min = 0.1;
params.sparsity_treshold = 0.01;
Ptr tracker = TrackerSRDCF::createTracker(params);
if( tracker == NULL )
{
cout << "_Error in the instantiation of the tracker..._\n";
return -1;
}

//get the first frame
cap >> frame;
frame.copyTo( image );
if(initBoxWasGivenInCommandLine){
selectObject=true;
paused=false;
boundingBox.x = coords[0];
boundingBox.y = coords[1];
boundingBox.width = std::abs( coords[2] - coords[0] );
boundingBox.height = std::abs( coords[3]-coords[1]);
printf("bounding box with vertices (%d,%d) and (%d,%d) was given in command line\n",coords[0],coords[1],coords[2],coords[3]);
rectangle( image, boundingBox, Scalar( 255, 0, 0 ), 2, 1 );
}
imshow( "Tracking API", image );

bool initialized = false;
int frameCounter = 0;

for ( ;; )
{
if( !paused )
{
if(initialized){
cap >> frame;
if(frame.empty()){
break;
}
frame.copyTo( image );
}

  if( !initialized && selectObject )
  {
    //initializes the tracker
    if( !tracker->init( frame, boundingBox ) )
    {
      cout << "***Could not initialize tracker...***\n";
      return -1;
    }
    initialized = true;
  }
  else if( initialized )
  {
    //updates the tracker
    if( tracker->update( frame, boundingBox ) )
    {
      rectangle( image, boundingBox, Scalar( 255, 0, 0 ), 2, 1 );
    }
  }
  imshow( "Tracking API", image );
  frameCounter++;
}

char c = (char) waitKey( 2 );
if( c == 'q' )
  break;
if( c == 'p' )
  paused = !paused;

}

return 0;
}

[imageshow]

I also encountered the same problem. It does not update the bounding box, and box flies to the upper left corner. Does anyone solve it?

[berak]

i just tried it, the boundingbox indeed never moves. tracked it down to the filterf mat returned from assemble_filters being all -Inf

[3fffff]

Error in function SRDCF::compute_filter
`cv::Mat SRDCF::compute_filter(const cv::SparseMat& lhs_data, const cv::SparseMat& reg_matrix, const cv::Mat& rhs) {
//TODO implement correct error function
//TODO include proper initial guess from previous optimization
cv::Mat initial_guess = cv::Mat::eye(rhs.rows, 1, CV_32FC1);
cv::Mat current_guess;
int iterations = 0;
int i_max = 100;
cv::Mat residual = rhs - sparse_mat_vector_product(lhs_data, initial_guess);
cv::Mat delta_matrix = rhs * rhs.t();

	float delta = delta_matrix.at<float>(0, 0);
	float delta_0 = delta;
	float error_term_crit = delta_0*pow(std::numeric_limits<float>::epsilon(), 2);

	while (iterations < i_max && delta > error_term_crit) {
		cv::Mat q = sparse_mat_vector_product(lhs_data, residual);
		**cv::Mat qr_mat = q.t() * residual;
		float alpha = delta / qr_mat.at<float>(0, 0);**
		current_guess = current_guess + alpha * residual;
		residual = residual - alpha * q;
		delta_matrix = residual.t() * residual;
		delta = delta_matrix.at<float>(0, 0);

		iterations += 1;
	}
	return current_guess;
}`