The implementation of particle filtering tracker #2
Conversation
|
@lenlen, will you accept this pull request? You're the author of the module, thus you should decide if the change is OK. |
ghost
assigned 
|
Kyrill, good to hear from you! Let me just use an opportunity to forward Below is the message: Там у меня есть серьезный технический вопрос по поводу всего репозитория Судя по описанию в README.md, можно код репозитория opencv ложить в одну Это все очень хорошо. Проблема начинается, когда мы начинаем компилировать
|
| std=(Mat_<double>(1,4)<<15.0,15.0,15.0,15.0); | ||
| } | ||
| TrackerPF::TrackerPF( const TrackerPF::Params ¶meters){ | ||
| params=parameters; |
There was a problem hiding this comment.
You should add this line in the constructor of the specialized tracker:
isInit = false;
otherwise the base class Tracker returns error
|
Formally the TrackerPF seems fitting the API, I commented one issue that is needed and maybe you should add the copyright notice in all files that you have added? |
|
Dear Antonell! First, I will change that line. By the way, I've compiled and ran my code Second, indeed, tracker is not very good -- it's rather a conceptual model Third, there's a problem with documentation, indeed. I've already told the Best, 2013/9/23 Antonella Cascitelli notifications@github.com
|
|
@nailbiter where is video that you refered? |
|
I read your code and I have some questions that I'd like discuss with you.
So it could be interesting create a TrackerStateEstimator with PF solver and a TrackerFeature with histogram in order to create in the future a tracker (similar to [1]) that uses these two components and eventually one of the sampling strategies already implemented. [1] J. Kwon and K. M. Lee, "Tracking of a non-rigid object via patch-based dynamic appearance modeling and adaptive Basin hopping Monte Carlo sampling," in Proc. Comput. Soc. Conf. Comput. Vis. Pattern Recognit. |
|
I've got the point. Will You be ready to discuss it, say, on Thursday? I I realize that Your GSoC is over, so that's rather an anchor for You, but By the way, about the documentation not generating. They are aware of this Regards, 2013/9/24 Antonella Cascitelli notifications@github.com
|
|
Ok perfect! |
|
Sorry for the late reply, I was too busy with other activities. Happy to see that you already know about the workaround for the documentation build. You can actually create a symbolic link, this way you will not need to copy docs every time you update them. And I want to add that the doc build procedure will be changed, so you don't need to copy docs manually. I don't know when it will be ready, but hopefully we'll have it early in October. |
|
Sorry for long delay. Now I'm ready to finalize the issue. I will try to 2013/9/24 Antonella Cascitelli notifications@github.com
|
|
@lenlen I've tried to put a solver component into TrackerStateEstimator and 2013/10/1 Alex Leontiev alozz1991@gmail.com
|
|
@vrabaud what do you think on the particle filter component in the API? |
|
@nailbiter in my opinion the TrackerPF doesn't fit the model of the author of the paper and consequently the implementation of the API, but my idea is that particle filter may be very useful in the tracking API, but I think it is better generalize the class in order to use it in several declinations: sampling, decisor or motion estimator (as Kalman), etc. So I suggest to put it in video where is the Kalman filter or in ML module. So anyone can use the component to develop a new tracker using particle filter. |
| for(int j=0;j<img.cols;j++){ | ||
| Vec3f pt=hsv.at<Vec3f>(i,j); | ||
|
|
||
| /*dprintf(("%d %d\n",i,j)); |
|
The current PF API does not seem generic enough to be included in ML or video. I'd like to be able to define my own particles, somehow (probably a vector of any size). That could be interesting for an implementation of the Bramble tracker which uses generalized cylinders. |
|
Dear Vincent! @vincent. Sorry for such a slow reply. I've fixed the issues You've raised, I guess. Now, about the API not being too generic. I developed it using my optim After all, I think that almost any sort of generalization can be done Best Regards, 2013/10/17 Vincent Rabaud notifications@github.com
|
| void PFSolver::getOptParam(OutputArray params)const{ | ||
| params.create(1,_std.rows,CV_64FC1); | ||
| Mat mat(1,_std.rows,CV_64FC1); | ||
| #ifdef WEIGHTED |
There was a problem hiding this comment.
shouldn't that be an option instead of an ifdef ?
|
@nailbiter don't apologize, this is great work ! Otherwise, generalizing your code to any kind of particle should be easy. I just need to look closer at the API to see if there could be something flexible enough. The cues should also be an input right ? You don't want to do the HSV conversion inside and you might have several clues (in Bramble, you also use DoG). Orwe leave your code as is but maybe it should just be renamed rectangularPF. I just want to make sure that we don't block ourselves with a more generic PF technique that we could implement (I coded Bramble many moons ago, I need time to see if it would fit with yours). |
|
The tracking API now works only with the rectangular region because all papers that I read use the rectangular shape. But I think that adding a vector of masks in the sampling component, could enabled the tracking for any shape. About "rectangularPF" I think that the tracker is too monolitic, it doesn't use the TrackingSampler and TrackingFeature components and the histograms are coupled in the solver. |
|
Dear Antonella! Thank You for critical review! To begin with, tracking non-rectangular objects... In my defense, I might Next, the coupledness of my code. We've been already talking about this. Finally, I'm not the huge fan of excessive decoupling. I mean, if one day Summarizing, I don't think further decoupling is possible. If You see the Sorry for being a bit rude maybe. I really respect Your skills as software Best Regards, 2013/10/22 Antonella Cascitelli notifications@github.com
|
|
@nailbiter I have a question: can your algorithm return N rectangles rather that one rectangle? I mean the N rectangles that made a good matching with the histogram of the region of the bounding box given in input. |
|
@lenlen Yes, in principle. However, I don't see how API can handle this, as |
|
I think that your algorithm can become a good sampling component, instead that a tracker with poor performance. |
fixes fixed compile error and warning Remove AGAST (for merger to upstream opencv) wrapping remaining xfeature2d classes to scripting added test for http://code.opencv.org/issues/3943; replaced "const InputArray" with "InputArray" to avoid warnings about "const const _InputArray&" added test for http://code.opencv.org/issues/3943; replaced "const InputArray" with "InputArray" to avoid warnings about "const const _InputArray&" adding LATCH fixed warnings in LATCH fixed errors fixed warnings fixed warnings2 fixed warnings3 fixed warnings4 fixed warnings5 added description of LATCH and fixed indentation cleaned the code a bit added tests and renamed LATCH Code to grab the red lined polygon from Google Maps added python support and completed documentation figure update added stdout welcome message and fixed warning fixed warning text enhancement trying to fix python wrapper warning on win64 trying to solve pyhton warnings bugfix4269 included remarks in http://answers.opencv.org/question/59293/problem-with-example-motemplcpp/ Replace tab with four space Add rotation invariance option for BRIEF descriptor. Fix docs and repush for buildbot. Bug fix for feature extraction According to CartToPolar() function documentation, result angles could be in range (0..360). To prevent index overflow this check is important. Adding edge-aware disparity filtering Added basic interface and demo for disparity filtering, added unoptimized fast weighted least squares filter implementation. Current demo tests domain transform, guided and weighted least squares filters on a dataset, measures speed and quality. Fix for Bug 4074. This seems to be just a typo-error, because the Tesseract API can handle correctly with RGB images (double-checked and it works). Fix for Bug opencv#3633: do away with "quads [2][3] = 255;" The four lowest bits in each quads[i][j] correspond to the 2x2 binary patterns Q_1, Q_2, Q_3 in the NM paper [1] (see in page 4 at the end of first column). Q_1 and Q_2 have four patterns, while Q_3 has only two. added INRIA pedestrian dataset autowbGrayworld: include+src+test+testdata+sample Add saturation based thresholding to grayworld WB Add basic perf tests for grayworld Add more doxygen comments Suppress uchar conv related warning on Windows Apply fixes suggested by Vadim Be more correct with int types Remove dangling N_good++ Use cvRound to suppress Windows warnings remove floor call vs2010 does not know, ceil, floor, round and friends. also, those are plain integer divisions, that do not need floor at all. New stereo module created and added some relevant files for this module Update README.md made some extra changes to the modules so I receive no warnings moved the opencv2/core/private.hpp from stereo_binary_sgbm.cpp to precomp.hpp fix for issue opencv#195 avoid overflow in histogram access SurfaceMatching: OpenMP indices Fixes compiler error: "index variable in OpenMP 'for' statement must have signed integral type" Adding confidence support and optimizing disparity filtering DisparityWLSFilter demonstrated the best results, so I removed all the other filters. Quality was significantly improved by adding confidence support to the filter (left-right consistency + penalty for areas near depth discontinuities). Filter was optimized using parallel_for_ and HAL intrinsics. Demo application was rewritten for better compliance with OpenCV standards. Added accuracy and performance tests. Documentation was added, as well as references to the original papers. added PASCAL VOC dataset + add KCF Tracker, initial commit, added: tutorial, trackerKCF.cpp, modified: tracker.cpp, tracker.hpp adding the resize feature References for KCF tracker and KCF-CN tracker Unified the formatting Fixed: ROI extraction when the given ROI is out of image; made the max_patch_size to be adjustable; add the CN feature extraction method Removing all shadowing variables, make functions to const, make the table of color-names become static change the color-names table to const Add a framework for choosing the descriptor Added error message for descriptor other than GRAY Added new line at end of file Fixing the ColorNames table initialization Fixed warning: conversion from double to int Updated the support for color-names features and fixing some typos Fixing the tabulation Split the training coefficient into numerator and denumerator Added the feature compression method Fixing some indentations Fixing some indentations Fixing alignments Fixing some alignments Use Doxigen format Remove whitespaces Removing whitespaces in featureColorName.cpp Add an example code for the KCF tracker update the header in example/kcf.cpp Updating the rectangle drawing, avoid warning from variable conversion Added doxygen documentations Fixing warnings remove warnings Fixing some warnings TLD Fixes & Optimizations 1. TLD now have module structure 2. Made some small code optimizations 3. Fixed Ensemble Classifier according to the original paper - 10 randomized ferns 4. Added comments to most of the functions and methods Added test on TLD Dataset Added BSD-compatible license Added BSD-compatible license to some files Fixed header Fixed build error Fix Fix opencv#2 Fix opencv#3 Fix opencv#4 Fixed Warnings opencv#1 Fixed Warnings opencv#2 Fixed Warnings opencv#3 Shadow Fix Fixing whitespaces Fixing whitespaces opencv#2 Fixing whitespaces opencv#3 Adds a first implementation of the OCRBeamSearchDecoder class using the Single Layer CNN character classifier described in Coates, Adam, et al. paper: Text detection and character recognition in scene images with unsupervised feature learning, ICDAR 2011 Add a demo program for the OCRBeamSearchDecoder class and needed data files trailing whitespaces fix compilation warnings fix win64 compilation error: arrays must be defined with compile-time fixed size :) fix doxygen warnings Fix for opencv#278 - core dump in the case of no match results. Modified reported poses by constraining to the number of poses found. ulong -> size_t fixed warnings in the tracking module Added OCL versions of Sr and Sc functions 2-nd level of parallelization + detector remake 1. Added 2-nd level of parallelization of NN on OpenCL 2. Restructured detector - now all filters work independently: Variance Filter->Ensemble->NN, through "buffers" Warnings Fix opencv#1 Fixing Warnings opencv#2 Fixing Warnings opencv#3 Fixing Warnings opencv#4 Fixing Warnings opencv#5 Fixing Warnings opencv#3 Fixing Warnings opencv#4 Fixing Warnings opencv#5 Added OCL version of "integrateAdditional" function Whitespace Fix Transparent API Support Fixing Warnings Fixed bug in LSDDetector where mask doesn't remove all undesired lines Fixing GCC 4.9 warning Fix memory leak bug #4420 Fix bug #4373: Error (Assertion failed in resize) when passing very elongated contours to the recognition module make sources compile again on MSVC 2012 (VC 11) by adding round() fix suffix that was in-compatible with MSVC 2012 (VC 11) eliminate some warnings use better condition for checking if compiler supports round() fixing facerecognizer tutorials and interface Adds createOCRHMMTransitionsTable() utility function to create a tailored language model transitions table from a given list of words (lexicon) update to use the new createOCRHMMTransitionsTable() function, and fix program description in header comments fix Winx64 warnings Better CNN model for character recognition. Trained with an augmented dataset by adding translation/scale variations. Updated the croped word recognition with new class numbering (compatible with previous NM classifier). Overload the run() method in BaseOCR class in order to adapt to different classifier callbacks. The original run() method accepts only one Mat input image, this is expected to be a binarzed image with black and white text and works both with the OCRTesseract class and the OCRHMMDecoder class when the character classifier callback works with binary images (e.g. NM). The new run() method accepts two Mat input parameters. One for the gray scale (or color) source image and the other for a binary mask where each connected component corresponds to a pre-segmented character in the input image. This way the OCRHMMDecoder is able to work with character classifiers that operate in grey scale (or color) images (e.g. a CNN). Adds example on segmented word recognition. Shows the use of the OCRHMMDecoder with the NM and CNN default classifiers. Minor bugfix: removes unwanted space character at the begining of recognition output strings. Fix w64 warnings Fix w64 warnings Improving DisparityWLSFilter interface and adding a tutorial Now the filter natively supports StereoBM and StereoSGBM with no parameter tuning required. Also, now user won't need to set the ROI and the right matcher parameters manually, it is all done in the respective convenience factory method based on the left matcher instance. Tutorial was added to clarify the provided example of use. doc update
Wrap the ROISelector class
Tld fixes & optimizations
More directory restructuring, getting CUDA through opencv rather than…
a few fixes to compile julia bindings on mac
This contribution aims to implement the tracker based on particle filtering within a generic tracking API that opencv has. The tracker still has to be worked on, but it has potential. This work is supported by Google within the Google Summer of Code 2013 initiative. Mentor: Vadim Pisarevsky.
Besides, the implementation, I've also added a few minor changes to the docs. First, upon agreement from the previous author, I've added some cosmetic fix to existing documentation. Second, I've changed the top README.md file, for it does not correctly display the sequence of steps needed to get things built.
It still remains to write the documentation, I will do so during the acceptance of pull request.