OpenTracker is a open sourced repository for Visual Tracking. It's written in C++, high speed, easy to use, and easy to be implemented in embedded system.
- AND this is not only boring Codes,
+ It also has Maths and Implement Notes!If you don't exactly know what this means:
Don't worry, it will be explained fully in the Notes. All the maths details of the Not-that-easy algorithms are explaned fully from the very beginning. If you have headache of reading the papers(as most of us have), this is a good tutorial. (Check Notes(draft now)).
Or, if you have problems with the implementation of a complicate cutting-edge algorithms, check this! You will get something!
Attention! OpenTracker is NOT designed just for tracking human beings as the demo images, it can track everything, even some special points!
2018/11/06 -- New features add cmake compile support for eco
2018/09/19 -- New features Performance tested on VOT2017 dataset!
2018/09/13 -- New features CN feature added!
2018/08/30 -- New features Support Initialize by Object Detection using Darknet and track.
2018/08/27 -- New features Support ECO API.
2018/08/24 -- New features Now ECO runs "almost" real-time on Raspberry Pi 3!
2018/08/24 -- New features Support FFTW.
2018/08/13 -- New features Speed up by multi-thread.
2018/08/09 -- New features Now it supports Raspberry Pi 3, and speed up with NEON!
2018/08/08 -- New features Speed up with NEON, speed up from ~32FPS to ~42FPS on Jetson TX2 with scale one.
2018/08/06 -- New features Speed up with SSE, speed up from ~86FPS to ~102FPS(quicker than matlab version) with scale one.
2018/07/07 -- New features OpenTracker Implement Notes draft published! Check notes/OpenTrackerNotes.pdf. Complete version is comming!
2018/07/06 -- New features Now it supports Nvidia Jetson TX1/2!
2018/07/05 -- New features Now it supports macOS!
2018/06/28 -- New features Now it supports automatic initialization with Web camera using OpenPose!
| Included | Tracker |
|---|---|
| ☑️ | CSK |
| ☑️ | KCF |
| ☑️ | DSST |
| ☑️ | GOTURN |
| 🔨 | ECO |
| Included | Dataset | Reference |
|---|---|---|
| ☑️ | VOT-2017 | Web |
| ☑️ | TB-2015 | Web |
| ☑️ | TLP | Web |
| ☑️ | UAV123 | Web |
| Included | Dataset | Reference |
|---|---|---|
| ☑️ | OpenPose | Web |
| Included | OS / Platform |
|---|---|
| ☑️ | Ubuntu 16.04 |
| ☑️ | macOS Sierra |
| ☑️ | NVIDIA Jetson TX1/2 |
| ☑️ | Rasperberry PI 3 |
| 🔨 | Windows10 |
"ECOHCMATLAB" is the original matlab full version ECO-HC.
"ECOHCMATLABHOGCN" is the matlab version ECO-HC without fDSST scale filter.
"ECOHCMATLABHOG" is the matlab version ECO-HC without fDSST scale filter and CN feature.
"ECOCPPHOGCN" is the c++ ECO tracker in OpenTracker without fDSST scale filter.
"ECOCPPHOG" is the c++ ECO tracker in OpenTracker without CN feature and fDSST scale filter.
"KCFCPP" is the c++ KCF tracker in OpenTracker.
"NCC" is a demo tracker in vot-toolkit.
The test is on dataset VOT2017, and parameters are set exactly the same as "VOT2016_HC_settings" in matlab version. This is just for proof of validation of c++ version code, thus the parameters are not tuned for VOT2017.
You can see from the plot that, full-featured "ECOHCMATLAB" has the highest performance, "ECOCPPHOGCN" has almost the same performance with "ECOHCMATLABHOGCN", and "ECOCPPHOG" quite similar to "ECOHCMATLABHOG". And "KCFCPP" perform even better than the HOG-only ECO version, so it seems that CN feature matters.
| Included | Method(single thread) | FPS(scale=1) | FPS(scale=7) |
|---|---|---|---|
| ☑️ | Matlab ECO-HOG(Intel i9) | ~73 | ~45 |
| ☑️ | no speed-up(Intel i9) | ~86 | ~36 |
| ☑️ | SSE(Intel i9) | ~260:cherries: | ~95:cherries: |
| ☑️ | no speed-up(MacBook Air Intel i5) | ~60 | ~22 |
| ☑️ | SSE(MacBook Air Intel i5) | ~140:cherries: | ~55:cherries: |
| ☑️ | no speed-up(Jestson TX2) | ~32 | ~10 |
| ☑️ | NEON(Jetson TX2) | ~60:cherries: | ~34:cherries: |
| ☑️ | no speed-up(Raspberrypi) | ~11 | ~3 |
| ☑️ | NEON(Raspberrypi) | ~24:cherries: | ~7.5 |
| 🔨 | GPU | 🔨 | 🔨 |
With quick start, you can have a quick first taste of this repository, without any panic. No need to install Caffe, CUDA etc. (But of course you have to install OpenCV 3.0 first).
OpenCV 3.0 Install on Ubuntu check this [Tutorial].
In eco/runecotracker.cc, make sure to choose the dataset Demo:
string databaseType = databaseTypes[0];
git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/eco
make -j`nproc`
sudo make install
./runecotracker.bin
brew install tesseract
git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/eco
make -j`nproc`
sudo make install
./runecotracker.bin
In file kcf/runkcftracker.cc, make sure to choose the dataset Demo:
string databaseType = databaseTypes[0];
git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/kcf
make
./runkcftracker.bin
brew install tesseract
git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/kcf
make
./runkcftracker.bin
git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker
make
sudo make install
./trackerscompare.bin
For the environment settings and detailed procedures (with all the packages from the very beginning), refer to: [My DeeplearningSettings].
The only extra-package is: Opencv3.x (already installed if you follow the environment settings above).
Of course, for trackers that use Deep features, you need to install [caffe] (maybe I will use Darknet with C in the future, I like Darknet 👄 ), and change the makefile according to your path. Compile of caffe refer to : [Install caffe by makefile].
If you want to autodetection the people with web camera, you need to install [OpenPose].
If you want to use Openpose, in ./makefile, set OPENPOSE=1, else set OPENPOSE=0.
Change the datasets, in inputs/readdatasets.hpp, change the number of string databaseType = databaseTypes[1];
Change the path of datasets, in inputs/readdatasets.cc, change the path to your path of data.
By raising your two arms higher than your nose, it will atomatically detect the person and start the tracking programme.
make all
sudo make install
./trackerscompare.bin
If you don't want to compile with Caffe, that means you cannot use Deep features, set in eco/makefile: USE_CAFFE=0.
If you don't want to compile with CUDA, that means you cannot use Deep features, set in eco/makefile: USE_CUDA=0.
If you want to compile with Caffe, set in makefile and eco/makefile: USE_CAFFE=1 USE_CUDA=1, and set the according caffe path of your system in eco/makefile:
CAFFE_PATH=<YOUR_CAFFE_PATH>
Download a pretrained [VGG_CNN_M_2048.caffemodel (370 MB)], put it into folder: eco/model
If you could not download through the link above (especially for the people from Mainland China), check this [link] and download.
In eco/parameters.hpp, change the path to your path:
struct CnnParameters
{
string proto = "<YOUR_PATH>/OpenTracker/eco/model/imagenet-vgg-m-2048.prototxt";
string model = "<YOUR_PATH>/OpenTracker/eco/model/VGG_CNN_M_2048.caffemodel";
string mean_file = "<YOUR_PATH>/OpenTracker/eco/model/VGG_mean.binaryproto";
In eco/runecotracker.cc, change the path:
parameters.useCnFeature = true;
parameters.cn_features.fparams.tablename = "<YOUR_PATH>/OpenTracker/eco/look_tables/CNnorm.txt"
If you are using Intel computer, in eco\makefile, set:
USE_SIMD=1
If you are using ARM like Jetson TX1/2, in eco\makefile, set:
USE_SIMD=2
If you are using ARM like Rasberrypi 3, in eco\makefile, set:
USE_SIMD=3
In eco\makefile, set:
USE_MULTI_THREAD=1
If you have a GPU, it can speed-up with gpu.
First don't forget to install Opencv with CUDA supported:
cmake -D OPENCV_EXTRA_MODULE_PATH=/media/elab/sdd/Amy/opencv_contrib/modules \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_CUDA=ON \
-D ENABLE_FAST_MATH=1 \
-D CUDA_FAST_MATH=1 \
-D WITH_CUBLAS=1 \
..
make -j`nproc`
sudo make install
in eco/makefile, set:
USE_CUDA=1
Change the path of your test images in eco/runecotracker.cc.
Change the datasets, in eco/runecotracker.cc, change the number of string databaseType = databaseTypes[1];.
If you want to show the heatmap of the tracking, in eco/parameters.cc, change to #define DEBUG 1.
cd eco
make -j`nproc`
./runecotracker.bin
Change the path of your test images in kcf/opencvtrackers.cc.
cd opencvtrackers
make
./opencvtrackers.bin
Change the path of your test images in kcf/runkcftracker.cc.
cd kcf
make -j`nproc`
./runkcftracker.bin
Change the path of your test images in goturn/rungoturntracker.cc.
You can download a pretrained [goturun_tracker.caffemodel (434 MB)], put it into folder: goturn/nets
cd goturn
make -j`nproc`
./rungoturntracker.bin
./classification.bin /media/elab/sdd/caffe/models/bvlc_reference_caffenet/deploy.prototxt /media/elab/sdd/caffe/models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel /media/elab/sdd/caffe/data/ilsvrc12/imagenet_mean.binaryproto /media/elab/sdd/caffe/data/ilsvrc12/synset_words.txt /media/elab/sdd/caffe/examples/images/cat.jpg
ATTENTION! Make sure that the parameter settings in makefile and eco/makefile are the same, else it will be errors!
To use the API of the trackers is really simple, just two steps. Check example/readme.md.
(not complete, tell me if I forgot you)
Learning to Track at 100 FPS with Deep Regression Networks,
David Held,
Sebastian Thrun,
Silvio Savarese,
European Conference on Computer Vision (ECCV), 2016 (In press)
J. F. Henriques, R. Caseiro, P. Martins, J. Batista,
"High-Speed Tracking with Kernelized Correlation Filters", TPAMI 2015.
J. F. Henriques, R. Caseiro, P. Martins, J. Batista,
"Exploiting the Circulant Structure of Tracking-by-detection with Kernels", ECCV 2012.
Martin Danelljan, Goutam Bhat, Fahad Khan, Michael Felsberg.
ECO: Efficient Convolution Operators for Tracking.
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Martin Danelljan, Andreas Robinson, Fahad Khan, Michael Felsberg.
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking.
In Proceedings of the European Conference on Computer Vision (ECCV), 2016.
http://www.cvl.isy.liu.se/research/objrec/visualtracking/conttrack/index.html
Martin Danelljan, Gustav Häger, Fahad Khan, Michael Felsberg.
Learning Spatially Regularized Correlation Filters for Visual Tracking.
In Proceedings of the International Conference in Computer Vision (ICCV), 2015.
http://www.cvl.isy.liu.se/research/objrec/visualtracking/regvistrack/index.html
Martin Danelljan, Gustav Häger, Fahad Khan, Michael Felsberg.
Convolutional Features for Correlation Filter Based Visual Tracking.
ICCV workshop on the Visual Object Tracking (VOT) Challenge, 2015.
http://www.cvl.isy.liu.se/research/objrec/visualtracking/regvistrack/index.html
Martin Danelljan, Gustav Häger, Fahad Khan and Michael Felsberg.
Accurate Scale Estimation for Robust Visual Tracking.
In Proceedings of the British Machine Vision Conference (BMVC), 2014.
http://www.cvl.isy.liu.se/research/objrec/visualtracking/scalvistrack/index.html
Martin Danelljan, Gustav Häger, Fahad Khan, Michael Felsberg.
Discriminative Scale Space Tracking.
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2017.
http://www.cvl.isy.liu.se/research/objrec/visualtracking/scalvistrack/index.html
N. Dalal and B. Triggs.
Histograms of oriented gradients for human detection.
In CVPR, 2005.
J. van de Weijer, C. Schmid, J. J. Verbeek, and D. Larlus.
Learning color names for real-world applications.
TIP, 18(7):1512–1524, 2009.
Y. Wu, J. Lim, and M.-H. Yang.
Online object tracking: A benchmark.
TPAMI 37(9), 1834-1848 (2015).
https://sites.google.com/site/trackerbenchmark/benchmarks/v10
Y. Wu, J. Lim, and M.-H. Yang.
Object tracking benchmark.
In CVPR, 2013.
KCF: joaofaro/KCFcpp.
DSST: liliumao/KCF-DSST, the max_scale_factor and min_scale_factor is set to 10 and 0.1 in case of divergence error (Tested on UAV123 dataset when the object is quite small, ex.uav2/3/4...).
GOTURN: davheld/GOTURN.
ECO: martin-danelljan/ECO.