Adaptive Aggregation of Arbitrary Online Trackers
with a Regret Bound

Adaptive Aggregation of Arbitrary Online Trackers with a Regret Bound [pdf] [poster]

Heon Song, Daiki Suehiro, and Seiichi Uchida

The IEEE Winter Conference on Applications of Computer Vision(WACV), 2020

We propose an online visual-object tracking method that is robust even in an adversarial environment, where various disturbances may occur on the target appearance, etc. The proposed method is based on a delayed-Hedge algorithm for aggregating multiple arbitrary online trackers with adaptive weights. The robustness in the tracking performance is guaranteed theoretically in term of "regret" by the property of the delayed-Hedge algorithm. Roughly speaking, the proposed method can achieve a similar tracking performance as the best one among all the trackers to be aggregated in an adversarial environment. The experimental study on various tracking tasks shows that the proposed method could achieve state-of-the-art performance by aggregating various online trackers.

Tracking examples

BlurBody	Hand	car1	motocross1	leaves

Experts

• DaSiamRPN[https://github.com/foolwood/DaSiamRPN,https://github.com/songheony/DaSiamRPN]^[1]
• ECO[https://github.com/martin-danelljan/ECO]^[2]
• MCCT[https://github.com/594422814/MCCT]^[2]
• SiamDW[https://github.com/cvpr2019/deeper_wider_siamese_trackers]
• SiamFC[https://github.com/huanglianghua/siamfc-pytorch]
• SiamMask[https://github.com/foolwood/SiamMask]
• SiamRPN[https://github.com/huanglianghua/siamrpn-pytorch]

[1] Since the original code of DaSiamRPN is for Python2, we've had to modify the code a little bit to be compatible with Python3.
[2] To run Matlab scripts in Python, we've used transplant which is much faster than official Matlab API.

Datasets

• OTB2015[http://cvlab.hanyang.ac.kr/tracker_benchmark/index.html]
• TColor128[http://www.dabi.temple.edu/~hbling/data/TColor-128/TColor-128.html]
• VOT2018[http://www.votchallenge.net/]^[3]

[3] VOT2018 is evaluated in unsupervised experiment as same as other datasets.

Frameworks

• got10k-toolkit[https://github.com/got-10k/toolkit] for tracking datasets.
• pysot-toolkit[https://github.com/StrangerZhang/pysot-toolkit] for evaluating trackers.

Requirements

conda create -n [ENV_NAME] python=[PYTHON_VERSION>=3.6]
conda install pytorch torchvision cudatoolkit=[CUDA_VERSION] -c pytorch
conda install pyzmq

How to run

git clone https://github.com/songheony/AAA-WACV
mkdir AAA-WACV/external
cd AAA-WACV/external
git clone [FRAMEWORK_GIT]
git clone [EXPERT_GIT]
conda activate [ENV_NAME]
docker run -it --name [TRACKER_NAME] --network [NETWORK_NAME] python tracker.py -e [TRACKER_NAME]
docker run -it --name server --network [NETWORK_NAME] python aggregate.py -t [TRACKERS_NAME] -d [DATASETS_NAME]

1. Clone this repository and make external directory.

2. Clone experts who you want to hire.^[4]

3. Create network over docker.

4. Run tracker as a server with docker.

5. Run our tracker a client with docker.

[4] Depending on the expert, you may need to install additional subparty libraries such as opencv.

Author

👤 Heon Song

• Github: @songheony
• Contact: songheony@gmail.com

Citation

@InProceedings{Song_2020_WACV,
    author = {Song, Heon and Suehiro, Daiki and Uchida, Seiichi},
    title = {Adaptive Aggregation of Arbitrary Online Trackers with a Regret Bound},
    booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
    month = {March},
    year = {2020}
}