Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

TubeTK

TubeTK is an one-step end-to-end multi-object tracking method, which is the first end-to-end open-source system that achieves 60+ MOTA on MOT-16 (64 MOTA) and MOT-17 (63 MOTA) datasets. Our paper "TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model" is accepted as an oral paper on CVPR-2020.

Contents

Results

Demo Video

MOT-16

Results on MOT-16 dataset:

Video MOTA IDF1 MT ML FP FN IDS
MOT16-01 48.9 45.5 8 9 175 3052 40
MOT16-03 76.3 69.5 86 12 3741 20828 177
MOT16-06 51.2 55.7 87 39 1863 3542 231
MOT16-07 55.0 43.5 21 3 2225 4938 190
MOT16-08 46.9 37.3 18 3 1694 6952 234
MOT16-12 52.4 50.8 27 20 533 3366 51
MOT16-14 35.8 39.8 7 61 731 10948 194
TubeTK (Mean) 64.0 59.4 33.5 19.4 10962 53626 1117
RAN 63.0 63.8 39.9 22.1 13663 53248 482
Tracktor 54.5 52.5 19.0 36.9 3280 79149 682

MOT-17

Results on MOT-17 dataset:

Video MOTA IDF1 MT ML FP FN IDS
MOT17-01 47.9 44.9 6 10 167 3154 41
MOT17-03 76.4 69.6 81 12 3181 21287 186
MOT17-06 52.4 54.8 85 36 1609 3699 307
MOT17-07 55.4 43.3 21 2 1944 5371 222
MOT17-08 42.3 34.1 18 12 970 10889 319
MOT17-12 50.3 49.4 28 23 494 3749 63
MOT17-14 35.6 39.5 6 61 655 11012 241
TubeTK (Mean) 63.0 58.6 31.2 19.9 27060 177483 4137
SCNet 60.0 54.4 34.4 16.2 72230 145851 7611
Tracktor 53.5 52.3 19.5 36.3 12201 248047 2072

Installation

  1. Get the code and build related modules:

      git clone ...(TO BE CONFIRM)
      cd TubeTK/install
      ./compile.sh
      # if something wrong, try:
      # sudo ldconfig <path/to/cuda>/lib64
      cd ..
  2. Install pytorch 1.10 and other dependencies:

pip install -r requirements.txt
  1. If the memory of your GPU < 16G, then you need NVIDIA APEX to conduct the mixed precision training.

    1. Install Apex:
    git clone https://github.com/NVIDIA/apex
    cd apex
    pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
    # if something wrong with the above pip install, try:
    # pip install -v --no-cache-dir ./
    1. We provide the --apex option to train with the APEX, see Quick Start for detail.
  2. Run fetch_model.sh to download our pre-trained models. Or download the models manually and put them in ./models:

    1. 3DResNet50_original (Baidu pan | Google drive)

Quick Start

Demo

Run TubeTK for a video and visualization the results with:

python launch.py --nproc_per <num of GPU> --training_script demo.py --batch_size=3 --config configs/TubeTK_resnet_50_FPN_8frame_1stride.yaml --video_url <folder/to/the/videos> --output_dir ./vis_video

Evaluation on MOT-17 (16)

  1. Download the data from MOT Challenge, and put or link it to ./data

  2. To get the tracking result with:

    python launch.py --nproc_per <num of GPU> --training_script evaluate.py --batch_size 3 --config configs/TubeTK_resnet_50_FPN_8frame_1stride.yaml --trainOrTest test
  3. To get the visualization with:

    python Visualization/Vis_Res.py --mode test

    The visualization videos are stored in ./vis_video .

Train on MOT-17 (16)

  1. Download the data from MOT Challenge, and put or link it to ./data

  2. Get the ground truth Btubes with:

    python ./pre_processing/get_tubes_MOT17.py
  3. Train the model with:

    python launch.py --nproc_per <num of GPU> --training_script main.py --batch_size 1 --config ./configs/TubeTK_resnet_50_FPN_8frame_1stride.yaml

    If out of memory, try:

    python launch.py --nproc_per <num of GPU> --training_script main.py --batch_size 1 --config ./configs/TubeTK_resnet_50_FPN_8frame_1stride.yaml --apex

    If still out of memory, modify the configuration file: TubeTK_resnet_50_FPN_8frame_1stride.yaml:

    tube_limit: 500  # or 300
    

Citation

@inproceedings{pang2020tubetk,
   title={TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model},
   author={Pang, Bo and Li, Yizhuo and Zhang, Yifan and Li, Muchen and Lu, Cewu},
   booktitle={CVPR},
   year={2020}
}

License

TubeTK is freely available for free non-commercial use, and may be redistributed under these conditions.

About

Official implementation of paper: TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model (CVPR 2020 oral)

Resources

License

Releases

No releases published

Packages

No packages published