We introduce the process of getting started on OC-SORT. This instruction is adapted from ByteTrack especially for the training part. We provide some simple pieces here, for details please refer to the source code and utils/args.py.
-
Download MOT17, MOT20, CrowdHuman, Cityperson, ETHZ, DanceTrack and put them under <OCSORT_HOME>/datasets in the following structure:
datasets |——————mot | └——————train | └——————test └——————crowdhuman | └——————Crowdhuman_train | └——————Crowdhuman_val | └——————annotation_train.odgt | └——————annotation_val.odgt └——————MOT20 | └——————train | └——————test └——————Cityscapes | └——————images | └——————labels_with_ids └——————ETHZ | └——————eth01 | └——————... | └——————eth07 └——————dancetrack └——————train └——————val └——————test -
Turn the datasets to COCO format and mix different training data:
# replace "dance" with ethz/mot17/mot20/crowdhuman/cityperson for others python3 tools/convert_dance_to_coco.py
-
[Optional] If you want to training for MOT17/MOT20, follow the following to create mixed training set.
# build mixed training sets for MOT17 and MOT20 python3 tools/mix_data_{ablation/mot17/mot20}.py
You can use OC-SORT without training by adopting existing detectors. But we borrow the training guidelines from ByteTrack in case you want work on your own detector.
Download the COCO-pretrained YOLOX weight here and put it under <OCSORT_HOME>/pretrained.
-
Train ablation model (MOT17 half train and CrowdHuman)
python3 tools/train.py -f exps/example/mot/yolox_x_ablation.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
-
Train MOT17 test model (MOT17 train, CrowdHuman, Cityperson and ETHZ)
python3 tools/train.py -f exps/example/mot/yolox_x_mix_det.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
-
Train MOT20 test model (MOT20 train, CrowdHuman)
For MOT20, you need to uncomment some code lines to add box clipping: [1],[2],[3] and [4]. Then run the command:
python3 tools/train.py -f exps/example/mot/yolox_x_mix_mot20_ch.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
-
Train on DanceTrack train set
python3 tools/train.py -f exps/example/dancetrack/yolox_x.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
-
Train custom dataset
First, you need to prepare your dataset in COCO format. You can refer to MOT-to-COCO or CrowdHuman-to-COCO. Then, you need to create a Exp file for your dataset. You can refer to the CrowdHuman training Exp file. Don't forget to modify get_data_loader() and get_eval_loader in your Exp file. Finally, you can train bytetrack on your dataset by running:
python3 tools/train.py -f exps/example/mot/your_exp_file.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth
-
on DanceTrack Val set
python tools/run_ocsort_dance.py -f exps/example/mot/yolox_dancetrack_val.py -c pretrained/bytetrack_dance_model.pth.tar -b 1 -d 1 --fp16 --fuse --expn $exp_nameWe follow the TrackEval protocol for evaluation on the officially released validation set. This gives HOTA = 52.1 ~ 52.6.
-
on DanceTrack Test set
python tools/run_ocsort_dance.py -f exps/example/mot/yolox_dancetrack_test.py -c pretrained/bytetrack_dance_model.pth.tar -b 1 -d 1 --fp16 --fuse --test --expn $exp_nameSubmit the outputs to the DanceTrack evaluation site. This gives HOTA = 54.6 ~ 55.2.
-
on MOT17 half val
python3 tools/run_ocsort.py -f exps/example/mot/yolox_x_ablation.py -c pretrained/bytetrack_ablation.pth.tar -b 1 -d 1 --fp16 --fuse --expn $exp_nameWe follow the TrackEval protocol for evaluation on the self-splitted validation set. This gives you HOTA = 66.5.
-
on MOT17/MOT20 Test set
# MOT17 python3 tools/run_ocsort.py -f exps/example/mot/yolox_x_mix_det.py -c pretrained/bytetrack_x_mot17.pth.tar -b 1 -d 1 --fp16 --fuse --expn $exp_name # MOT20 python3 tools/run_ocsort.py -f exps/example/mot/yolox_x_mix_mot20_ch.py -c pretrained/bytetrack_x_mot20.tar -b 1 -d 1 --fp16 --fuse --track_thresh 0.4 --mot20 --expn $exp_name
Submit the zipped output files to MOTChallenge system. Following the adaptive detection thresholds by ByteTrack can further boost the performance. After interpolation (see below), this gives you HOTA = ~63.1 on MOT17 and HOTA = ~61.9 on MOT20.
-
on KITTI Test set
For fair comparison of association performance only, we use the detection results from a public PermaTrack model weights as indicated here. The output of PermanceTrack is provided in exps/permatrack_kitti_test/ already. Simply run the command to use OC-SORT over KITTI-test set:
python tools/run_ocsort_public.py --hp --out_path kitti_test --dataset kitti --raw_results_path exps/permatrack_kitti_test
Then, submit the output .txt files to the KITTI evaluation server.
-
on HeadTrack21 (CroHD)
Similar to KITTI, we do not provide detector on CroHD head tracking dataset but adapt from detection results from existing results. Please download the results reported to the evaluation server and place it at datasets/headtrack/$tracker
python tools/run_ocsort_public.py --raw_results_path datasets/headtrack/$tracker --hp --dataset headtrack --out_path out_headtrack_resultsThen, submit the output files to the evaluation server above for evaluation.
Note: We find the current implementation may show some randomness in different running trials. We are still inspecting this.
OC-SORT is designed for online tracking, but offline interpolation has been demonstrated efficient for many cases. To use the linear interpolation over existing tracking results:
# optional offline post-processing
python3 tools/interpolation.py $result_path $save_pathFurthermore, we provide a piece of attempt of using Gaussian Process Regression in interpolating trajectories, which work upon existing linear interpolation results:
python3 tools/gp_interpolation.py $raw_results_path $linear_interp_path $save_pathNote: for the results in our paper on MOT17/MOT20 private settings and HeadTrack, we use linear interpolation by default.