motchallenge

Trackers Evaluation on MOT Challenge Data

An easy way to run Similari trackers and measure their performance on real data from Multiple Object Tracking Benchmark (MOTChallenge).

Download The Data

For the MOT20 use one of the links:

• Get files (no img) only (13.9 MB);
• Get all data (5.0 GB).

curl -o MOT20Labels.zip https://motchallenge.net/data/MOT20Labels.zip

Extract the label files to the data/ folder:

unzip MOT20Labels.zip
mkdir data
mv MOT20Labels data/MOT20

The data folder structure for MOT20 should be:

MOT20/
  test/
    MOT20-04/
    ...
  train/
    MOT20-01/
      det/
      gt/
      ...
    ...

Build The Docker Image

docker build -t similari_py_mot -f python/motchallenge/Dockerfile .

Run The Processing and Evaluation

Execute code to process default data/MOT20/train data using default Similari SORT that uses IoU metric with 0.3 threshold:

docker run --rm -v $(pwd)/data:/data similari_py_mot

Predefined config file for the canonical (python+numpy) SORT:

docker run --rm \
    -v $(pwd)/data:/data \
    -v $(pwd)/python/motchallenge/confs/original-sort-config.toml.yml:/opt/custom_config.yml \
    similari_py_mot /opt/custom_config.yml

Predefined config file for the Similari SORT that uses Mahalanobis metric:

docker run --rm \
    -v $(pwd)/data:/data \
    -v $(pwd)/python/motchallenge/confs/similari-maha-sort-config.toml.yml:/opt/custom_config.yml \
    similari_py_mot /opt/custom_config.yml

Interpreting The Results

The whole process is logged on the stdout.

The FPS processing performance is displayed for every processed file like:

Processing MOT20-01...
Read file "/data/MOT20/train/MOT20-01/det/det.txt".
429 frames collected, with an average of 29.4 detections per frame.

100%|██████████| 429/429 [00:00<00:00, 1166.01it/s]MOT20-01 processing ended after 0:00:00.368002 (1165.76 FPS).
Resulting file /data/MOT20/output/sort_iou/data/MOT20-01.txt was successfully written.

The resulting files will be saved to the specified folder (output_path in the config file).

The data/MOT20/output/<NAME>/data folder output structure:

• pedestrian_detailed.csv - detailed evaluation info;
• pedestrian_summary.txt - evaluation summary;
• processing_stats.csv - processing statistics: frames, detections, FPS, etc.