Feature: Offline Tracker (KSP Tracker)

[Ashp116]

Description

This PR introduces a new offline Multiple Object Tracker (MOT) based on the K-Shortest Paths (KSP) optimization algorithm. This tracker is based on the method proposed in the paper: "Multiple Object Tracking using K-Shortest Paths Optimization" by Jérôme Berclaz et al. The tracker models detections across frames as a directed acyclic graph and solves for globally optimal object trajectories by computing disjoint shortest paths.

The inspiration for this PR: #59

List any dependencies that are required for this change.

Yes, this PR needs networkx to build a directed acyclic graph.

Type of change

• New feature (non-breaking change which adds functionality)
• [➖] This change requires a documentation update: maybe? I'm not sure

How has this change been tested? Please provide a testcase or example of how you tested the change?

This PR can be tested using the Colab notebook provided here. The notebook can be used to test KSP Tracker. Currently, the notebook is using one of the MOT17 Challenge Datasets, specifically MOT17-02-FRCNN.

Any specific deployment considerations

N/A

Docs

• [❌] Docs updated? What were the changes? Not yet

[CLAassistant]

All committers have signed the CLA.

[soumik12345]

Hi @Ashp116, thanks for the PR!

I have mentioned some feedback regarding the accuracy of the implementation as per the paper and also regarding how the API for using a offline tracker should be like.

[Ashp116]

Hi @soumik12345,

Thank you for reviewing the PR, and also thank you for your kind words. I’m really happy this will be merged soon.

I have made the requested changes and also added documentation. Regarding the box_iou_batch function, I find it to be fast when computing IoUs for large batches of bounding boxes. However, in situations with only a few bounding boxes to compare, it takes more time than the original function. Of course, we can discuss this further in the Supervision repo, but I wanted to mention that having a dedicated IoU function optimized for single bounding box comparisons might be useful.

how it differs in implementation from the original paper.

Just curious — how in-depth should I go when explaining the implementation differences? Should I keep it high-level, or go into the full algorithm and cost differences compared to the paper? Also, should I explicitly mention this paper in the documentation when explaining the differences?

Also, I was experimenting with the tracker docs UI style. I was wondering if the new tracker docs style (showcased in this video) might be something to consider changing? It fits well with the rest of the docs (e.g., Supervision, Inference, etc.). If such a change is appropriate, should I push it to this PR or create a separate one?

Thanks again!

[soumik12345]

Hi @Ashp116, thanks for adding the documentations, they seem to be sufficient. However, I recommend that we also cite Tracklet Association Tracker and just mention that ideas from this paper served as inspiration for the modified edge cost function. As for an in-depth explanation, would you be open to writing a blog post on blog.roboflow.com on KSPTracker?

As for the docs UI style, I think that is out of scope for this PR. Feel free to open an issue and we can discuss further, I am sure @capjamesg would love discuss this.

Overall, I the PR is shaping up great; I left some minor feedbacks.

[soumik12345]

Please fix this colab link. Can you please raise a corresponding PR on roboflow/notebooks?

[soumik12345]

This should just be a list of sv.Detections, not a function.

[Ashp116]

Are we not making track call the model to inference the frame? When we send a video source path, it is assumed we will resolve for the model detections. With this in mind and the various ways to inference; a function is more versatile.

[soumik12345]

Instead of using custom IoU computation for KSPTracker, if you can come up with a way to optimize box_iou_batch, not only all the other trackers will benefit, but also the community will benefit in general 🙂

[Ashp116]

Hi @soumik12345 ,

Thank you for reviewing my PR. I’ve applied all the requested changes except for two:

1. Notebook-style code: I’d prefer to create the notebook after the full KSP tracker is finalized. This way, I can submit a dedicated PR to that repository, and once that’s merged, I’ll update this PR accordingly.

2. sv.Detections comment: Regarding your note that “this should just be a list of sv.Detections, not a function”—I’m not entirely sure how to resolve this. The current implementation is designed to use a function as a single entry point, which I believe helps encapsulate the logic cleanly. I'd appreciate your thoughts on how you'd suggest restructuring it.

Major Improvement: Disjoint Paths in KSP

I’ve made a significant improvement to how disjoint paths are handled in the K-Shortest Paths (KSP) algorithm. Previously, there were frequent tracker ID switches, especially when objects entered or exited the scene.

To address this, I implemented the approach described in the paper:

“We do this by introducing two additional nodes, υ_source and υ_sink, into our graph, which are linked to all the nodes representing positions through which objects can respectively enter or exit the area, such as doors or borders of the camera field of view.”

Following this, I introduced support for entry/exit regions—including both custom-defined regions and automatic frame-border regions. These act like "doors" through which objects can enter or leave the scene, enabling the graph to represent disjoint paths more accurately.

This change dramatically improves tracking stability. Tracker switches are now significantly reduced, and the disjoint paths are now handled at the graph-building stage, while _assign_tracker_ids_from_paths is now solely responsible for assigning tracker IDs.

I’ll attach videos showing the improvements.

Also, I'd love to write a blog post explaining how this KSP tracker works, including the recent changes. What’s the best way to go about contributing a blog post?

Here is the collab notebook

Thanks again!

Vidoes

Example: Look at the person labeled with tracker ID 18 in the old KSP and 19 in the new KSP.
You can clearly see that the new KSP has a much more stable tracker.

Old KSP

download.mp4

New KSP

download.2.mp4