Skip to content

Commit

Permalink
Updated documentation with new vectorized distances.
Browse files Browse the repository at this point in the history
  • Loading branch information
facundo-lezama committed Nov 3, 2022
1 parent 703bf5d commit 0815f94
Show file tree
Hide file tree
Showing 4 changed files with 9 additions and 9 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Using Norfair, you can add tracking capabilities to any detector with just a few

- Supports moving camera, re-identification with appearance embeddings, and n-dimensional object tracking (see [Advanced features](#advanced-features)).

- The function used to calculate the distance between tracked objects and detections is defined by the user, enabling the implementation of different tracking strategies.
- Norfair provides several predefined distance functions to compare tracked objects and detections. The distance functions can also be defined by the user, enabling the implementation of different tracking strategies.

- Fast. The only thing bounding inference speed will be the detection network feeding detections to Norfair.

Expand Down Expand Up @@ -98,9 +98,9 @@ Most tracking demos are showcased with vehicles and pedestrians, but the detecto

## How it works

Norfair works by estimating the future position of each point based on its past positions. It then tries to match these estimated positions with newly detected points provided by the detector. For this matching to occur, Norfair can rely on any distance function specified by the user of the library. Therefore, each object tracker can be made as simple or as complex as needed.
Norfair works by estimating the future position of each point based on its past positions. It then tries to match these estimated positions with newly detected points provided by the detector. For this matching to occur, Norfair can rely on any distance function. There are some predefined distances already integrated in Norfair, and the users can also define their own custom distances. Therefore, each object tracker can be made as simple or as complex as needed.

The following is an example of a particularly simple distance function calculating the Euclidean distance between tracked objects and detections. This is possibly the simplest distance function you could use in Norfair, as it uses just one single point per detection/object.
The following is an example of a particularly simple distance function calculating the Euclidean distance between tracked objects and detections. This distance is already integrated in Norfair and can be used simply by setting the string `"euclidean"` when building the tracker. This is possibly the simplest distance function you could use in Norfair, as it uses just one single point per detection/object.

```python
def euclidean_distance(detection, tracked_object):
Expand Down Expand Up @@ -132,7 +132,7 @@ detector = DefaultPredictor(cfg)

# Norfair
video = Video(input_path="video.mp4")
tracker = Tracker(distance_function=euclidean_distance, distance_threshold=20)
tracker = Tracker(distance_function="euclidean", distance_threshold=20)

for frame in video:
detections = detector(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
Expand Down
6 changes: 3 additions & 3 deletions docs/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ from norfair import Detection, Tracker, Video, draw_tracked_objects

detector = MyDetector() # Set up a detector
video = Video(input_path="video.mp4")
tracker = Tracker(distance_function="frobenius", distance_threshold=100)
tracker = Tracker(distance_function="euclidean", distance_threshold=100)

for frame in video:
detections = detector(frame)
Expand Down Expand Up @@ -76,11 +76,11 @@ After inspecting the detections you might find issues with the tracking, several
- Objects take **too long to start**, this can have multiple causes:
- `initialization_delay` is too big on the Tracker. Makes the TrackedObject stay on initializing for too long, `3` is usually a good value to start with.
- `distance_threshold` is too small on the Tracker. Prevents the Detections to be matched with the correct TrackedObject. The best value depends on the distance used.
- Incorrect `distance_function` on the Tracker. Some distances might not be valid in some cases, for instance, if using IoU but the objects in your video move so quickly that there is never an overlap between the detections of consecutive frames. Try different distances, `frobenius` or `create_normalized_mean_euclidean_distance` are good starting points.
- Incorrect `distance_function` on the Tracker. Some distances might not be valid in some cases, for instance, if using IoU but the objects in your video move so quickly that there is never an overlap between the detections of consecutive frames. Try different distances, `euclidean` or `create_normalized_mean_euclidean_distance` are good starting points.
- Objects take **too long to disappear**. Lower `hit_counter_max` on the Tracker.
- Points or bounding boxes **jitter too much**. Increase `R` (measurement error) or lower `Q` (estimate or process error) on the `OptimizedKalmanFilterFactory` or `FilterPyKalmanFilterFactory`. This makes the Kalman Filter put less weight on the measurements and trust more on the estimate, stabilizing the result.
- **Camera motion** confuses the Tracker. If the camera moves, the apparent movement of objects can become too erratic for the Tracker. Use `MotionEstimator`.
- **Incorrect matches** between Detections and TrackedObjects, a couple of scenarios can cause this:
- `distance_threshold` is too big so the Tracker matches Detections to TrackedObjects that are simply too far. Lower the threshold until you fix the error, the correct value will depend on the distance function that you're using.
- Mismatches when objects overlap. In this case, tracking becomes more challenging, usually, the quality of the detection degrades causing one of the objects to be missed or creating a single big detection that includes both objects. On top of the detection issues, the tracker needs to decide which detection should be matched to which TrackedObject which can be error-prone if only considering spatial information. The solution is not easy but incorporating the notion of the appearance similarity based on some kind of embedding to your distance_finction can help.
- Mismatches when objects overlap. In this case, tracking becomes more challenging, usually, the quality of the detection degrades causing one of the objects to be missed or creating a single big detection that includes both objects. On top of the detection issues, the tracker needs to decide which detection should be matched to which TrackedObject which can be error-prone if only considering spatial information. The solution is not easy but incorporating the notion of the appearance similarity based on some kind of embedding to your distance_function can help.
- Can't **recover** an object **after occlusions**. Use ReID distance, see [this demo](https://github.com/tryolabs/norfair/tree/master/demos/reid) for an example but for real-world use you will need a good ReID model that can provide good embeddings.
2 changes: 1 addition & 1 deletion norfair/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
>>> from norfair import Detection, Tracker, Video, draw_tracked_objects
>>> detector = MyDetector() # Set up a detector
>>> video = Video(input_path="video.mp4")
>>> tracker = Tracker(distance_function="frobenious", distance_threshold=50)
>>> tracker = Tracker(distance_function="euclidean", distance_threshold=50)
>>> for frame in video:
>>> detections = detector(frame)
>>> norfair_detections = [Detection(points) for points in detections]
Expand Down
2 changes: 1 addition & 1 deletion norfair/drawing.py
Original file line number Diff line number Diff line change
Expand Up @@ -790,7 +790,7 @@ class FixedCamera:
Examples
--------
>>> # setup
>>> tracker = Tracker("frobenious", 100)
>>> tracker = Tracker("euclidean", 100)
>>> motion_estimator = MotionEstimator()
>>> video = Video(input_path="video.mp4")
>>> fixed_camera = FixedCamera()
Expand Down

0 comments on commit 0815f94

Please sign in to comment.