Skip to content

Commit

Permalink
Add requested changes for vectorized distances.
Browse files Browse the repository at this point in the history
  • Loading branch information
facundo-lezama committed Nov 10, 2022
1 parent 0815f94 commit 74700c0
Show file tree
Hide file tree
Showing 6 changed files with 38 additions and 34 deletions.
7 changes: 0 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,13 +100,6 @@ Most tracking demos are showcased with vehicles and pedestrians, but the detecto

Norfair works by estimating the future position of each point based on its past positions. It then tries to match these estimated positions with newly detected points provided by the detector. For this matching to occur, Norfair can rely on any distance function. There are some predefined distances already integrated in Norfair, and the users can also define their own custom distances. Therefore, each object tracker can be made as simple or as complex as needed.

The following is an example of a particularly simple distance function calculating the Euclidean distance between tracked objects and detections. This distance is already integrated in Norfair and can be used simply by setting the string `"euclidean"` when building the tracker. This is possibly the simplest distance function you could use in Norfair, as it uses just one single point per detection/object.

```python
def euclidean_distance(detection, tracked_object):
return np.linalg.norm(detection.points - tracked_object.estimate)
```

As an example we use [Detectron2](https://github.com/facebookresearch/detectron2) to get the single point detections to use with this distance function. We just use the centroids of the bounding boxes it produces around cars as our detections, and get the following results.

![Tracking cars with Norfair](https://raw.githubusercontent.com/tryolabs/norfair/master/docs/videos/traffic.gif)
Expand Down
2 changes: 1 addition & 1 deletion demos/yolov4/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@ WORKDIR /demo
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

RUN pip install git+https://github.com/tryolabs/norfair.git@master#egg=norfair[metrics]
RUN pip install git+https://github.com/tryolabs/norfair.git@master

WORKDIR /demo/src
1 change: 1 addition & 0 deletions demos/yolov4/requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
opencv-python==4.6.0.66
importlib-metadata==4.8.3
55 changes: 33 additions & 22 deletions norfair/distances.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ def get_distances(
Parameters
----------
objects: Sequence[TrackedObject]
objects : Sequence[TrackedObject]
Sequence of [TrackedObject][norfair.tracker.TrackedObject] to be compared with potential [Detection][norfair.tracker.Detection] or [TrackedObject][norfair.tracker.TrackedObject]
candidates.
candidates: Union[List[Detection], List[TrackedObject]], optional
candidates : Union[List[Detection], List[TrackedObject]], optional
List of candidates ([Detection][norfair.tracker.Detection] or [TrackedObject][norfair.tracker.TrackedObject]) to be compared to [TrackedObject][norfair.tracker.TrackedObject].
Returns
Expand All @@ -48,10 +48,7 @@ class ScalarDistance(Distance):
Parameters
----------
distance_function : Union[
Callable[["Detection", "TrackedObject"], float],
Callable[["TrackedObject", "TrackedObject"], float],
],
distance_function : Union[Callable[["Detection", "TrackedObject"], float], Callable[["TrackedObject", "TrackedObject"], float]]
Distance function used to determine the pointwise distance between new candidates and objects.
This function should take 2 input arguments, the first being a `Union[Detection, TrackedObject]`,
and the second [TrackedObject][norfair.tracker.TrackedObject]. It has to return a `float` with the distance it calculates.
Expand All @@ -76,10 +73,10 @@ def get_distances(
Parameters
----------
objects: Sequence[TrackedObject]
objects : Sequence[TrackedObject]
Sequence of [TrackedObject][norfair.tracker.TrackedObject] to be compared with potential [Detection][norfair.tracker.Detection] or [TrackedObject][norfair.tracker.TrackedObject]
candidates.
candidates: Union[List[Detection], List[TrackedObject]], optional
candidates : Union[List[Detection], List[TrackedObject]], optional
List of candidates ([Detection][norfair.tracker.Detection] or [TrackedObject][norfair.tracker.TrackedObject]) to be compared to [TrackedObject][norfair.tracker.TrackedObject].
Returns
Expand Down Expand Up @@ -107,12 +104,14 @@ def get_distances(

class VectorizedDistance(Distance):
"""
VectorizedDistance class represents a distance that is calculated pairwise.
VectorizedDistance class represents a distance that is calculated in a vectorized way. This means
that instead of going through every pair and explicitly calculating its distance, VectorizedDistance
uses the entire vectors to compare to each other in a single operation.
Parameters
----------
distance_function : Callable[[np.ndarray, np.ndarray], np.ndarray]
Distance function used to determine the pairwise distances between new candidates and objects.
Distance function used to determine the distances between new candidates and objects.
This function should take 2 input arguments, the first being a `np.ndarray` and the second
`np.ndarray`. It has to return a `np.ndarray` with the distance matrix it calculates.
"""
Expand All @@ -133,10 +132,10 @@ def get_distances(
Parameters
----------
objects: Sequence[TrackedObject]
objects : Sequence[TrackedObject]
Sequence of [TrackedObject][norfair.tracker.TrackedObject] to be compared with potential [Detection][norfair.tracker.Detection] or [TrackedObject][norfair.tracker.TrackedObject]
candidates.
candidates: Union[List[Detection], List[TrackedObject]], optional
candidates : Union[List[Detection], List[TrackedObject]], optional
List of candidates ([Detection][norfair.tracker.Detection] or [TrackedObject][norfair.tracker.TrackedObject]) to be compared to [TrackedObject][norfair.tracker.TrackedObject].
Returns
Expand Down Expand Up @@ -191,12 +190,13 @@ def _compute_distance(
) -> np.ndarray:
"""
Method that computes the pairwise distances between new candidates and objects.
It is intended to use the entire vectors to compare to each other in a single operation.
Parameters
----------
stacked_candidates: np.ndarray
stacked_candidates : np.ndarray
np.ndarray containing a stack of candidates to be compared with the stacked_objects.
stacked_objects: np.ndarray
stacked_objects : np.ndarray
np.ndarray containing a stack of objects to be compared with the stacked_objects.
Returns
Expand All @@ -209,7 +209,7 @@ def _compute_distance(

class ScipyDistance(VectorizedDistance):
"""
ScipyDistance class extends VectorizedDistance for the use of Scipy's pairwise distances.
ScipyDistance class extends VectorizedDistance for the use of Scipy's vectorized distances.
This class uses `scipy.spatial.distance.cdist` to calculate distances between two `np.ndarray`.
Expand All @@ -235,12 +235,13 @@ def _compute_distance(
) -> np.ndarray:
"""
Method that computes the pairwise distances between new candidates and objects.
It is intended to use the entire vectors to compare to each other in a single operation.
Parameters
----------
stacked_candidates: np.ndarray
stacked_candidates : np.ndarray
np.ndarray containing a stack of candidates to be compared with the stacked_objects.
stacked_objects: np.ndarray
stacked_objects : np.ndarray
np.ndarray containing a stack of objects to be compared with the stacked_objects.
Returns
Expand Down Expand Up @@ -472,21 +473,31 @@ def iou_opt(detection: "Detection", tracked_object: "TrackedObject") -> float:
"sqeuclidean",
"yule",
]
AVAILABLE_VECTORIZED_DISTANCES = (
list(_VECTORIZED_DISTANCE_FUNCTIONS.keys()) + _SCIPY_DISTANCE_FUNCTIONS
)


def get_distance_by_name(name: str) -> Distance:
f"""
"""
Select a distance by name.
Valid names are: `{list(_SCALAR_DISTANCE_FUNCTIONS.keys()) + list(_VECTORIZED_DISTANCE_FUNCTIONS.keys()) + _SCIPY_DISTANCE_FUNCTIONS}`.
Parameters
----------
name : str
A string defining the metric to get.
Returns
-------
Distance
The distance object.
"""

if name in _SCALAR_DISTANCE_FUNCTIONS:
warning(
"You are using a scalar distance function. If you want to speed up the"
" tracking process please consider using a vectorized distance function"
" such as"
f" {list(_VECTORIZED_DISTANCE_FUNCTIONS.keys()) + _SCIPY_DISTANCE_FUNCTIONS}."
f" such as {AVAILABLE_VECTORIZED_DISTANCES}."
)
distance = _SCALAR_DISTANCE_FUNCTIONS[name]
distance_function = ScalarDistance(distance)
Expand All @@ -498,7 +509,7 @@ def get_distance_by_name(name: str) -> Distance:
else:
raise ValueError(
f"Invalid distance '{name}', expecting one of"
f" {list(_SCALAR_DISTANCE_FUNCTIONS.keys()) + list(_VECTORIZED_DISTANCE_FUNCTIONS.keys()) + _SCIPY_DISTANCE_FUNCTIONS}"
f" {list(_SCALAR_DISTANCE_FUNCTIONS.keys()) + AVAILABLE_VECTORIZED_DISTANCES}"
)

return distance_function
Expand Down
6 changes: 2 additions & 4 deletions norfair/tracker.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@
from norfair.camera_motion import CoordinatesTransformation

from .distances import (
_SCIPY_DISTANCE_FUNCTIONS,
_VECTORIZED_DISTANCE_FUNCTIONS,
AVAILABLE_VECTORIZED_DISTANCES,
ScalarDistance,
get_distance_by_name,
)
Expand Down Expand Up @@ -103,8 +102,7 @@ def __init__(
warning(
"You are using a scalar distance function. If you want to speed up the"
" tracking process please consider using a vectorized distance"
" function such as"
f" {list(_VECTORIZED_DISTANCE_FUNCTIONS.keys()) + _SCIPY_DISTANCE_FUNCTIONS}."
f" function such as {AVAILABLE_VECTORIZED_DISTANCES}."
)
distance_function = ScalarDistance(distance_function)
else:
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ classifiers = [
python = "^3.6.1"
filterpy = "^1.4.5"
rich = ">=9.10.0, <13.0.0"
scipy = ">=1.5.4"
opencv-python = { version = ">= 3.2.0, < 5.0.0", optional = true }
motmetrics = { version = "1.2.5", optional = true }
importlib-metadata = { version = "4.8.3", python = "<3.8" }
Expand Down

0 comments on commit 74700c0

Please sign in to comment.