While using DeepOCSort algorithm failes with KeyError: 'last_measurement'

[alaap001]

Search before asking

• I have searched the Yolo Tracking issues and found no similar bug report.

Question

Hey, thanks for the work.
I was trying DEEPOCSORT on a video and got this error:

traceback (most recent call last):
  File "/media/videos/t/tracktest.py", line 60, in <module>
    tracker.update(dets, im)
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/utils/__init__.py", line 81, in wrapper
    tracks = self.update(instance, dets, im)
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/trackers/deepocsort/deep_ocsort.py", line 397, in update
    trk.apply_affine_correction(transform)
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/trackers/deepocsort/deep_ocsort.py", line 266, in apply_affine_correction
    self.kf.apply_affine_correction(m, t, self.new_kf)
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/motion/kalman_filters/xysr_kf.py", line 407, in apply_affine_correction
    self.attr_saved["last_measurement"][:2] = m @ self.attr_saved["last_measurement"][:2] + t
KeyError: 'last_measurement'

But when I use BotSort everything is working just fine. Not sure why that is.
Here is my code for reference:

import numpy as np
from pathlib import Path
import cv2
from ultralytics import YOLOv10
import traceback
def create_video_writer(video_cap, output_filename):
    # Grab the width, height, and fps of the frames in the video stream.
    frame_width = int(video_cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    frame_height = int(video_cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    fps = int(video_cap.get(cv2.CAP_PROP_FPS))

    # Initialize the FourCC and a video writer object
    fourcc = cv2.VideoWriter_fourcc(*'XVID')
    writer = cv2.VideoWriter(output_filename, fourcc, fps, (frame_width, frame_height))

    return writer

from boxmot import BoTSORT, DeepOCSORT

# Initialize the tracker
tracker = DeepOCSORT(
    model_weights = Path('osnet_x1_0_msmt17.pt'),  # which ReID model to use
    device = 'cuda:0',
    fp16 = False,
)

yolo_model = YOLOv10('yolov10l.pt')

input_video_path = 'input.mp4'
vid = cv2.VideoCapture(input_video_path)

if not vid.isOpened():
    print("Error: Could not open input video.")
    exit()

# Define the output video path
output_video_path = 'output.mp4'

# Create the video writer
out = create_video_writer(vid, output_video_path)
frame_count = 4500
while frame_count:
    frame_count -= 1
    ret, im = vid.read()
    if not ret:
        break

    try:
        results = yolo_model(im)
        dets = []
        for result in results:
            for detection in result.boxes.data.cpu().numpy():
                x1, y1, x2, y2, conf, cls = detection
                dets.append([x1, y1, x2, y2, conf, int(cls)])
        dets = np.array(dets)

        tracker.update(dets, im)
        tracker.plot_results(im, show_trajectories = False)

        out.write(im)

    except Exception as e:
        print(f"An error occurred: {e}")
        print(traceback.print_exc())
        break

I am unable to debug this, any help is appreciated, thanks.

[mikel-brostrom]

Fixed. I have reverted my changes. They need more testing...

[alaap001]

Hey, thanks, it worked after your fix but there seems to be 1 more bug in the repo.

I am on latest version, 10.0.73, and when I set the param new_kf_off=True I get the following error:

tracks = tracker.update(dets, im)
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/utils/__init__.py", line 81, in wrapper
    tracks = self.update(instance, dets, im)
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/trackers/deepocsort/deep_ocsort.py", line 497, in update
    trk = KalmanBoxTracker(
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/trackers/deepocsort/deep_ocsort.py", line 140, in __init__
    self.kf = OCSortKalmanFilterAdapter(dim_x=7, dim_z=4)
NameError: name 'OCSortKalmanFilterAdapter' is not defined

Let me know what I can do to fix this.
Thanks.

[mikel-brostrom]

Yup, on this.

[alaap001]

Thanks a lot, @mikel-brostrom

On another note, I also have a curious case related to association(if I am not wrong) I have been thinking about how to solve it, and any advice from an expert would be appreciated.

Brief of issue:

I have a human walking in 1 direction and id is being properly tracked to 0.
Later he instantly chose to go in opposite direction, and as he does that the detector misses detection for 1 frame and 1 frame only. As soon as I get the new bbox (which is diff in size and In different direction) tracker assign it a different id of 1.

Now what I cannot fathom is how just 1 missed det led to change in id. I checked the boxes, and IOU is > 0.3 and operator hasn't moved too far, only in opposite dir. The size of bbox changes de to angle so I fig crop could have diff features.

But I suspect association messed it up or could be kalman filter by I tried setting w_association_emb=0.99 to put more weight on ReID in hope that ReID will surely work but apparently that did not help.

I have tried playing with inertia, max age, and other params to diff values but problem persists.

How could we have solved this problem if we were to? it will be a common thing that object changed it's motion and in btw missed 1 or 2 dets, new det should still be assigned the same id but here in this case it is not.

Thank you.

[mikel-brostrom]

Hey, thanks, it worked after your fix but there seems to be 1 more bug in the repo.

I am on latest version, 10.0.73, and when I set the param new_kf_off=True I get the following error:

tracks = tracker.update(dets, im)
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/utils/__init__.py", line 81, in wrapper
    tracks = self.update(instance, dets, im)
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/trackers/deepocsort/deep_ocsort.py", line 497, in update
    trk = KalmanBoxTracker(
  File "/home/alaap/anaconda3/envs/env_py310/lib/python3.10/site-packages/boxmot/trackers/deepocsort/deep_ocsort.py", line 140, in __init__
    self.kf = OCSortKalmanFilterAdapter(dim_x=7, dim_z=4)
NameError: name 'OCSortKalmanFilterAdapter' is not defined

Let me know what I can do to fix this. Thanks.

Should be fixed 🚀

[mikel-brostrom]

Thanks a lot, @mikel-brostrom

On another note, I also have a curious case related to association(if I am not wrong) I have been thinking about how to solve it, and any advice from an expert would be appreciated.

Brief of issue:

I have a human walking in 1 direction and id is being properly tracked to 0. Later he instantly chose to go in opposite direction, and as he does that the detector misses detection for 1 frame and 1 frame only. As soon as I get the new bbox (which is diff in size and In different direction) tracker assign it a different id of 1.

Now what I cannot fathom is how just 1 missed det led to change in id. I checked the boxes, and IOU is > 0.3 and operator hasn't moved too far, only in opposite dir. The size of bbox changes de to angle so I fig crop could have diff features.

But I suspect association messed it up or could be kalman filter by I tried setting w_association_emb=0.99 to put more weight on ReID in hope that ReID will surely work but apparently that did not help.

I have tried playing with inertia, max age, and other params to diff values but problem persists.

How could we have solved this problem if we were to? it will be a common thing that object changed it's motion and in btw missed 1 or 2 dets, new det should still be assigned the same id but here in this case it is not.

Thank you.

Please open a new issue 😄 . As it is out of scope for this one. I will try to help you there