modes/track/

[giscus[bot]]

modes/track/

Learn how to use Ultralytics YOLO for object tracking in video streams. Guides to use different trackers and customise tracker configurations.

https://docs.ultralytics.com/modes/track/

[AdadAlShabab]

Can I run two models simultaneously in one video?
Ex:
model = YOLO('E:/Ultralytics/best.pt')
model = YOLO('yolov8m-pose.pt')
results = model.track(source="file.mp4", show=True, conf=0.3, save=False)
results = model.track(source="file.mp4", show=True, conf=0.3, save=False)

I want that two models will works simultaneously with cumulative results? Is it possible? Please let me know. Thanks in advance!!

[pderrenger]

@AdadAlShabab yes, you can run two models simultaneously on one video, but you'll need to manage the inference in a way that allows both models to process the video frames concurrently. Here's a simplified example using threading to achieve this:

import threading
from ultralytics import YOLO

# Load the models
model1 = YOLO('E:/Ultralytics/best.pt')
model2 = YOLO('yolov8m-pose.pt')

# Function to run tracking
def track_video(model, source, conf, show, save):
    results = model.track(source=source, show=show, conf=conf, save=save)
    # Process results here or return them

# Create threads for each model
thread1 = threading.Thread(target=track_video, args=(model1, "file.mp4", 0.3, True, False))
thread2 = threading.Thread(target=track_video, args=(model2, "file.mp4", 0.3, True, False))

# Start the threads
thread1.start()
thread2.start()

# Wait for both threads to finish
thread1.join()
thread2.join()

This script will run both models in separate threads. If you want to combine the results from both models, you'll need to implement a mechanism to synchronize and merge the results from both threads. Keep in mind that displaying the results (show=True) from two threads might not work as expected, and you might need to handle the visualization separately.

Remember to handle the GPU memory usage carefully, as running two models simultaneously will require more memory. If you encounter any issues, consider running the models sequentially or on separate GPUs if available.

[Rid1Whitehead]

Hi,

First of all, I have been loving working with Yolov8. Great tool!

However, I have been having difficulties with a certain task. I want to use model.track on videos that I have, and then use save_crop = True, but save with a naming convention where I can track each persons ID. Currently, save_crop just gives me the cropped images of the objectes detected, but there is not way to know from which frame of the video are the crops, also, which ID is attached which cropped image. The visualization through cv2.imshow shows the IDs accross the different frames, but I cant find a way to save them. The naming convention I am looking for is something like this: "frame_30_ID_1.jpg" My current code looks something like this:

from ultralytics import YOLO
import cv2

model = YOLO("yolov8n.pt") # load model

video_path = "path/to/video.mp4"
cap = cv2.VideoCapture(video_path)

ret = True
frame_count = 0 # Initialize a frame counter

while ret:
ret, frame = cap.read()
frame_count += 1 # Increment the frame counter

# Check if the frame is the 30th frame
if ret and frame_count % 30 == 0:
    results = model.track(frame, persist=True, save_crop = True, save_txt = True)
    frame_ = results[0].plot()

    cv2.imshow("frame", frame_)
    if cv2.waitKey(25) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Any help would be greatly apprecitated! Thanks!

[pderrenger]

@Rid1Whitehead hello! I'm glad to hear you're enjoying working with YOLOv8! 😊

For your task of saving cropped images with a specific naming convention that includes the frame number and ID, you'll need to customize the saving part of the code. The save_crop functionality in YOLOv8 does not currently support custom naming directly related to frame and ID out of the box.

Here's a modified version of your code that manually saves the cropped images with the desired naming convention:

from ultralytics import YOLO
import cv2
import os

model = YOLO("yolov8n.pt")  # load model

video_path = "path/to/video.mp4"
cap = cv2.VideoCapture(video_path)

ret = True
frame_count = 0  # Initialize a frame counter
save_dir = "path/to/save/crops"  # Directory to save the crops
os.makedirs(save_dir, exist_ok=True)  # Create the directory if it doesn't exist

while ret:
    ret, frame = cap.read()
    frame_count += 1  # Increment the frame counter

    if ret:
        results = model.track(frame, persist=True)
        for i, det in enumerate(results[0].boxes.xyxy):
            # Extract bounding box coordinates
            x1, y1, x2, y2 = map(int, det[:4])
            crop = frame[y1:y2, x1:x2]  # Crop the detected object

            # Save the cropped image with the desired naming convention
            crop_name = f"frame_{frame_count}_ID_{results[0].boxes.id[i]}.jpg"
            crop_path = os.path.join(save_dir, crop_name)
            cv2.imwrite(crop_path, crop)

        frame_ = results[0].plot()

        cv2.imshow("frame", frame_)
        if cv2.waitKey(25) & 0xFF == ord('q'):
            break

cap.release()
cv2.destroyAllWindows()

In this code, we loop through each detection in the current frame, extract the bounding box coordinates, crop the detected object from the frame, and save it with the naming convention you specified. Make sure to replace "path/to/save/crops" with the actual path where you want to save the cropped images.

Please note that this code assumes that the model.track() method returns the IDs in the results[0].boxes.id attribute. If the IDs are not available or the attribute name is different, you'll need to adjust the code accordingly.

I hope this helps! If you have any more questions or need further assistance, feel free to ask. Happy coding! 🚀

[AdadAlShabab]

Hi @pderrenger . Can I run the models using my phones camera? Can you please share the code to invoke my mobile's camera to test the model? Thanks in advance.

[pderrenger]

@AdadAlShabab hi there! 👋 Yes, you can use your phone's camera to test the YOLO models. If you're looking to connect directly from your phone, you might need an app that can stream your camera feed to a URL. Then, you can use that URL as the source in the model.track() method. Here's a basic example of how you might set it up in Python:

from ultralytics import YOLO

# Load your trained model
model = YOLO('path/to/your/model.pt')

# Replace 'your_stream_url' with the URL from your phone's camera stream
results = model.track(source='your_stream_url', show=True)

Make sure your phone and the computer running the model are on the same network. Happy tracking! 😊

[Liuchunxiao111]

There's no problem at all using your way, but when I use it, the video streaming latency is extraordinarily high, why is that?

import cv2
from ultralytics import YOLO

# Load the YOLOv8 model
model= YOLO('weights/det/yolov8n.pt')

# Open the video file
cap = cv2.VideoCapture("http://192.168.9.123:4747/video")

# Loop through the video frames
while cap.isOpened():
    # Read a frame from the video
    success, frame = cap.read()

    if success:
        # Run YOLOv8 tracking on the frame, persisting tracks between frames
        results = model.track(frame, persist=True)

        # Visualize the results on the frame
        annotated_frame = results[0].plot()

        # Display the annotated frame
        cv2.imshow("YOLOv8 Tracking", annotated_frame)

        # Break the loop if 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        # Break the loop if the end of the video is reached
        break

# Release the video capture object and close the display window
cap.release()
cv2.destroyAllWindows()

[pderrenger]

Hi there! 🙌 The high latency you're experiencing during video streaming might be due to a few factors. Here are a couple of things you could check or adjust to potentially reduce the latency:

1. Network Speed: If you're streaming the video over a network, ensure that your network connection is fast and stable. Camera streaming apps can sometimes consume a lot of bandwidth.

2. Frame Processing: The model's predictions and the layer of annotations being added to each frame can be computationally intensive. You could try to reduce the image size for detection or lower the frame rate to see if it helps.

3. Hardware Acceleration: If possible, make sure that your system is using any available hardware acceleration to speed up video decoding and model inference.

4. Less Compute-Intensive Model: If latency is still an issue, consider using a smaller, less compute-intensive YOLO model.

Here’s a quick example of reducing image size which might help speed up the process:

import cv2
from ultralytics import YOLO

# Load the YOLOv8 model
model = YOLO('weights/det/yolov8n.pt')

# Reduce frames size to increase performance
def resize_frame(frame, scale=0.5):
    width = int(frame.shape[1] * scale)
    height = int(frame.shape[0] * scale)
    dimensions = (width, height)

    return cv2.resize(frame, dimensions, interpolation=cv2.INTER_AREA)

# Open the video file
cap = cv2.VideoCapture("http://192.168.9.123:4747/video")

# Loop through the video frames
while cap.isOpened():
    success, frame = cap.read()

    if success:
        # Resize frame
        frame = resize_frame(frame, scale=0.5)

        # Run YOLOv8 tracking on the frame
        results = model.track(frame, persist=True)

        # Visualize the results
        annotated_frame = results[0].plot()
        cv2.imshow("YOLOv8 Tracking", annotated_frame)

        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        break

cap.release()
cv2.destroyAllWindows()

This adjusts the frame size being processed, which could potentially reduce the latency you're experiencing. Hope this helps! 😊 If issues persist, feel free to provide more information about your setup.

[Liuchunxiao111]

@pderrenger Thanks for your reply!

What puzzles me is that when I use the first method, the picture is smooth and when I use the second method, the picture has a high latency, why is that? I don't think it's a network or hardware acceleration issue.

Way 1

from ultralytics import YOLO

# Load your trained model
model= YOLO('weights/det/yolov8n.pt')

# Replace 'your_stream_url' with the URL from your phone's camera stream
results = model.track(source="http://192.168.9.123:4747/video", show=True)

Way2

import cv2
from ultralytics import YOLO

# Load the YOLOv8 model
model= YOLO('weights/det/yolov8n.pt')

# Open the video file
cap = cv2.VideoCapture("http://192.168.9.123:4747/video")

# Loop through the video frames
while cap.isOpened():
    # Read a frame from the video
    success, frame = cap.read()

    if success:
        # Run YOLOv8 tracking on the frame, persisting tracks between frames
        results = model.track(frame, persist=True)

        # Visualize the results on the frame
        annotated_frame = results[0].plot()

        # Display the annotated frame
        cv2.imshow("YOLOv8 Tracking", annotated_frame)

        # Break the loop if 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        # Break the loop if the end of the video is reached
        break

# Release the video capture object and close the display window
cap.release()
cv2.destroyAllWindows()

[pderrenger]

Hey @Liuchunxiao111! 🌟 The difference in latency between the two methods might be due to how the frames are processed and displayed. In the first method, the model.track() function handles both the tracking and the frame retrieval internally, which is optimized for efficiency. In the second method, using OpenCV's VideoCapture and processing each frame in a loop with model.track() introduces overhead, especially if the frame processing isn't keeping up with the frame rate of the video stream.

To reduce latency in the second method, you might consider:

• Reducing the resolution of the video stream.
• Simplifying the model if possible, or using a lighter model.
• Ensuring that the code runs on the most capable hardware available (like using a GPU if available).

These tweaks can help align the processing speed more closely with the incoming frame rate, reducing the noticeable lag. Hope this helps! 😊

[Boaruzhanchik]

Hi, help me understand why I get this error when tracking with segmentation model . My ultimate goal is to use a custom car plate segmentation model for tracking. Thank you very much

  File "/home/iosif/Documents/Dev/yolo_model/yolov8_train/yolov8_tracking_car_number/tracking_v4.py", line 80, in <module>
    annotated_frame = results[0].plot()
  File "/home/iosif/.local/lib/python3.10/site-packages/ultralytics/engine/results.py", line 239, in plot
    annotator.masks(pred_masks.data, colors=[colors(x, True) for x in idx], im_gpu=im_gpu)
  File "/home/iosif/.local/lib/python3.10/site-packages/ultralytics/utils/plotting.py", line 237, in masks
    self.im[:] = im_mask_np if retina_masks else ops.scale_image(im_mask_np, self.im.shape)
ValueError: could not broadcast input array from shape (384,640,3) into shape (568,1012,3)


from ultralytics import YOLO
import cv2
from collections import defaultdict
import numpy as np

class_list = [0, 2]
model = YOLO("/home/iosif/Documents/Dev/yolo_model/yolov8_train/yolov8s-seg.pt")
cap = cv2.VideoCapture('/home/iosif/Documents/Dev/yolo_model/yolov8_train/yolov8_tracking_car_number/reac_high_way2.mp4')
track_history = defaultdict(lambda: [])

while cap.isOpened():
    success, frame = cap.read()

    if success:
        results = model.track(frame, show=False, verbose=False, conf=0.4, classes=class_list, persist=True)
        if results[0].boxes.id is not None:
            boxes = results[0].boxes.xywh.cpu()
            track_ids = results[0].boxes.id.cpu().numpy().astype(int)
            annotated_frame = results[0].plot()

            for box, track_id in zip(boxes, track_ids):
                x, y, w, h = box
                track = track_history[track_id]
                track.append((float(x), float(y)))
                if len(track) > 40:
                    track.pop(0)
                points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))
                cv2.polylines(annotated_frame, [points], isClosed=False, color=(0, 0, 255), thickness=5)
            cv2.imshow("YOLOv8 Tracking", annotated_frame)


            class_ids = []
            boxes_id = results[0].boxes.id
            if len(boxes_id) > 1:
                id_values = boxes_id.tolist()
            else:
                id_value = boxes_id.item()

            clss = results[0].boxes.cls.cpu().tolist()
            names = results[0].names
            for box, cls in zip(boxes_id, clss):
                class_ids.append(cls)
                if cls == 0.0:
                    print("People detect")

        

        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        break

cap.release()
cv2.destroyAllWindows()

[pderrenger]

Hey there! 👋 It looks like you're encountering a shape mismatch error when plotting your tracking results. This often happens when the image dimensions don't align with the mask dimensions expected by the plot() function.

To resolve this, ensure that the image size you're using for tracking matches the size expected by the model. If you're using a custom model, double-check that the image size during training and inference is consistent.

Here's a quick fix you might try:

# Before plotting, resize your frame to match the model's expected input size
frame_resized = cv2.resize(frame, (model.imgsz[1], model.imgsz[0]))

# Now use the resized frame for tracking and plotting
results = model.track(frame_resized, show=False, verbose=False, conf=0.4, classes=class_list, persist=True)

Make sure to replace model.imgsz[1] and model.imgsz[0] with the actual width and height used during training if model.imgsz is not available.

Give this a shot, and if you're still stuck, feel free to share more details or reach out again! Happy tracking! 🚗💨

[Boaruzhanchik]

Thank you more for such a quick response, it worked

[glenn-jocher]

@Boaruzhanchik hey, that's awesome to hear it worked out for you! 🎉 If you dive into tracking with your custom car plate segmentation model and have any more questions or cool results to share, don't hesitate to reach out. Happy tracking! 🚗🔍

[Elvis-Kiilu2]

@glenn-jocher Hey, how do you implement with custom seg modem, kindly illustrate

[glenn-jocher]

Hey there! Implementing tracking with a custom segmentation model is pretty straightforward. You'll want to load your model and then use the track() method, specifying any particular classes you're interested in. Here's a quick example using a custom segmentation model for car plates:

from ultralytics import YOLO

# Load your custom segmentation model
model = YOLO('path/to/your/custom-seg-model.pt')

# Run tracking on your video source
results = model.track(source='path/to/your/video.mp4', classes=[your_car_plate_class_id])

Just replace 'path/to/your/custom-seg-model.pt' with the actual path to your model and your_car_plate_class_id with the class ID for car plates in your model. If you need more help or have specific questions, feel free to ask! 🚀

[potatoouoo]

Yolov8 has very high overall practicality. Can I implement tracking with two cameras? I hope that when a car tracked by camera A moves to camera B, its frame ID remains the same. However, currently there is always an ID switch happening. Is it because of the model's accuracy?
EX:
def cam1():
model = YOLO('updated-model.pt')
# cap.imgCap()
results = model.track(source="http://192.168.50.204:8080/?action=stream",show = True)
cv2.waitKey(0)
cv2.destroyAllWindows()

def cam2():
model = YOLO('updated-model.pt')
# cap.imgCap()
results = model.track(source="http://192.168.50.69:8080/?action=stream",show = True)
cv2.waitKey(0)
cv2.destroyAllWindows()

cap=cam

a = threading.Thread(target=cam1)
b = threading.Thread(target=cam2)

a.start()
b.start()

[glenn-jocher]

Hey there! Tracking continuity across multiple cameras, especially to maintain the same ID as an object moves from camera A to camera B, is indeed a complex task that involves more than just the accuracy of the model. It requires a method to correlate tracks across the different camera views, often referred to as multi-camera tracking or re-identification (Re-ID).

Regarding access to the global track ID or maintaining consistent IDs across cameras, it's correct that basetrack.py (or similar files, depending on the tracking implementation you're using) manages these IDs. However, for multi-camera setups, you might need to implement or use an existing Re-ID system that can match IDs based on features or appearance.

A simplistic approach could involve:

1. Extracting features of tracked objects.
2. Comparing features of new detections in the second camera with ongoing tracks from the first camera.
3. Assigning the same ID based on the similarity of the features.

This can get quite involved, as it requires managing the timing and spatial-correlation of detections across cameras, and the model's accuracy is just one of many factors.

For directly accessing or setting the track ID, if you're using a specific tracker that has its global track ID in basetrack.py, you might interact with it directly within your code (varying by implementation). However, multi-camera ID continuity typically requires a higher-level management system than what's provided in individual tracker files.

I hope that sheds a bit more light on it! Let me know if you delve into the re-identification approach or if you have other questions. 😊👍

[potatoouoo]

I have a new question, can I know the distance to the camera while tracking?

[glenn-jocher]

Hey there!

Regarding your question about knowing the distance to the camera while tracking, YOLOv8 or most object tracking algorithms, including those implemented in Ultralytics YOLO, primarily focus on detecting objects and potentially tracking their movement across frames. They don't inherently provide distance measurements since that would require additional depth-sensing hardware or methods.

However, if you have a calibrated camera setup or additional information about the scene, you might estimate distances using object sizes or other cues. For instance, in specific, controlled environments, the apparent size of known objects can give clues about distance based on their expected real-world size.

Alternatively, employing depth cameras or stereo vision setups can directly provide depth information. If using such sensors, depth data can be combined with tracking data to estimate distances of tracked objects.

I hope that helps a bit! If you're diving deeper into this, consider exploring topics like camera calibration, depth sensing technologies, and stereo vision. 😊📏🎥

[potatoouoo]

Hello, I would like to ask, my current tracking program is as shown below
retList2, frameTmp2 = cap2.imgCap()
results = model.track(frameTmp2,persist=True,show = True)

             if text2:
                 boxes = results[0].boxes.xywh.cpu()
                 if results[0].boxes.id is not None:
                     track_ids1 = results[0].boxes.id.int().cpu().tolist()
                     plate_info = tracker.get_plate_info()
                     print(plate_info)
                     time.sleep(1)

I need the X1X2Y1Y2 of the tracking frame in each frame to calculate the distance. Is there a way to get it?

[glenn-jocher]

Hello! To get the bounding box coordinates (X1, Y1, X2, Y2) for each tracked object, you can access the .xyxy attribute from the results object. Here’s how you can modify your code to print these coordinates along with the track IDs:

def cam1():
    global shard_track
    model = YOLO('updated-model.pt')
    shard_track = model.track(source="http://192.168.50.204:8080/?action=stream", show=True)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

def track_and_print_cam1():
    global shard_track
    if hasattr(shard_track, 'xyxy'):
        for det in shard_track.xyxy[0]:  # Assuming shard_track is the results object
            x1, y1, x2, y2, conf, cls, track_id = det
            print(f'Track ID: {track_id.item()}, BBox: ({x1.item()}, {y1.item()}, {x2.item()}, {y2.item()})')
    else:
        print("No tracking data found.")

This modification ensures you're extracting the bounding box coordinates directly from the detection results, which should help you calculate distances or any other metrics based on the positions of the objects. If you have any more questions or need further assistance, feel free to ask! 😊

[samyak197]

Hey there,
First of all, I love working with YOLO , functionalities for basically everything.

          Currently i was working with a project that requires me to track different employee in a shop and was wandering if YOLO could help me assigning identities to each employees and if i t cpuld track for the entire working day.

[pderrenger]

@samyak197 hey there! 😊

Absolutely, YOLO can assist with tracking employees in a shop. You can use our tracking models to assign unique IDs and track them throughout the day. For long-term tracking, make sure your video feed is continuous and the model is configured for persistent tracking.

Here's a quick example using YOLOv8:

from ultralytics import YOLO

# Load your custom trained model or an official YOLOv8 model
model = YOLO('path/to/your/custom-model.pt')  # Custom trained model

# Track employees in a video
results = model.track(source="path/to/shop/video.mp4", persist=True)

This will keep the identities consistent across frames. For more details, check out our tracking documentation. If you have any more questions or need further assistance, feel free to ask. Happy tracking! 🚀

[ParkerLischwe]

Hi I saw that I can use an openvino IR format model just like any other pytorch model and then run tracking like normal. I was wondering how I would load the IR '.xml' and '.bin' files as arguments into YOLO(), or if I should load my model using openvino library? Thanks.

[glenn-jocher]

Hey there! 👋

To load OpenVINO IR '.xml' and '.bin' files into YOLO(), you'll want to simply point to their containing directory, i.e.:

from ultralytics import YOLO

# Load your YOLOv8 OpenVINO model
model = YOLO('path/to/yolov8n_openvino_model/')  # exported with YOLO('yolov8n.pt').export(format='openvino')

# Track employees in a video
results = model.track(source="path/to/shop/video.mp4", persist=True)

Once you have your model loaded with OpenVINO, you can then proceed with tracking as you would normally with the YOLO model. You might want to use stream=True also for longer videos and streams. If you need further assistance, feel free to ask. Happy tracking! 😊

[ParkerLischwe]

Will give this a try, appreciate the timely response!

[pderrenger]

@ParkerLischwe glad to hear you're giving it a try! If you have any questions or need a hand with the tracking setup, just drop a message. Happy tracking! 😊🚀

[ANIRUDH1080Z]

Can i use yolov8 model to track and reidentify person with same id assigned to it in multiple camera feed ?

[glenn-jocher]

@ANIRUDH1080Z yes, you can use YOLOv8 models for multi-object tracking across multiple camera feeds. Each detected person can be assigned a unique ID that persists as they move through the field of view of different cameras. You can choose from different trackers like BoT-SORT and ByteTrack, and even customize the tracker configurations to suit your specific needs. Here's a simple example using the default tracker:

from ultralytics import YOLO

# Load a YOLOv8 model
model = YOLO('yolov8n.pt')

# Perform tracking on a video source
results = model.track(source="path/to/camera_feed.mp4", show=True)

For tracking across multiple cameras, you might need to synchronize the cameras and handle the ID reidentification logic, as the trackers maintain IDs within a single video stream. If you have a custom solution or improvements, consider contributing to our Trackers section! 🚀

[ANIRUDH1080Z]

Thank u so much for enlightening me about this.

[glenn-jocher]

You're welcome! I'm glad the information was helpful. If you're looking to implement multi-object tracking across multiple camera feeds, you can certainly use YOLOv8 models. While YOLOv8 can assign unique IDs within a single video stream, tracking across multiple cameras (especially with non-overlapping fields of view) typically requires additional logic to handle ID reidentification.

For such cases, you might consider integrating your YOLOv8 tracking with a reidentification (ReID) system that can match IDs based on appearance features. Here's a simple example of how you might set up tracking for a single camera feed:

from ultralytics import YOLO

# Load a YOLOv8 model
model = YOLO('yolov8n.pt')

# Perform tracking on a video source
results = model.track(source="path/to/camera_feed.mp4", show=True)

For multi-camera setups, after tracking, you would extract features from detected objects and use a ReID model to match identities across different camera feeds.

If you're developing a solution or have improvements, we'd love to see your contributions to our tracking capabilities! 🚀

[kshweikeh]

How can we only track moving objects in the Plotting Tracks Over Time code:

from collections import defaultdict

import cv2
import numpy as np

from ultralytics import YOLO

Load the YOLOv8 model

model = YOLO('yolov8n.pt')

Open the video file

video_path = "path/to/video.mp4"
cap = cv2.VideoCapture(video_path)

Store the track history

track_history = defaultdict(lambda: [])

Loop through the video frames

while cap.isOpened():
# Read a frame from the video
success, frame = cap.read()

if success:
    # Run YOLOv8 tracking on the frame, persisting tracks between frames
    results = model.track(frame, persist=True)

    # Get the boxes and track IDs
    boxes = results[0].boxes.xywh.cpu()
    track_ids = results[0].boxes.id.int().cpu().tolist()

    # Visualize the results on the frame
    annotated_frame = results[0].plot()

    # Plot the tracks
    for box, track_id in zip(boxes, track_ids):
        x, y, w, h = box
        track = track_history[track_id]
        track.append((float(x), float(y)))  # x, y center point
        if len(track) > 30:  # retain 90 tracks for 90 frames
            track.pop(0)

        # Draw the tracking lines
        points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))
        cv2.polylines(annotated_frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)

    # Display the annotated frame
    cv2.imshow("YOLOv8 Tracking", annotated_frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break
else:
    # Break the loop if the end of the video is reached
    break

Release the video capture object and close the display window

cap.release()
cv2.destroyAllWindows()

[kshweikeh]

I forgot to write "only " so its MOVING OBJECTS ONLY

[glenn-jocher]

@kshweikeh hey there! 👋 To track only moving objects, you can modify the code to calculate the displacement of each object between frames and only plot the tracks for those with significant movement. Here's a quick example of how you might do that:

# ... [previous code]

# Define a movement threshold (this value may need tuning)
movement_threshold = 5  # pixels

# Loop through the video frames
while cap.isOpened():
    # ... [previous code]

    # Get the previous and current center points
    for track_id in track_ids:
        if len(track_history[track_id]) >= 2:
            prev_center = track_history[track_id][-2]
            curr_center = track_history[track_id][-1]
            displacement = np.sqrt((curr_center[0] - prev_center[0]) ** 2 + (curr_center[1] - prev_center[1]) ** 2)

            # Only plot the track if the displacement is above the threshold
            if displacement > movement_threshold:
                points = np.array(track_history[track_id], dtype=np.int32).reshape((-1, 1, 2))
                cv2.polylines(annotated_frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)

    # ... [previous code]

This snippet calculates the Euclidean distance between the previous and current positions of each tracked object and only draws the track if the object has moved more than the movement_threshold. You can adjust the threshold to fit the specifics of your scenario. Happy coding! 🚀

[kshweikeh]

Thanks!!

[glenn-jocher]

@kshweikeh hey there! 👋

Absolutely, happy to help! For tracking moving objects with YOLOv8, you can utilize the track method with a custom tracker configuration that focuses on objects with significant displacement between frames. Here's a quick snippet to get you started:

from ultralytics import YOLO

# Load your model
model = YOLO('path/to/your/model.pt')

# Run the tracker
results = model.track(source='path/to/your/video.mp4', tracker='custom_tracker.yaml')

In your custom_tracker.yaml, you might want to adjust parameters that affect the detection of stationary vs. moving objects, such as minimum displacement thresholds or object persistence criteria.

If you need more guidance or have specific requirements, feel free to share, and we can dive deeper! 🚀

[RNVALA]

import cv2
from ultralytics import YOLO
import numpy as np

model = YOLO('yolov8_custom_train.engine', task="detect")

Path to the input video file

input_video_path = '/content/gdrive/MyDrive/yolov8-tensorrt/inference/output_video.mp4'

Path to the output video file

output_video_path = 'outputtest_video.mp4'

Define the coordinates of the polygon

polygon_points = [(670, 66), (1237, 550), (514, 1054), (161, 295)]
polygon_points = np.array(polygon_points)

Open the input video file

cap = cv2.VideoCapture(input_video_path)

Get video properties

frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = int(cap.get(cv2.CAP_PROP_FPS))
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))

Define the codec and create VideoWriter object

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter(output_video_path, fourcc, fps, (frame_width, frame_height))

Function for finding the centroid

def calculate_centroid(box):
x_min, y_min, x_max, y_max = map(int, box)
centroid_x = (x_min + x_max) // 2
centroid_y = (y_min + y_max) // 2
return (centroid_x, centroid_y)

Function to check if two bounding boxes overlap

def check_overlap(box1, box2):
x1_min, y1_min, x1_max, y1_max = map(int, box1)
x2_min, y2_min, x2_max, y2_max = map(int, box2)
overlap_x = max(0, min(x1_max, x2_max) - max(x1_min, x2_min))
overlap_y = max(0, min(y1_max, y2_max) - max(y1_min, y2_min))
overlap_area = overlap_x * overlap_y
return overlap_area

Read until video is completed

while cap.isOpened():
# Read a frame from the video
ret, frame = cap.read()

# If frame is read correctly ret is True
if not ret:
    break

results = model.track(source=frame, device=0)
# Extract bounding boxes, classes, names, and confidences
boxes = results[0].boxes.xyxy.tolist()
classes = results[0].boxes.cls.tolist()
names = results[0].names
confidences = results[0].boxes.conf.tolist()
#id=results[0].boxes.id
# Iterate through the results
red_area_count = 0
blue_area_count = 0
crowded_area_detected = False
crowded_box = None

for i, box1 in enumerate(boxes):
    overlap_count = 0
    for j, box2 in enumerate(boxes):
        if i != j:
            overlap_area = check_overlap(box1, box2)
            if overlap_area > 0:
                overlap_count += 1
                if overlap_count >= 2:
                    crowded_area_detected = True
                    if crowded_box is None:
                        crowded_box = box1
                    else:
                        crowded_box = [
                            min(crowded_box[0], box1[0]),
                            min(crowded_box[1], box1[1]),
                            max(crowded_box[2], box1[2]),
                            max(crowded_box[3], box1[3])
                        ]
                    break

for box, cls, conf in zip(boxes, classes, confidences):
    x1, y1, x2, y2 = box
    confidence = conf
    detected_class = cls
    name = names[int(cls)]
    
    if cv2.pointPolygonTest(polygon_points, calculate_centroid(box), False) >= 0:
        red_area_count += 1
        color = (0, 0, 255)
    else:
        color = (0, 255, 0)
        blue_area_count += 1

    x1, y1, x2, y2 = int(x1), int(y1), int(x2), int(y2)
    cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)
for i,det in enumerate(results[0].boxes.xyxy):
    x1, y1, x2, y2 = map(int, det[:4])
    cv2.putText(frame, f'ID: {results[0].boxes.id[i]}', (x1+5, y1+25), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255,255,255), 2, cv2.LINE_AA)
    
if crowded_area_detected:
    crowded_boxes = [box for box in boxes if check_overlap(crowded_box, box) > 0]
    x1, y1, x2, y2 = crowded_box
    for box in crowded_boxes:
        if box != crowded_box and check_overlap(crowded_box, box) > 0:
            x1 = min(x1, box[0])
            y1 = min(y1, box[1])
            x2 = max(x2, box[2])
            y2 = max(y2, box[3])
    cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 255), 2)
    cv2.putText(frame, 'Crowded Area', (int(x1), int(y1) - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 255, 255), 2)

font = cv2.FONT_HERSHEY_SIMPLEX

cv2.putText(frame, 'People working with attention - {}'.format(blue_area_count), (1125, 24), font, 1, (0, 255, 0), 2, cv2.LINE_AA)
cv2.putText(frame, 'People not working - {}'.format(red_area_count), (1124, 75), font, 1, (0, 0, 255), 2, cv2.LINE_AA)
cv2.line(frame, (668, 66), (1237, 550), (255, 0, 0), 2)
cv2.line(frame, (1237, 550), (515, 1052), (255, 0, 0), 2)

# Write the frame into the output video file
out.write(frame)

# Display the frame
# cv2_imshow(frame)

# Press Q on keyboard to exit
if cv2.waitKey(25) & 0xFF == ord('q'):
    break

Release video objects

cap.release()
out.release()

Close all OpenCV windows

cv2.destroyAllWindows()

In this, I am tracking a label Person but in the next 2 to 3 frames, ids are changing so any solution for this?
means if person having id 1, so after few frame it give as id 3, and every time the id is changing

[pderrenger]

Hey there! 👋 It looks like you're experiencing some ID consistency issues with your custom-trained model during tracking. This can happen due to various reasons, such as similar-looking objects or occlusions.

One approach to mitigate this is to fine-tune your tracker's hyperparameters. If you're using the default BoT-SORT tracker, you might want to experiment with the bytetrack.yaml configuration, which is designed to handle ID switches more robustly. Here's a quick example of how to switch to ByteTrack:

from ultralytics import YOLO

# Load your custom-trained model
model = YOLO('yolov8_custom_train.engine', task="detect")

# Perform tracking with ByteTrack
results = model.track(source="your_video.mp4", tracker="bytetrack.yaml")

If you've already tried adjusting the tracker settings and the issue persists, consider revisiting your training dataset to ensure it's diverse and representative enough to handle the scenarios where ID switches occur.

Keep us posted on your progress, and if you need more assistance, we're here to help! 😊

[RNVALA]

I have already used the bytetrack.yaml and this is ouput video , https://drive.google.com/file/d/1E95m456dbZAUDRrodfrvrsSA4zcs3KiA/view?usp=sharing , please analyze it

[pderrenger]

Hey there! Thanks for reaching out and sharing the video link. 🎥 I took a look, and it seems like the ID switches might be due to occlusions or similar-looking objects. Here are a couple of quick suggestions:

1. Tweak the IOU Threshold: Adjust the iou_thresh parameter in your bytetrack.yaml to a value that might help in maintaining consistent IDs.
2. Increase Detection Confidence: A higher conf_thresh could reduce false positives, which sometimes cause ID confusion.

If you've already fine-tuned these without success, consider:

• Re-training with Augmented Data: More varied training data can improve tracking robustness.
• Post-Processing: Implement a custom ID management system in post-processing to smooth out ID switches.

If you're still facing issues, feel free to share more details or reach out for further assistance. We're here to help! 😊👍

[Dmonzer]

What is the difference between these attributes of results[0].boxes:
xywh: tensor([], size=(0, 4))
xywhn: tensor([], size=(0, 4))
xyxy: tensor([], size=(0, 4))
xyxyn: tensor([], size=(0, 4))

[glenn-jocher]

Hey there! 👋 The attributes you're asking about represent different formats of bounding box coordinates:

• xywh: Bounding boxes in (center x, center y, width, height) format.
• xywhn: Normalized (center x, center y, width, height), with values from 0 to 1 relative to image size.
• xyxy: Bounding boxes in (top left x, top left y, bottom right x, bottom right y) format.
• xyxyn: Normalized (top left x, top left y, bottom right x, bottom right y), with values from 0 to 1.

The tensors being empty (size=(0, 4)) suggests no detections were made in the frame. If you have further questions or need assistance, feel free to ask! Happy coding! 😊

[WooFy11]

is it possible to use our own weighs as a model to track? or we must include the yolov8n.pt?

[pderrenger]

Absolutely, you can use your own custom weights for tracking! Just load your trained model instead of the default yolov8n.pt. Here's a quick example in Python:

from ultralytics import YOLO

# Load your custom trained model
model = YOLO('path/to/your/custom-weights.pt')

# Perform tracking with your model
results = model.track(source="path/to/your/video.mp4", show=True)

And that's it! Your custom model will now be used for tracking objects in your video stream. 🚀 If you have any more questions or need further assistance, feel free to ask. Happy tracking!

[WooFy11]

do you a discord so we can discuss about my project?

[pderrenger]

Hey there! 👋 We don't have an official Discord channel, but we're happy to help you right here or on our GitHub discussions page. If you have any questions about your project or need assistance with object tracking using YOLOv8, feel free to drop your questions, and we'll do our best to assist you! 🚀 If it's code-specific, you can also share snippets or error messages, and we'll take a look. Happy tracking!

[WooFy11]

i had an error when everytime i try to run a code, it crashed on 4xx/600, and i must restart it, and 2nd question can we make a custom annotator?

[pderrenger]

Hey there! 👋 It sounds like you're encountering a crash during your code execution. To better assist you, could you please share the specific error message you're getting? This will help us pinpoint the issue more accurately.

Regarding your second question, yes, you can create a custom annotator. If you're working with YOLOv8, you can use tools like CVAT or LabelImg for annotation, and then train your model with these custom annotations. Here's a quick example of how you might load and use your custom model for tracking:

from ultralytics import YOLO

# Load your custom trained model
model = YOLO('path/to/your/custom_model.pt')

# Perform tracking on your video
results = model.track(source='path/to/your/video.mp4', show=True)

This will use your custom weights for tracking objects in the video. If you have any more questions or need further assistance, feel free to ask! 🚀

[tumitoto007]

So I am using Yolov8 for my current project, it's been a breeze so far. I do have a question on the tracking method provided by the Yolov8. When I am using the generic yolov8n model(or even a custom model mixed with few objects), I know I can specifically filter out things that doesn't interest me by their ID as below:

    for data in .....boxes.data.tolist():
            # extract the id associated with the detection
            objectID= data[5]
   if not interested skip this object....

But, when I caught the an object that I am interested, can I at that time or at that frame, issue a track command to start tracking it? if it can be done, can you tell me how? an short example will be even better! thanks in advance.

[glenn-jocher]

@tumitoto007 hey there! 😊 It's great to hear that YOLOv8 has been a breeze for your project. Regarding your question about initiating tracking on a specific object at a certain frame, YOLOv8's tracking functionality is designed to automatically track all detected objects continuously across frames. However, if you're interested in tracking a specific object after its detection, you can filter detections by their IDs as you've mentioned and then focus on tracking that specific object ID across the frames.

Here's a simplified example to illustrate how you might approach this:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8n.pt')  # or your custom model

# Assuming you're processing video frames in a loop
interested_object_id = None  # Placeholder for the ID of the object you're interested in tracking
for frame in video_frames:
    results = model.track(frame, persist=True)
    for box in results[0].boxes:
        objectID = box.id.item()  # Get the ID of the current object
        if objectID == interested_object_id:
            # This is the object you're interested in, now you can track it specifically
            # Process the tracking data as needed
            pass

In this example, interested_object_id should be set to the ID of the object you're interested in tracking once you've identified it. From that point on, you can specifically process or track that object in subsequent frames.

Remember, the tracking starts automatically for all detected objects, so the key is filtering and focusing on the object of interest in your processing loop.

Hope this helps! If you have any more questions or need further assistance, feel free to ask. Happy tracking! 🚀

[tumitoto007]

Cool! I will give this a try once I get back to town. Thanks again for your great support.

[glenn-jocher]

Hey there! 🎉 I'm thrilled to hear you're planning to give tracking a try with YOLOv8 once you're back in town. It sounds like you're on an exciting journey with your project! If you have any questions or need further assistance when you dive into it, don't hesitate to reach out. Our community and docs are here to support you every step of the way. Happy tracking, and safe travels back to town! 🚀

[KhizarHashmi]

Hi, I want some detailed help and guidance on how to use custom tracker models with my custom yolov8 pose model, the Re-identification problem is being face using bytetrack.yaml so I think I should use StrongSORT or DeepSORT. Therefore, I want the ultralytics team to help me on selecting my tracker model or use multiple tracker models, and guide me properly on how to use them with my YOLOv8 custom trained model.
Thanks

[KhizarHashmi]

@glenn-jocher

[glenn-jocher]

@KhizarHashmi hi there! 👋 It sounds like you're exploring advanced tracking options for your custom YOLOv8 pose model. If ByteTrack isn't meeting your re-identification needs, exploring StrongSORT or DeepSORT could indeed be beneficial. These trackers offer sophisticated mechanisms for handling occlusions and maintaining identity across frames, which might be what you're looking for.

To integrate a custom tracker like StrongSORT or DeepSORT with your YOLOv8 model, you'll typically need to:

1. Ensure your tracker can process the detection output format of YOLOv8. This usually involves bounding box coordinates, confidence scores, and class IDs.
2. Modify your tracking script to use the YOLOv8 model for detection, feeding these detections into your chosen tracker for each frame of your video.

Here's a simplified example of how you might set this up in Python, assuming you have a function custom_tracker that takes detections and returns tracked objects:

from ultralytics import YOLO

# Load your custom trained YOLOv8 model
model = YOLO('path/to/your/custom_model.pt')

# Function to perform detection and tracking on a frame
def detect_and_track(frame):
    # Perform detection
    results = model(frame)
    
    # Extract detections (example format, adjust as needed)
    detections = results.pandas().xyxy[0]  # DataFrame of detections
    
    # Perform tracking with your custom tracker
    tracked_objects = custom_tracker(detections)
    
    return tracked_objects

# Example usage with a video stream
# ... video capture setup here ...
while True:
    ret, frame = cap.read()
    if not ret:
        break
    tracked_objects = detect_and_track(frame)
    # ... visualization and further processing here ...

Regarding using multiple trackers, it's possible to switch between trackers based on the scenario or even combine their strengths, though this can get complex. You might start with one tracker and evaluate its performance thoroughly before experimenting with combining them.

For more detailed guidance on integrating specific trackers or handling complex tracking scenarios, the documentation and community forums for those trackers can be valuable resources. Additionally, our Ultralytics Docs provide further insights into tracking configurations and options.

If you're developing new tracking capabilities or have insights to share, we'd love to see your contributions to our GitHub repo. Your expertise could greatly benefit the YOLO and tracking community! 🚀

Hope this helps! Let us know if you have more questions.

[Vinrub]

import random
import cv2
import numpy as np
from ultralytics import YOLO

opening the file in read mode

my_file = open("utils/coco.txt", "r")

reading the file

data = my_file.read()

replacing end splitting the text | when newline ('\n') is seen.

class_list = data.split("\n")
my_file.close()

Generate random colors for class list

detection_colors = []
for i in range(len(class_list)):
r = random.randint(0, 255)
g = random.randint(0, 255)
b = random.randint(0, 255)
detection_colors.append((b, g, r))

load a pretrained YOLOv8n model

model = YOLO("weights/yolov8n.pt", "v8")

Vals to resize video frames | small frame optimise the run

frame_wid = 640
frame_hyt = 480

def CarBehaviour(frame, color_threshold=1100):
# Convert the frame to HSV for color thresholding
hsv_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

# Define the lower and upper bounds for the color of the rear lights (adjust as needed)
lower_color = np.array([0, 100, 100])  # Lower bound for red color
upper_color = np.array([10, 255, 255]) # Upper bound for red color

# Create a mask using the inRange function to threshold the color
mask = cv2.inRange(hsv_frame, lower_color, upper_color)

# Calculate the number of nonzero pixels in the mask
color_pixels = cv2.countNonZero(mask)

# Check if color pixels are present
if color_pixels > color_threshold:
    return True, 'Braking'  # If color threshold is met, indicate braking
else:
    return False, 'Not Braking'  # Otherwise, indicate not braking

def detect_and_draw(frame, model, class_list, detection_colors):
# Predict on image
detect_params = model.track(source=[frame], conf=0.45, save=False)

# Convert tensor array to numpy
DP = detect_params[0].cpu().numpy()

if len(DP) != 0:
    for i in range(len(detect_params[0])):
        boxes = detect_params[0].boxes
        box = boxes[i]  # returns one box
        clsID = box.cls.cpu().numpy()[0]
        conf = box.conf.cpu().numpy()[0]
        bb = box.xyxy.cpu().numpy()[0]

        cv2.rectangle(
            frame,
            (int(bb[0]), int(bb[1])),
            (int(bb[2]), int(bb[3])),
            detection_colors[int(clsID)],
            3,
        )

        # Display class name and confidence
        font = cv2.FONT_HERSHEY_COMPLEX
        cv2.putText(
            frame,
            class_list[int(clsID)] + " " + str(round(conf, 3)) + "%",
            (int(bb[0]), int(bb[1]) - 10),
            font,
            1,
            (255, 255, 255),
            2,
        )

# Check car behavior
is_braking, behavior = CarBehaviour(frame)
# Display car behavior
cv2.putText(
    frame,
    "Car Behavior: " + behavior,
    (50, 50),
    cv2.FONT_HERSHEY_SIMPLEX,
    1,
    (0, 255, 0),
    2,
    cv2.LINE_AA,
)

Open video capture

cap = cv2.VideoCapture("/home/opencv_env/Vehicle-rear-lights-analyser-master/testing_data/road_2.mp4")

if not cap.isOpened():
print("Cannot open camera")
exit()

while True:
# Capture frame-by-frame
ret, frame = cap.read()
# if frame is read correctly ret is True

if not ret:
    print("Can't receive frame (stream end?). Exiting ...")
    break

#  resize the frame | small frame optimise the run
# frame = cv2.resize(frame, (frame_wid, frame_hyt))

# Detect and draw objects
detect_and_draw(frame, model, class_list, detection_colors)

# Display the resulting frame
cv2.imshow("ObjectDetection", frame)

# Terminate run when "Q" pressed
if cv2.waitKey(1) == ord("q"):
    break

When everything done, release the capture

cap.release()
cv2.destroyAllWindows()
in this code can i add tracking id number using yolov8

[glenn-jocher]

Absolutely, you can add tracking IDs to your detections using YOLOv8! 🚀 To do this, you'll want to utilize the track method available in the Ultralytics YOLO API. This method not only performs object detection but also assigns unique IDs to each detected object, allowing you to track their movement across frames.

Here's a simplified example to get you started:

from ultralytics import YOLO

# Load your model
model = YOLO("weights/yolov8n.pt")

# Assuming 'frame' is your input frame
results = model.track(source=[frame])

# Now, each detected object in 'results' has a unique tracking ID
for det in results[0].boxes:
    # Extract tracking ID
    track_id = det.id.cpu().numpy()
    print(f"Tracking ID: {track_id}")

This will print the tracking ID for each detected object in the frame. Make sure to loop over your video frames to continuously track objects. Happy tracking! 🎉

[zhaozhaotnl]

I have a question which is how can I get some tracking metrics such as MOTA ,MOTP from video.The follows are my code about track.
from collections import defaultdict
import cv2
import numpy as np
from ultralytics import YOLO

Load the YOLOv8 model

model = YOLO('3.pt')

Open the video file

video_path = "5.mp4"
cap = cv2.VideoCapture(video_path)

Store the track history

track_history = defaultdict(lambda: [])

output_path = 'video/5.mp4'

fps = cap.get(cv2.CAP_PROP_FPS)
width, height = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)), int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter(output_path, fourcc, fps, (width, height))

Loop through the video frames

while cap.isOpened():
# Read a frame from the video
success, frame = cap.read()

if success:
    # Run YOLOv8 tracking on the frame, persisting tracks between frames
    results = model.track(frame, persist=True)

    # Get the boxes and track IDs
    boxes = results[0].boxes.xywh.cpu()
    track_ids = results[0].boxes.id.int().cpu().tolist()

    # Uncomment the following line if you want to visualize the tracking lines only
    # annotated_frame = np.zeros_like(frame)

    # Plot the tracks
    for box, track_id in zip(boxes, track_ids):
        x, y, w, h = box
        track = track_history[track_id]
        track.append((float(x), float(y)))  # x, y center point
        if len(track) > 100:  # retain 90 tracks for 90 frames
            track.pop(0)

        # Draw the tracking lines
        points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))
        cv2.polylines(frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)

        # Add track ID as text
        cv2.putText(frame, str(track_id), (int(x), int(y)), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)

    out.write(frame)

    # Display the frame without YOLOv8 detection boxes
    cv2.imshow("YOLOv8 Tracking", frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break
else:
    # Break the loop if the end of the video is reached
    break

Release the video capture object and close the display window

cap.release()
cv2.destroyAllWindows()

out.release()

[pderrenger]

To calculate tracking metrics like MOTA and MOTP, you'll need to integrate an evaluation library that supports these metrics, as they are not directly provided by the Ultralytics YOLO package. A popular choice for this is the MOTChallenge evaluation toolkit, which can be used to compute these metrics from the tracking output.

Here's a brief guide on how you might proceed:

1. Track Output Formatting: Ensure your tracking output is in a format compatible with the evaluation toolkit, typically a CSV or TXT file listing frame number, track ID, bounding box coordinates, and confidence scores.
2. Evaluation Toolkit: Integrate an evaluation toolkit into your workflow. You might need to adapt your tracking output to the specific format required by the toolkit.
3. Run Evaluation: Use the toolkit to evaluate your tracking results against a ground-truth dataset that includes object IDs and bounding boxes.

For more detailed instructions on setting up and using an evaluation toolkit, you should refer to the specific documentation of the toolkit you choose to use. This setup will allow you to measure the accuracy and effectiveness of your tracking implementation comprehensively.

[zhaozhaotnl]

Thanks for answering!It helps me.

[pderrenger]

You're welcome! I'm glad to hear that the information was helpful. If you have any more questions or need further assistance as you continue with your tracking projects, feel free to reach out. Happy tracking! 🚀

[duhyeonlee]

Hello,

I am trying to run the above code in Jupyter Notebook.
I have been using YOLO without any crush or issue. But this time when I use "annotated_frame = results[0].plot()" (I guess this line causes the issue), my Jupyter Notebook keeps restarting.

How can I solve this issue?

Thank you!

[pderrenger]

Hello,

It sounds like you're encountering a resource-intensive operation that's causing your Jupyter Notebook to restart. This can sometimes happen with visualization tasks, especially with large images or videos.

Here are a few steps you can try to resolve the issue:

1. Reduce Image Size: Before plotting, try reducing the image size to decrease the computation load.
2. Increase Resources: Ensure that your Jupyter Notebook has enough RAM and CPU allocated. Sometimes, increasing available resources can prevent such crashes.
3. Use External Visualization Tools: Instead of plotting directly in the notebook, you could save the results to files and use an external program to view them.
4. Debugging: Check if there are any specific errors in the Jupyter log that might give more insight into why the crash is happening.

If the problem persists, consider running the code outside of Jupyter, such as in a Python script, to see if the issue is related to Jupyter itself.

Hope this helps! Let us know how it goes.

[AshishreddyM26]

While inferencing 'tracking' on a video, is it possible to find that the performance of the object detection model is affecting the overall tracking result or the trackers performance, because (there were some objects even without any detections (bboxes) )
I am using ultralytics 'track' function, and it is directly giving me the tracking results, so I want to know how the obj det model is performing during tracking. or Am I missing the trick of about the performance of obj det model that stays the same irrespective of detection or tracking ?

If not how would I approach this, can you give some insights into it, I will figure it out.

Thankyou,
Ashish

[glenn-jocher]

Hello Ashish,

Great question! The performance of the object detection model indeed influences the tracking results. If the detection model misses some objects (i.e., does not produce bounding boxes for them), these objects won't be tracked since tracking algorithms rely on the initial detections to follow objects across frames.

To assess how the detection model performs during tracking, you might consider running the detection model separately on the same video or images to see the detection results independently. This can help you identify if the model is missing detections or if adjustments to confidence thresholds (conf) or other parameters might improve detection before tracking.

If you're noticing missed detections, you might want to:

• Adjust the detection confidence threshold.
• Re-train or fine-tune the model with more representative data or augmentations.
• Experiment with different models or versions that might perform better on your specific data.

For more detailed insights and adjustments, reviewing the detection results separately from tracking, as suggested, can be very helpful. If you need further assistance, feel free to reach out!

[AshishreddyM26]

Glenn, thank you so much for your guidance.

I ran detection model separately (using pre-trained weights and tweaked with conf threshold) , the results look great.

I visually checked the detection and tracking resultant videos, I observed that there were missing bboxes in tracking, while they were detected by the detection model.

Also I have used both trackers - BoT-SORT & ByteTrack

The issue I am encountering of missing objects is with 'ByteTrack', whereas 'BoT-SORT' performed exceptionally well (in maintaining the id's of the objects across frames and also very few number of missing objects, but Bytetrack is assigning new id's to the same object if it appears again after losing its track, in this process, there were lot of missing detections, new id's to the same object with Bytetrack even though the detection model was well finetuned.)

I can proceed with BoT-SORT tracker as its results were brilliant but ByteTrack is capable of handling videos at 30 FPS (approx.) where as BoT-SORT settles for 6.6 FPS, that's why I am trying to understand the issue with ByteTrack and wanting to improve its performance.

Your insights would be really helpful.

Thank you Glenn,
Ashish

[glenn-jocher]

Hello Ashish,

Thank you for your detailed feedback and observations with ByteTrack and BoT-SORT. It's great to hear that BoT-SORT is performing well in maintaining object IDs and minimizing missed detections. Your experience with ByteTrack, particularly the issues with ID persistence and missed detections, is valuable for further optimization.

ByteTrack's tendency to assign new IDs may be influenced by several factors, including its handling of occlusions or objects that leave and re-enter the frame. Adjusting the track_high_thresh parameter might help reduce ID switches by increasing the confidence threshold for detections considered for tracking. Additionally, tweaking the iou thresholds used in the association step could potentially improve the tracker's ability to maintain IDs consistently.

Since ByteTrack is optimized for higher frame rates, ensuring that the frame rate of your input video matches the tracker's expectations is also crucial. Discrepancies here might affect tracking performance.

If you continue to prefer ByteTrack for its frame rate handling but need to address the ID persistence issue, consider integrating a post-processing step that analyzes the trajectory and appearance consistency to refine ID assignments.

For further tuning and assistance, our community and documentation on advanced tracker configurations might provide additional insights. Your continued feedback is invaluable in enhancing the tracking capabilities of Ultralytics YOLO.

[duhyeonlee]

Hello,

1. Is there a way I can "force" the model to not switch IDs and keep tracking the original ID and elmininating the other bounding boxes which are redundant to the same object.
2. Similar to 1. If the model misses the object in some frames, (how) can I use interpolation or the tracking algorithms to estimate the missed detections and draw its bounding box?

Thank you!

[glenn-jocher]

Hello,

1. To maintain consistent object IDs and eliminate redundant bounding boxes, you can adjust the track_high_thresh parameter in your tracker configuration file. This parameter helps in maintaining the ID by setting a higher confidence threshold for track continuation. You might also explore using the iou_threshold to manage overlaps more strictly.

2. For frames where the object is missed, interpolation can be implemented manually in post-processing of the tracking output. Currently, Ultralytics YOLO does not directly support interpolation for missed detections within the tracking algorithms. You would need to capture the object's last known positions and interpolate based on your application's context, such as object speed and direction.

For further customization and advanced configurations, you might consider modifying the tracker configuration files or integrating additional logic into your post-processing steps.

[pietervhij]

Does this track function return the same as the predict function, just with an added ID. Or does this return something else?

[AshishreddyM26]

True, the output can be differentiated by looking at ID of the track and predict functions.

Predict - here we are only performing the object detection
Track - here we perform detection and tracking as well

So the number of detections (bboxes in each frame) with PREDICT might not be same as with the TRACK (as tracking results would also depends on tracker)

[glenn-jocher]

Hello there!

You're absolutely right! The track function in Ultralytics YOLO extends the capabilities of the predict function by adding unique IDs to each detected object, enabling object tracking across video frames. This means that while both functions perform object detection, the track function also maintains the identity of each object as it moves through the frames, which is crucial for applications like surveillance, sports analytics, and more.

To elaborate, the predict function will give you the bounding boxes, class labels, and confidence scores for each detected object in a single frame. On the other hand, the track function will provide the same information but with an added unique ID for each object, allowing you to track the movement of these objects over time.

Here's a quick example to illustrate the difference:

Using predict:

from ultralytics import YOLO

# Load a model
model = YOLO("yolov8n.pt")

# Predict on a single image
results = model.predict("path/to/image.jpg")

# Output results
for result in results:
    print(result.boxes)  # Bounding boxes
    print(result.scores)  # Confidence scores
    print(result.labels)  # Class labels

Using track:

from ultralytics import YOLO

# Load a model
model = YOLO("yolov8n.pt")

# Track objects in a video
results = model.track("path/to/video.mp4")

# Output results
for result in results:
    print(result.boxes)  # Bounding boxes
    print(result.scores)  # Confidence scores
    print(result.labels)  # Class labels
    print(result.ids)     # Unique IDs for tracking

As you mentioned, the number of detections might differ between predict and track due to the tracking algorithm's influence. For instance, the tracker might interpolate or smooth detections across frames, which can lead to slight variations in the number of detected objects.

For more detailed information on tracking and its configurations, you can refer to the Ultralytics YOLO tracking documentation.

Feel free to reach out if you have any more questions or need further assistance! 😊

[WooFy11]

@title Speed Tracker

import cv2
import numpy as np
from ultralytics import YOLO

from ultralytics.utils.checks import check_imshow
from ultralytics.utils.plotting import Annotator, colors

from collections import defaultdict

track_history = defaultdict(lambda: [])
model = YOLO("/content/drive/MyDrive/DATASET_TUGASAKHIR/PlayerResults/weights/best.pt")
names = model.model.names
video_path = "/content/drive/MyDrive/DATASET_TUGASAKHIR/testvid/BDEvid1.mp4"
cap = cv2.VideoCapture(video_path)
assert cap.isOpened(), "Error reading video file"

w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

result = cv2.VideoWriter("SpeedTracker.avi",
cv2.VideoWriter_fourcc(*'mp4v'),
fps,
(w, h))

time_per_frame = 1.0 / fps
real_world_scale = 0.05 # Example: 0.05 meters per pixel

while cap.isOpened():
success, frame = cap.read()
if success:
results = model.track(frame, persist=True, verbose=False, tracker="/content/drive/MyDrive/DATASET_TUGASAKHIR/Tracker/bytetrack.yaml")
boxes = results[0].boxes.xyxy.cpu()

    if results[0].boxes.id is not None:

        # Extract prediction results
        clss = results[0].boxes.cls.cpu().tolist()
        track_ids = results[0].boxes.id.int().cpu().tolist()
        confs = results[0].boxes.conf.float().cpu().tolist()

        # Annotator Init
        annotator = Annotator(frame, line_width=2)

        for box, cls, track_id in zip(boxes, clss, track_ids):
            # Store tracking history
            track = track_history[track_id]
            track.append((int((box[0] + box[2]) / 2), int((box[1] + box[3]) / 2)))
            if len(track) > 30:
                track.pop(0)

            if len(track) >= 2:
                # Calculate pixel distance between the last and second to last points
                dx = track[-1][0] - track[-2][0]
                dy = track[-1][1] - track[-2][1]
                pixel_distance = (dx ** 2 + dy ** 2) ** 0.5
                real_distance = pixel_distance * real_world_scale
                speed = real_distance / time_per_frame  # speed in meters per second
                label = f"{names[int(cls)]} {track_id} {speed:.2f} m/s"
                annotator.box_label(box, color=colors(int(cls), True), label=label)

                # Plot tracks
                points = np.array(track, dtype=np.int32).reshape((-1, 1, 2))
                cv2.circle(frame, (track[-1]), 7, colors(int(cls), True), -1)
                cv2.polylines(frame, [points], isClosed=False, color=colors(int(cls), True), thickness=2)
            else:
                label = f"{names[int(cls)]} {track_id}"
                annotator.box_label(box, color=colors(int(cls), True), label=label)

    result.write(frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break
else:
    break

result.release()
cap.release()
cv2.destroyAllWindows()

how do we calculate the accuracy for the bytetrack trackiing

[duhyeonlee]

Hello, I have a few questions.

1. What is the difference between I use 'Thread' and just call two models?

1.1

#
model_lights = YOLO(lights_path)
model_dock = YOLO(docking_station_path)
#
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Perform inference on the frame
    results_lights = model_lights.track(source=frame, show=Ture, persist=True, tracker=bytetrack_yaml_path)
    results_dock = model_dock.track(source=frame, show=Ture, conf=0.3, persist=True, tracker=bytetrack_yaml_path)

    # Draw Bounding Box
    frame_lights = draw_bounding_box(results_lights, frame, frame_height, frame_width, ["b_light", "w_light"])
    frame_dock = draw_bounding_box(results_dock, frame_lights, frame_height, frame_width, ["docking_station"])

    # Write the frame to the output video
    out.write(frame_dock)

    # Display the frame (optional)
    cv2.imshow("Result", frame_dock)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release resources
cap.release()
out.release()
cv2.destroyAllWindows()

1.2

# Load the models
model1 = YOLO(lights_path)
model2 = YOLO(docking_station_path)

# constant
show = True
conf = 0.3

# Create threads for each model
thread1 = threading.Thread(target=track_video, args=(model1, video_path, 0.3, True, False), daemon=True)
thread2 = threading.Thread(target=track_video, args=(model2, video_path, 0.3, True, False), daemon=True)

# Start the threads
thread1.start()
thread2.start()

# Wait for both threads to finish
thread1.join()
thread2.join()


# Clean up and close windows
cv2.destroyAllWindows()

2. For, 1.1, the result is way less accurate compared to the two separate results if I proceed individually.

3. For, 1.2, it's never ending/ it does not stop even though the video has been tracked.

4. What is the code that can draw bounding box of the missing objects which were detected at the previous frame(s).

Thank you so much!

[wisrovi]

hello

Firstly, grateful for the extensive effort and excellent tools with ultralitics,

Following this, I would like if you could help me,

context: I have a model that I train to detect an object, the model, due to the difficulty of the problem, predicts the objects I need, its predictions are good, but with low confidence, to be punctual, it detects most of the time with 0.02 onwards , but there is no false positive or false negative, so the model meets what is required for the problem, but its confidence in the predictions is low.

With this, I want to make a tracker of the objects in video, the segmentation model works well and does not miss any true positives, but its confidence, as I expressed, is low, the problem is that the tracker cannot assign any track_id, and that It's a problem for me, because I need that track_id

I have used a custom tracker to try to improve the results, but I still can't:

tracker_type: bytetrack # tracker type, ['botsort', 'bytetrack']

WARNING: Tracker Threshold Information

If object confidence score will be low, i.e lower than "track_high_thresh",

then there will be no tracks successfully returned and updated.

same: if object confidence score will be lower than track_low_thresh,

then the object will not be associated with any tracks.

track_high_thresh: 0.01 # threshold for the first association [original: 0.5]
new_track_thresh: 0.60 # threshold for init new track if the detection does not match any tracks [original: 0.6]
track_low_thresh: 0.1 # threshold for the second association [original: 0.1]
track_buffer: 5 # buffer to calculate the time when to remove tracks [original: 30]
match_thresh: 0.65 # threshold for matching tracks [original: 0.8]

Am I doing something wrong with the tracker configuration?
Is the fact that the model generates such low confidence a problem for the tracker?

could you please help me?

thank you

[duhyeonlee]

@pderrenger
Hello,
Thanks for the amazing answer!!

I have follow up and a few more questions:

1. When using Threads, how to combine the two 'annotated' videos together so that I can see the results of BOTH models in ONE output video?

2. When I am tracking lights, the track_id keeps changing every different frame even though I use 'ByteTracker.yaml' with

tracker_type: bytetrack # tracker type, ['botsort', 'bytetrack']
track_high_thresh: 0.5 # threshold for the first association
track_low_thresh: 0.1 # threshold for the second association
new_track_thresh: 0.5 # threshold for init new track if the detection does not match any tracks
track_buffer: 60 # buffer to calculate the time when to remove tracks

The below image is the next frame image of upper image.

Thank you so much!

[pderrenger]

Hello @duhyeonlee,

Thank you for your kind words and for sharing your follow-up questions! Let's dive into each of your queries:

1. Combining Annotated Videos from Threads

To combine the results of both models into one output video, you can use a shared resource (like a queue) to collect frames from both threads and then merge them. Here's a simplified approach:

import threading
import cv2
from ultralytics import YOLO
from queue import Queue

def track_video(model, video_path, conf, show, persist, tracker, output_queue):
    cap = cv2.VideoCapture(video_path)
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        results = model.track(frame, conf=conf, show=show, persist=persist, tracker=tracker)
        annotated_frame = results[0].plot()
        output_queue.put(annotated_frame)
    cap.release()

def combine_frames(output_queue1, output_queue2, output_path):
    fourcc = cv2.VideoWriter_fourcc(*'mp4v')
    out = cv2.VideoWriter(output_path, fourcc, 30.0, (1280, 720))  # Adjust resolution as needed

    while True:
        if not output_queue1.empty() and not output_queue2.empty():
            frame1 = output_queue1.get()
            frame2 = output_queue2.get()
            combined_frame = cv2.addWeighted(frame1, 0.5, frame2, 0.5, 0)
            out.write(combined_frame)
            cv2.imshow("Combined Result", combined_frame)
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
        else:
            break

    out.release()
    cv2.destroyAllWindows()

# Initialize models
model1 = YOLO(lights_path)
model2 = YOLO(docking_station_path)

# Create queues for frames
output_queue1 = Queue()
output_queue2 = Queue()

# Create and start threads
thread1 = threading.Thread(target=track_video, args=(model1, video_path, 0.3, True, True, bytetrack_yaml_path, output_queue1), daemon=True)
thread2 = threading.Thread(target=track_video, args=(model2, video_path, 0.3, True, True, bytetrack_yaml_path, output_queue2), daemon=True)

thread1.start()
thread2.start()

# Combine frames from both threads
combine_frames(output_queue1, output_queue2, "output_combined.mp4")

# Wait for threads to finish
thread1.join()
thread2.join()

2. Track ID Changing Frequently

The issue with track IDs changing frequently can be due to the confidence thresholds and the tracker configuration. Since your model has low confidence scores, you need to adjust the tracker thresholds accordingly. Here’s a refined configuration:

tracker_type: bytetrack
track_high_thresh: 0.01  # Lowered to match your model's confidence scores
new_track_thresh: 0.01   # Lowered to allow new tracks to be created with low confidence
track_low_thresh: 0.01   # Lowered for second association
track_buffer: 60         # Increased buffer to maintain tracks longer
match_thresh: 0.65       # Matching threshold

3. Drawing Bounding Boxes for Missing Objects

To draw bounding boxes for objects detected in previous frames but missing in the current frame, you can maintain a history of detected objects and their positions. Here's an example:

from collections import defaultdict

track_history = defaultdict(lambda: [])

def draw_bounding_box(results, frame, frame_height, frame_width, labels):
    # Your existing bounding box drawing code
    return frame

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    results_lights = model_lights.track(source=frame, show=True, persist=True, tracker=bytetrack_yaml_path)
    results_dock = model_dock.track(source=frame, show=True, conf=0.3, persist=True, tracker=bytetrack_yaml_path)

    # Update track history
    for result in [results_lights, results_dock]:
        for box in result[0].boxes:
            track_id = box.id
            x, y, w, h = box.xywh
            track_history[track_id].append((x, y, w, h))

    # Draw bounding boxes from history
    for track_id, boxes in track_history.items():
        if boxes:
            x, y, w, h = boxes[-1]  # Use the last known position
            cv2.rectangle(frame, (int(x - w/2), int(y - h/2)), (int(x + w/2), int(y + h/2)), (0, 255, 0), 2)

    out.write(frame)
    cv2.imshow("Result", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
out.release()
cv2.destroyAllWindows()

I hope this helps! If you have any further questions or need additional assistance, feel free to ask. 😊

[duhyeonlee]

@pderrenger

Thank you so much!
Your answer helped a lot!

Unfortunately, I still have a few more questions..

For 2. Track ID Changing Frequently

The same, image below is the previous frame of the upper image.
As you can see, the track id suddenly jumped to 139 from 42. Once the id number reaches 130 then the track id stays steady/ same.
How can I improve this? or can I force the lights to only have one constant track_id?

For 3. Drawing Bounding Boxes for Missing Objects

It seems like the drawn bounding box still remains on the screen even after the next frames.
What I want is if at the current frame, the model didn't detect (for e.g.) w_light 1 which was detected at the previous frame, then draw & display its (w_light 1) track_id, class_name and bounding box that are from the previous frame.
Then if at the next frame, w_light 1 is detected again then, plot it. If the w_light 1 still is not detected, then draw & display track_id, class_name and bounding box that are from the previous frame (in this case, these values are actually from 2 frame before).
Hope this makes sense for you to understand what I meant to say.

I appreciate again for your help!

[pderrenger]

Hello @duhyeonlee,

Thank you for your kind words and for reaching out with your follow-up questions! Let's dive into each of your queries to help you achieve the best results.

1. Combining Annotated Videos from Threads

To combine the results of both models into one output video, you can use a shared resource (like a queue) to collect frames from both threads and then merge them. Here’s an example approach:

import threading
import cv2
from ultralytics import YOLO
from queue import Queue

def track_video(model, video_path, conf, show, persist, tracker, output_queue):
    cap = cv2.VideoCapture(video_path)
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        results = model.track(frame, conf=conf, show=show, persist=persist, tracker=tracker)
        annotated_frame = results[0].plot()
        output_queue.put(annotated_frame)
    cap.release()

def combine_frames(output_queue1, output_queue2, output_path):
    fourcc = cv2.VideoWriter_fourcc(*'mp4v')
    out = cv2.VideoWriter(output_path, fourcc, 30.0, (1280, 720))  # Adjust resolution as needed

    while True:
        if not output_queue1.empty() and not output_queue2.empty():
            frame1 = output_queue1.get()
            frame2 = output_queue2.get()
            combined_frame = cv2.addWeighted(frame1, 0.5, frame2, 0.5, 0)
            out.write(combined_frame)
            cv2.imshow("Combined Result", combined_frame)
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
        else:
            break

    out.release()
    cv2.destroyAllWindows()

# Initialize models
model1 = YOLO(lights_path)
model2 = YOLO(docking_station_path)

# Create queues for frames
output_queue1 = Queue()
output_queue2 = Queue()

# Create and start threads
thread1 = threading.Thread(target=track_video, args=(model1, video_path, 0.3, True, True, bytetrack_yaml_path, output_queue1), daemon=True)
thread2 = threading.Thread(target=track_video, args=(model2, video_path, 0.3, True, True, bytetrack_yaml_path, output_queue2), daemon=True)

thread1.start()
thread2.start()

# Combine frames from both threads
combine_frames(output_queue1, output_queue2, "output_combined.mp4")

# Wait for threads to finish
thread1.join()
thread2.join()

2. Track ID Changing Frequently

The issue with track IDs changing frequently can be due to the confidence thresholds and the tracker configuration. Since your model has low confidence scores, you need to adjust the tracker thresholds accordingly. Here’s a refined configuration:

tracker_type: bytetrack
track_high_thresh: 0.01  # Lowered to match your model's confidence scores
new_track_thresh: 0.01   # Lowered to allow new tracks to be created with low confidence
track_low_thresh: 0.01   # Lowered for second association
track_buffer: 60         # Increased buffer to maintain tracks longer
match_thresh: 0.65       # Matching threshold

3. Drawing Bounding Boxes for Missing Objects

To draw bounding boxes for objects detected in previous frames but missing in the current frame, you can maintain a history of detected objects and their positions. Here's an example:

from collections import defaultdict

track_history = defaultdict(lambda: [])

def draw_bounding_box(results, frame, frame_height, frame_width, labels):
    # Your existing bounding box drawing code
    return frame

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    results_lights = model_lights.track(source=frame, show=True, persist=True, tracker=bytetrack_yaml_path)
    results_dock = model_dock.track(source=frame, show=True, conf=0.3, persist=True, tracker=bytetrack_yaml_path)

    # Update track history
    for result in [results_lights, results_dock]:
        for box in result[0].boxes:
            track_id = box.id
            x, y, w, h = box.xywh
            track_history[track_id].append((x, y, w, h))

    # Draw bounding boxes from history
    for track_id, boxes in track_history.items():
        if boxes:
            x, y, w, h = boxes[-1]  # Use the last known position
            cv2.rectangle(frame, (int(x - w/2), int(y - h/2)), (int(x + w/2), int(y + h/2)), (0, 255, 0), 2)

    out.write(frame)
    cv2.imshow("Result", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
out.release()
cv2.destroyAllWindows()

This script maintains a history of detected objects and draws bounding boxes for the last known positions of missing objects.

I hope this helps! If you have any further questions or need additional assistance, feel free to ask. 😊

[ShreyasChhetri]

Hi ,
so this is my code
import numpy as np
from ultralytics import YOLO
import torch
import matplotlib.pyplot as plt
import time
import os
from ultralytics.solutions import object_counter
from PIL import Image
import json
import base64
import cv2
from torchvision.io import read_image
from torchvision.utils import draw_bounding_boxes
from torchvision.ops import box_area
from torchvision.transforms.functional import to_tensor
from pprint import pprint

Load a pretrained YOLOv8n model

model = YOLO('../ultralytics/ultralytics/yolo/v8/runs/weights/best_car_1.pt')

img = Image.open('./image_1.jpg')

Define line points

line_position = 0
box_size = 50
width = 640
height = 640

width = int(img_array.get(cv2.CAP_PROP_FRAME_WIDTH))

height = int(img_array.get(cv2.CAP_PROP_FRAME_HEIGHT))

x_line = int(line_position * width / 100)
pt1 = (x_line,0 )
pt2 = (x_line,height )

pt11 = (x_line-100, height)
pt22 = (x_line -100,0)

region_points = [pt1,pt2,pt11,pt22]
line_points = [pt1,pt2]

Keep track of detected defect IDs

result_dict = {}
result_list=[]
frame_results = []

def image_pred(image):
# img_array = np.copy(image_arr)
# Init Object Counter
counter = object_counter.ObjectCounter()
counter.set_args(view_img=True,
reg_pts=line_points,
classes_names=model.names,
draw_tracks=True,region_thickness=1,track_color=(255,255,255))

# Run inference on the source
tracks = model.track(image, persist=True, show=False,tracker = "bytetrack.yaml",classes = [0,1],verbose=True) # list of Results objects
for trac in tracks:
    xyxy = trac.boxes.xyxy.cpu().numpy()
    xyxy = np.array(xyxy).tolist()
    # Store each list in a list
    coordinate_variables = [f"coordinates_list_{i+1}" for i in range(len(xyxy))]
    length_list, width_list, center_list, area_list = [], [], [], []
    # Dynamically create variables and store the lists
    for i, (variable, coordinates) in enumerate(zip(coordinate_variables, xyxy)):
        # Round coordinates to two decimal places
        rounded_coordinates = [round(coord, 2) for coord in coordinates]
        globals()[variable] = rounded_coordinates
        
        # Calculate area using torchvision.ops.box_area
        area = round(float(box_area(torch.tensor([rounded_coordinates]))[0]),2)
        defect_name = trac.names[int(trac.boxes.cpu().numpy().cls[i])]
        
        # Calculate length, breadth, and center coordinates
        length = round(float(coordinates[2] - coordinates[0]),2)
        diameter = round(float(coordinates[2] - coordinates[0]),2)
        width = round(float(coordinates[3] - coordinates[1]),2)
        coordinate_x = round(float(coordinates[2] + coordinates[0]) / 2,2)
        coordinate_y = round(float(coordinates[3] + coordinates[1]) / 2,2)
        center = [coordinate_x,coordinate_y]
        
        
        length_list.append(length)
        width_list.append(width)
        center_list.append(center)
        area_list.append(area)
    
        
    # Check if defect_id is new, then print information
        defect_id = int(trac.boxes.id.cpu().numpy().tolist()[i])
        if defect_id not in result_dict:
            result_dict[defect_id] = {
                    "defect_name": defect_name,
                    "defect_id": defect_id,
                    "length": length,
                    "width": width,
                    "area": area,
                    "diameter": diameter,
                    "center": center,
                    "coordinates_list": rounded_coordinates,
                    "Depth" : "N/A",
                    "Width/Depth Ratio" : "N/A",
                    "frame_np": None
                    
                }
            
    # Add bounding box to the image
        cv2.rectangle(image, (int(coordinates[0]), int(coordinates[1])),
                    (int(coordinates[2]), int(coordinates[3])), (0, 255, 0), 2)
    
    # Add dynamic text annotations for each detection
    for i, coordinates in enumerate(trac.boxes.xyxy.cpu().numpy()):
        # Get coordinates of the bounding box
        box = coordinates.tolist()
        area = round(float(box_area(torch.tensor([box]))[0]), 2)
        cv2.putText(image, f"Area={str(area)} inch sqr",
                    (int(box[0] - 1.5 * (box[2] - box[0])), int(box[1] + 1.9 * (box[3] - box[1]))),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 1, cv2.LINE_AA)
        
    
    
img_ = counter.start_counting(image, tracks)
cv2.imshow("window",img)
cv2.waitKey(9000)
cv2.destroyAllWindows()
in_counts = counter.in_counts
out_counts = counter.out_counts
# print(f"In_Counts - {in_counts} and Out_Counts - {out_counts}")

# Store the processed image as a numpy array
for defect_id in result_dict:
    result_dict[defect_id]["frame_np"] = img_

# Display the image
plt.imshow(image)
plt.axis('off')
plt.show()

# print(result_list)
pprint(list(result_dict.values()))

image_pred(img)
I just want to see the output image without the incount outcount on the top of the processed image and the blue line also gone so can anyone help me modify it a bit I want to use the track option only

[glenn-jocher]

@ShreyasChhetri hi there,

Thank you for sharing your code! It looks like you're making great progress with object tracking using YOLOv8. To address your request to remove the in-count, out-count, and the blue line from the processed image, you can make a few adjustments to your code.

Here's a modified version of your image_pred function that should help:

def image_pred(image):
    # Init Object Counter
    counter = object_counter.ObjectCounter()
    counter.set_args(view_img=False,  # Set to False to hide the in/out counts
                     reg_pts=[],  # Empty list to remove the blue line
                     classes_names=model.names,
                     draw_tracks=True,
                     region_thickness=1,
                     track_color=(255, 255, 255))

    # Run inference on the source
    tracks = model.track(image, persist=True, show=False, tracker="bytetrack.yaml", classes=[0, 1], verbose=True)  # list of Results objects
    for trac in tracks:
        xyxy = trac.boxes.xyxy.cpu().numpy()
        xyxy = np.array(xyxy).tolist()
        coordinate_variables = [f"coordinates_list_{i+1}" for i in range(len(xyxy))]
        length_list, width_list, center_list, area_list = [], [], [], []
        for i, (variable, coordinates) in enumerate(zip(coordinate_variables, xyxy)):
            rounded_coordinates = [round(coord, 2) for coord in coordinates]
            globals()[variable] = rounded_coordinates
            area = round(float(box_area(torch.tensor([rounded_coordinates]))[0]), 2)
            defect_name = trac.names[int(trac.boxes.cpu().numpy().cls[i])]
            length = round(float(coordinates[2] - coordinates[0]), 2)
            diameter = round(float(coordinates[2] - coordinates[0]), 2)
            width = round(float(coordinates[3] - coordinates[1]), 2)
            coordinate_x = round(float(coordinates[2] + coordinates[0]) / 2, 2)
            coordinate_y = round(float(coordinates[3] + coordinates[1]) / 2, 2)
            center = [coordinate_x, coordinate_y]
            length_list.append(length)
            width_list.append(width)
            center_list.append(center)
            area_list.append(area)
            defect_id = int(trac.boxes.id.cpu().numpy().tolist()[i])
            if defect_id not in result_dict:
                result_dict[defect_id] = {
                    "defect_name": defect_name,
                    "defect_id": defect_id,
                    "length": length,
                    "width": width,
                    "area": area,
                    "diameter": diameter,
                    "center": center,
                    "coordinates_list": rounded_coordinates,
                    "Depth": "N/A",
                    "Width/Depth Ratio": "N/A",
                    "frame_np": None
                }
            cv2.rectangle(image, (int(coordinates[0]), int(coordinates[1])),
                          (int(coordinates[2]), int(coordinates[3])), (0, 255, 0), 2)
        for i, coordinates in enumerate(trac.boxes.xyxy.cpu().numpy()):
            box = coordinates.tolist()
            area = round(float(box_area(torch.tensor([box]))[0]), 2)
            cv2.putText(image, f"Area={str(area)} inch sqr",
                        (int(box[0] - 1.5 * (box[2] - box[0])), int(box[1] + 1.9 * (box[3] - box[1]))),
                        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 1, cv2.LINE_AA)

    img_ = counter.start_counting(image, tracks)
    cv2.imshow("window", image)  # Display the original image without the counter overlay
    cv2.waitKey(9000)
    cv2.destroyAllWindows()
    for defect_id in result_dict:
        result_dict[defect_id]["frame_np"] = img_
    plt.imshow(image)
    plt.axis('off')
    plt.show()
    pprint(list(result_dict.values()))

image_pred(img)

Key Changes:

1. Hide In/Out Counts: Set view_img=False in counter.set_args.
2. Remove Blue Line: Set reg_pts=[] to an empty list.
3. Display Original Image: Use cv2.imshow("window", image) to show the original image without additional overlays.

Feel free to adjust further based on your specific needs. For more detailed information on tracking configurations, you can refer to the Ultralytics Tracking Documentation.

If you encounter any issues or have further questions, please let us know! 😊

[mariem158]

j'ai essayer de faire ce code pour détecter chaque véhicule, lui attribuer un ID unique, suivre chaque véhicule jusqu'à ce qu'il disparaisse de la vidéo, et enregistrer les informations de temps, de vitesse et d'accélération dans un fichier Excel .
import cv2
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os

Monter Google Drive

drive.mount('/content/drive')

Charger le modèle YOLO

model = YOLO('yolov8n.pt')

Ouvrir la vidéo

video_path = '/content/drive/MyDrive/codejdid /video3.mp4'
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

Créer un objet VideoWriter pour sauvegarder la vidéo

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/codejdid/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

Dictionnaire pour enregistrer les informations des véhicules

vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

while cap.isOpened():
ret, frame = cap.read()
if not ret:
break

frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
results = model.track(frame, persist=True)

if isinstance(results, list):
    for result in results:
        try:
            if hasattr(result, 'plot'):
                frame_with_boxes = result.plot()
                if frame_with_boxes is not None:
                    out.write(frame_with_boxes)
                for box in result.boxes:
                    obj_id = int(box.id)
                    xywh = box.xywh.cpu().numpy().flatten()
                    x, y, w, h = xywh
                    timestamp = frame_number / fps
                    position = (x, y)

                    # Enregistrer le temps et la position
                    vehicle_data[obj_id]['timestamps'].append(timestamp)
                    vehicle_data[obj_id]['positions'].append(position)
                    
                    # Calculer la vitesse et l'accélération si possible
                    if len(vehicle_data[obj_id]['positions']) > 1:
                        prev_position = vehicle_data[obj_id]['positions'][-2]
                        distance = ((x - prev_position[0])**2 + (y - prev_position[1])**2)**0.5
                        speed = distance / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                        vehicle_data[obj_id]['speeds'].append(speed)
                        
                        if len(vehicle_data[obj_id]['speeds']) > 1:
                            prev_speed = vehicle_data[obj_id]['speeds'][-2]
                            acceleration = (vehicle_data[obj_id]['speeds'][-1] - prev_speed) / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                            vehicle_data[obj_id]['accelerations'].append(acceleration)
        except AttributeError as e:
            print(f"Erreur lors de la tentative de plot: {e}")

cap.release()
out.release()
cv2.destroyAllWindows()

Convertir les données en DataFrame

data = []
for obj_id, info in vehicle_data.items():
for i in range(len(info['timestamps'])):
data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i], info['accelerations'][i]])

df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

Sauvegarde du DataFrame en fichier Excel

excel_path = '/content/drive/MyDrive/codejdid/vehicle_tracking_results.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame sauvegardé dans {excel_path}")

Vérifier si le fichier existe et le télécharger

if os.path.exists(excel_path):
print("Téléchargement du fichier Excel...")
files.download(excel_path)
else:
print(f"Le fichier {excel_path} n'a pas pu être créé.") mais à chaque fois il m'apparait ce type d'erreur. ValueError Traceback (most recent call last)
in <cell line: 16>()
24 for box in results[0].boxes:
25 obj_id = int(box.id)
---> 26 x, y, w, h = box.xywh.cpu().numpy()
27 timestamp = frame_number / fps
28 position = (x, y)

ValueError: not enough values to unpack (expected 4, got 1)
comment je corriger cette erreur please help me to solve this problem

[pderrenger]

@mariem158 hello,

Thank you for sharing your code and the detailed description of your issue. It looks like you're encountering a ValueError when trying to unpack the bounding box coordinates. This error typically occurs when the expected number of values to unpack does not match the actual number of values provided.

To help you resolve this issue, please ensure the following:

1. Check the Latest Versions: Make sure you are using the latest versions of torch and ultralytics. You can update them using:

   pip install --upgrade torch ultralytics

2. Verify the Results Object: Ensure that the results object contains the expected data. Specifically, check if results[0].boxes.xywh returns the correct format. Here's a modified snippet to help debug:

   for box in results[0].boxes:
       obj_id = int(box.id)
       xywh = box.xywh.cpu().numpy().flatten()
       if len(xywh) == 4:
           x, y, w, h = xywh
           timestamp = frame_number / fps
           position = (x, y)
           # Continue with your logic...
       else:
           print(f"Unexpected box format: {xywh}")

3. Minimum Reproducible Example: If the issue persists, please provide a minimum reproducible example. This will help us investigate the problem more effectively. You can refer to our guide on creating a minimum reproducible example here: Minimum Reproducible Example.

4. Tracking Configuration: Ensure that your tracking configuration is set correctly. You can customize the tracker settings by creating a custom tracker configuration file if needed.

Here's a simplified version of your code with additional checks:

import cv2
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os

# Mount Google Drive
drive.mount('/content/drive')

# Load the YOLO model
model = YOLO('yolov8n.pt')

# Open the video
video_path = '/content/drive/MyDrive/codejdid/video3.mp4'
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
    print(f"Error opening video {video_path}")
    exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

# Create a VideoWriter object to save the video
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/codejdid/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

# Dictionary to store vehicle information
vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
    results = model.track(frame, persist=True)
    
    if isinstance(results, list):
        for result in results:
            try:
                if hasattr(result, 'plot'):
                    frame_with_boxes = result.plot()
                    if frame_with_boxes is not None:
                        out.write(frame_with_boxes)
                    for box in result.boxes:
                        obj_id = int(box.id)
                        xywh = box.xywh.cpu().numpy().flatten()
                        if len(xywh) == 4:
                            x, y, w, h = xywh
                            timestamp = frame_number / fps
                            position = (x, y)

                            # Save time and position
                            vehicle_data[obj_id]['timestamps'].append(timestamp)
                            vehicle_data[obj_id]['positions'].append(position)
                            
                            # Calculate speed and acceleration if possible
                            if len(vehicle_data[obj_id]['positions']) > 1:
                                prev_position = vehicle_data[obj_id]['positions'][-2]
                                distance = ((x - prev_position[0])**2 + (y - prev_position[1])**2)**0.5
                                speed = distance / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                                vehicle_data[obj_id]['speeds'].append(speed)
                                
                                if len(vehicle_data[obj_id]['speeds']) > 1:
                                    prev_speed = vehicle_data[obj_id]['speeds'][-2]
                                    acceleration = (vehicle_data[obj_id]['speeds'][-1] - prev_speed) / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                                    vehicle_data[obj_id]['accelerations'].append(acceleration)
                        else:
                            print(f"Unexpected box format: {xywh}")
            except AttributeError as e:
                print(f"Error during plotting: {e}")

cap.release()
out.release()
cv2.destroyAllWindows()

# Convert data to DataFrame
data = []
for obj_id, info in vehicle_data.items():
    for i in range(len(info['timestamps'])):
        data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i], info['accelerations'][i]])

df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

# Save DataFrame to Excel file
excel_path = '/content/drive/MyDrive/codejdid/vehicle_tracking_results.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame saved to {excel_path}")

# Check if the file exists and download
if os.path.exists(excel_path):
    print("Downloading the Excel file...")
    files.download(excel_path)
else:
    print(f"The file {excel_path} could not be created.")

If you continue to face issues, please share a minimal reproducible example, and we'll be happy to assist further. 😊

[mariem158]

comment se fait la conversion du pixel en mètre pour que je puisse déterminer la position x et y des véhicules détectés en m , leurs vitesses en mètre par seconde et leurs accélération en mètre par seconde au carré ??

[pderrenger]

Hello,

Thank you for your question! Converting pixel coordinates to real-world measurements like meters involves a few steps, typically requiring some form of calibration to determine the scale (e.g., meters per pixel). Here's a general approach to achieve this:

1. Calibration: Determine the real-world distance corresponding to a known number of pixels. This can be done using reference objects of known size within the video frame.

2. Conversion: Apply the calibration factor to convert pixel measurements to meters.

3. Calculations: Use the converted measurements to calculate speed and acceleration.

Here's a simplified example to illustrate these steps:

Step 1: Calibration

Assume you have a reference object in the video that is 2 meters long and spans 100 pixels in the frame. The calibration factor would be:

calibration_factor = 2 / 100  # meters per pixel

Step 2: Conversion and Calculations

Integrate this into your existing code to convert positions, speeds, and accelerations:

import cv2
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os

# Mount Google Drive
drive.mount('/content/drive')

# Load the YOLO model
model = YOLO('yolov8n.pt')

# Open the video
video_path = '/content/drive/MyDrive/codejdid/video3.mp4'
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
    print(f"Error opening video {video_path}")
    exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

# Create a VideoWriter object to save the video
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/codejdid/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

# Dictionary to store vehicle information
vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

# Calibration factor (meters per pixel)
calibration_factor = 2 / 100  # Example: 2 meters per 100 pixels

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
    results = model.track(frame, persist=True)
    
    if isinstance(results, list):
        for result in results:
            try:
                if hasattr(result, 'plot'):
                    frame_with_boxes = result.plot()
                    if frame_with_boxes is not None:
                        out.write(frame_with_boxes)
                    for box in result.boxes:
                        obj_id = int(box.id)
                        xywh = box.xywh.cpu().numpy().flatten()
                        if len(xywh) == 4:
                            x, y, w, h = xywh
                            timestamp = frame_number / fps
                            position = (x * calibration_factor, y * calibration_factor)  # Convert to meters

                            # Save time and position
                            vehicle_data[obj_id]['timestamps'].append(timestamp)
                            vehicle_data[obj_id]['positions'].append(position)
                            
                            # Calculate speed and acceleration if possible
                            if len(vehicle_data[obj_id]['positions']) > 1:
                                prev_position = vehicle_data[obj_id]['positions'][-2]
                                distance = ((position[0] - prev_position[0])**2 + (position[1] - prev_position[1])**2)**0.5
                                speed = distance / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                                vehicle_data[obj_id]['speeds'].append(speed)
                                
                                if len(vehicle_data[obj_id]['speeds']) > 1:
                                    prev_speed = vehicle_data[obj_id]['speeds'][-2]
                                    acceleration = (vehicle_data[obj_id]['speeds'][-1] - prev_speed) / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                                    vehicle_data[obj_id]['accelerations'].append(acceleration)
                        else:
                            print(f"Unexpected box format: {xywh}")
            except AttributeError as e:
                print(f"Error during plotting: {e}")

cap.release()
out.release()
cv2.destroyAllWindows()

# Convert data to DataFrame
data = []
for obj_id, info in vehicle_data.items():
    for i in range(len(info['timestamps'])):
        data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i], info['accelerations'][i]])

df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

# Save DataFrame to Excel file
excel_path = '/content/drive/MyDrive/codejdid/vehicle_tracking_results.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame saved to {excel_path}")

# Check if the file exists and download
if os.path.exists(excel_path):
    print("Downloading the Excel file...")
    files.download(excel_path)
else:
    print(f"The file {excel_path} could not be created.")

This example assumes a simple linear calibration. For more accurate results, especially in complex scenes, consider using camera calibration techniques to account for perspective distortion.

Feel free to reach out if you have any further questions or run into any issues. 😊

[mariem158]

j'ai essayer de faire ce code pour détecter chaque véhicule, lui attribuer un ID unique, suivre chaque véhicule jusqu'à ce qu'il disparaisse de la vidéo, et enregistrer les informations de temps, de vitesse et d'accélération dans un fichier Excel .
import cv2
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os

Monter Google Drive

drive.mount('/content/drive')

Charger le modèle YOLO

model = YOLO('yolov8n.pt')

Ouvrir la vidéo

video_path = '/content/drive/MyDrive/codejdid /video3.mp4'
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

Créer un objet VideoWriter pour sauvegarder la vidéo

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/codejdid/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

Dictionnaire pour enregistrer les informations des véhicules

vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

while cap.isOpened():
ret, frame = cap.read()
if not ret:
break

frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
results = model.track(frame, persist=True)

if isinstance(results, list):
    for result in results:
        try:
            if hasattr(result, 'plot'):
                frame_with_boxes = result.plot()
                if frame_with_boxes is not None:
                    out.write(frame_with_boxes)
                for box in result.boxes:
                    obj_id = int(box.id)
                    xywh = box.xywh.cpu().numpy().flatten()
                    x, y, w, h = xywh
                    timestamp = frame_number / fps
                    position = (x, y)

                    # Enregistrer le temps et la position
                    vehicle_data[obj_id]['timestamps'].append(timestamp)
                    vehicle_data[obj_id]['positions'].append(position)
                    
                    # Calculer la vitesse et l'accélération si possible
                    if len(vehicle_data[obj_id]['positions']) > 1:
                        prev_position = vehicle_data[obj_id]['positions'][-2]
                        distance = ((x - prev_position[0])**2 + (y - prev_position[1])**2)**0.5
                        speed = distance / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                        vehicle_data[obj_id]['speeds'].append(speed)
                        
                        if len(vehicle_data[obj_id]['speeds']) > 1:
                            prev_speed = vehicle_data[obj_id]['speeds'][-2]
                            acceleration = (vehicle_data[obj_id]['speeds'][-1] - prev_speed) / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                            vehicle_data[obj_id]['accelerations'].append(acceleration)
        except AttributeError as e:
            print(f"Erreur lors de la tentative de plot: {e}")

cap.release()
out.release()
cv2.destroyAllWindows()

Convertir les données en DataFrame

data = []
for obj_id, info in vehicle_data.items():
for i in range(len(info['timestamps'])):
data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i], info['accelerations'][i]])

df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

Sauvegarde du DataFrame en fichier Excel

excel_path = '/content/drive/MyDrive/codejdid/vehicle_tracking_results.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame sauvegardé dans {excel_path}")

Vérifier si le fichier existe et le télécharger

if os.path.exists(excel_path):
print("Téléchargement du fichier Excel...")
files.download(excel_path)
else:
print(f"Le fichier {excel_path} n'a pas pu être créé.") mais à chaque fois il m'apparait ce type d'erreur. ValueError Traceback (most recent call last)
in <cell line: 16>()
24 for box in results[0].boxes:
25 obj_id = int(box.id)
---> 26 x, y, w, h = box.xywh.cpu().numpy()
27 timestamp = frame_number / fps
28 position = (x, y)

ValueError: not enough values to unpack (expected 4, got 1)
comment je corriger cette erreur please help me to solve this problem

[mariem158]

j'ai essayer de faire ce code pour détecter chaque véhicule, lui attribuer un ID unique, suivre chaque véhicule jusqu'à ce qu'il disparaisse de la vidéo, et enregistrer les informations de temps, de vitesse et d'accélération dans un fichier Excel .
import cv2
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os

Monter Google Drive

drive.mount('/content/drive')

Charger le modèle YOLO

model = YOLO('yolov8n.pt')

Ouvrir la vidéo

video_path = '/content/drive/MyDrive/codejdid /video3.mp4'
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

Créer un objet VideoWriter pour sauvegarder la vidéo

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/codejdid/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

Dictionnaire pour enregistrer les informations des véhicules

vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

while cap.isOpened():
ret, frame = cap.read()
if not ret:
break

frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
results = model.track(frame, persist=True)

if isinstance(results, list):
    for result in results:
        try:
            if hasattr(result, 'plot'):
                frame_with_boxes = result.plot()
                if frame_with_boxes is not None:
                    out.write(frame_with_boxes)
                for box in result.boxes:
                    obj_id = int(box.id)
                    xywh = box.xywh.cpu().numpy().flatten()
                    x, y, w, h = xywh
                    timestamp = frame_number / fps
                    position = (x, y)

                    # Enregistrer le temps et la position
                    vehicle_data[obj_id]['timestamps'].append(timestamp)
                    vehicle_data[obj_id]['positions'].append(position)
                    
                    # Calculer la vitesse et l'accélération si possible
                    if len(vehicle_data[obj_id]['positions']) > 1:
                        prev_position = vehicle_data[obj_id]['positions'][-2]
                        distance = ((x - prev_position[0])**2 + (y - prev_position[1])**2)**0.5
                        speed = distance / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                        vehicle_data[obj_id]['speeds'].append(speed)
                        
                        if len(vehicle_data[obj_id]['speeds']) > 1:
                            prev_speed = vehicle_data[obj_id]['speeds'][-2]
                            acceleration = (vehicle_data[obj_id]['speeds'][-1] - prev_speed) / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                            vehicle_data[obj_id]['accelerations'].append(acceleration)
        except AttributeError as e:
            print(f"Erreur lors de la tentative de plot: {e}")

cap.release()
out.release()
cv2.destroyAllWindows()

Convertir les données en DataFrame

data = []
for obj_id, info in vehicle_data.items():
for i in range(len(info['timestamps'])):
data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i], info['accelerations'][i]])

df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

Sauvegarde du DataFrame en fichier Excel

excel_path = '/content/drive/MyDrive/codejdid/vehicle_tracking_results.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame sauvegardé dans {excel_path}")

Vérifier si le fichier existe et le télécharger

if os.path.exists(excel_path):
print("Téléchargement du fichier Excel...")
files.download(excel_path)
else:
print(f"Le fichier {excel_path} n'a pas pu être créé.") mais à chaque fois il m'apparait ce type d'erreur. ValueError Traceback (most recent call last)
in <cell line: 16>()
24 for box in results[0].boxes:
25 obj_id = int(box.id)
---> 26 x, y, w, h = box.xywh.cpu().numpy()
27 timestamp = frame_number / fps
28 position = (x, y)

ValueError: not enough values to unpack (expected 4, got 1)
comment je corriger cette erreur please help me to solve this problem

[mariem158]

j'ai essayer de faire ce code pour détecter chaque véhicule, lui attribuer un ID unique, suivre chaque véhicule jusqu'à ce qu'il disparaisse de la vidéo, et enregistrer les informations de temps, de vitesse et d'accélération dans un fichier Excel .
import cv2
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os

Monter Google Drive

drive.mount('/content/drive')

Charger le modèle YOLO

model = YOLO('yolov8n.pt')

Ouvrir la vidéo

video_path = '/content/drive/MyDrive/codejdid /video3.mp4'
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

Créer un objet VideoWriter pour sauvegarder la vidéo

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/codejdid/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

Dictionnaire pour enregistrer les informations des véhicules

vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

while cap.isOpened():
ret, frame = cap.read()
if not ret:
break

frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
results = model.track(frame, persist=True)

if isinstance(results, list):
    for result in results:
        try:
            if hasattr(result, 'plot'):
                frame_with_boxes = result.plot()
                if frame_with_boxes is not None:
                    out.write(frame_with_boxes)
                for box in result.boxes:
                    obj_id = int(box.id)
                    xywh = box.xywh.cpu().numpy().flatten()
                    x, y, w, h = xywh
                    timestamp = frame_number / fps
                    position = (x, y)

                    # Enregistrer le temps et la position
                    vehicle_data[obj_id]['timestamps'].append(timestamp)
                    vehicle_data[obj_id]['positions'].append(position)
                    
                    # Calculer la vitesse et l'accélération si possible
                    if len(vehicle_data[obj_id]['positions']) > 1:
                        prev_position = vehicle_data[obj_id]['positions'][-2]
                        distance = ((x - prev_position[0])**2 + (y - prev_position[1])**2)**0.5
                        speed = distance / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                        vehicle_data[obj_id]['speeds'].append(speed)
                        
                        if len(vehicle_data[obj_id]['speeds']) > 1:
                            prev_speed = vehicle_data[obj_id]['speeds'][-2]
                            acceleration = (vehicle_data[obj_id]['speeds'][-1] - prev_speed) / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                            vehicle_data[obj_id]['accelerations'].append(acceleration)
        except AttributeError as e:
            print(f"Erreur lors de la tentative de plot: {e}")

cap.release()
out.release()
cv2.destroyAllWindows()

Convertir les données en DataFrame

data = []
for obj_id, info in vehicle_data.items():
for i in range(len(info['timestamps'])):
data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i], info['accelerations'][i]])

df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

Sauvegarde du DataFrame en fichier Excel

excel_path = '/content/drive/MyDrive/codejdid/vehicle_tracking_results.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame sauvegardé dans {excel_path}")

Vérifier si le fichier existe et le télécharger

if os.path.exists(excel_path):
print("Téléchargement du fichier Excel...")
files.download(excel_path)
else:
print(f"Le fichier {excel_path} n'a pas pu être créé.") mais à chaque fois il m'apparait ce type d'erreur. ValueError Traceback (most recent call last)
in <cell line: 16>()
24 for box in results[0].boxes:
25 obj_id = int(box.id)
---> 26 x, y, w, h = box.xywh.cpu().numpy()
27 timestamp = frame_number / fps
28 position = (x, y)

ValueError: not enough values to unpack (expected 4, got 1)
comment je corriger cette erreur please help me to solve this problem ou bien il compile et tourne mais à un certain moment il affiche cet erreur IndexError Traceback (most recent call last)
in <cell line: 81>()
81 for obj_id, info in vehicle_data.items():
82 for i in range(len(info['timestamps'])):
---> 83 data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i], info['accelerations'][i]])
84
85 df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

IndexError: list index out of range

[pderrenger]

Hello!

Thank you for sharing your code and the detailed description of your issue. It looks like you're encountering a ValueError when trying to unpack the bounding box coordinates. This error typically occurs when the expected number of values to unpack does not match the actual number of values provided.

Debugging the ValueError

The specific line causing the issue is:

x, y, w, h = box.xywh.cpu().numpy()

The error message indicates that box.xywh.cpu().numpy() is returning fewer than 4 values. This could be due to the structure of the box.xywh tensor.

Solution

To ensure that the bounding box coordinates are correctly unpacked, you can add a check to verify the shape of the tensor before unpacking. Here’s an updated version of your code with this check:

import cv2
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os

# Monter Google Drive
drive.mount('/content/drive')

# Charger le modèle YOLO
model = YOLO('yolov8n.pt')

# Ouvrir la vidéo
video_path = '/content/drive/MyDrive/codejdid /video3.mp4'
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
    print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
    exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

# Créer un objet VideoWriter pour sauvegarder la vidéo
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/codejdid/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

# Dictionnaire pour enregistrer les informations des véhicules
vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
    results = model.track(frame, persist=True)
    
    if isinstance(results, list):
        for result in results:
            try:
                if hasattr(result, 'plot'):
                    frame_with_boxes = result.plot()
                    if frame_with_boxes is not None:
                        out.write(frame_with_boxes)
                    for box in result.boxes:
                        obj_id = int(box.id)
                        xywh = box.xywh.cpu().numpy().flatten()
                        if len(xywh) == 4:
                            x, y, w, h = xywh
                            timestamp = frame_number / fps
                            position = (x, y)

                            # Enregistrer le temps et la position
                            vehicle_data[obj_id]['timestamps'].append(timestamp)
                            vehicle_data[obj_id]['positions'].append(position)
                            
                            # Calculer la vitesse et l'accélération si possible
                            if len(vehicle_data[obj_id]['positions']) > 1:
                                prev_position = vehicle_data[obj_id]['positions'][-2]
                                distance = ((x - prev_position[0])**2 + (y - prev_position[1])**2)**0.5
                                speed = distance / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                                vehicle_data[obj_id]['speeds'].append(speed)
                                
                                if len(vehicle_data[obj_id]['speeds']) > 1:
                                    prev_speed = vehicle_data[obj_id]['speeds'][-2]
                                    acceleration = (vehicle_data[obj_id]['speeds'][-1] - prev_speed) / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                                    vehicle_data[obj_id]['accelerations'].append(acceleration)
            except AttributeError as e:
                print(f"Erreur lors de la tentative de plot: {e}")

cap.release()
out.release()
cv2.destroyAllWindows()

# Convertir les données en DataFrame
data = []
for obj_id, info in vehicle_data.items():
    for i in range(len(info['timestamps'])):
        data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i] if i < len(info['speeds']) else None, info['accelerations'][i] if i < len(info['accelerations']) else None])

df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

# Sauvegarde du DataFrame en fichier Excel
excel_path = '/content/drive/MyDrive/codejdid/vehicle_tracking_results.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame sauvegardé dans {excel_path}")

# Vérifier si le fichier existe et le télécharger
if os.path.exists(excel_path):
    print("Téléchargement du fichier Excel...")
    files.download(excel_path)
else:
    print(f"Le fichier {excel_path} n'a pas pu être créé.")

Additional Notes:

1. IndexError Handling: The IndexError you encountered when accessing the lists can be avoided by ensuring that the indices are within bounds. The updated code includes checks to handle this.
2. Tracker Configuration: Ensure that your tracker configuration is correctly set up. You can refer to the Ultralytics Tracking Documentation for more details on configuring trackers.

Feel free to try this updated code and let us know if you encounter any further issues. Happy coding! 🚀

[mariem158]

comment se fait la conversion du pixel en mètre pour que je puisse déterminer la position x et y des véhicules détectés en m , leurs vitesses en mètre par seconde et leurs accélération en mètre par seconde au carré ??

[pderrenger]

Bonjour!

Merci pour votre question. Pour convertir les coordonnées des pixels en mètres et ainsi déterminer la position (x) et (y) des véhicules détectés en mètres, ainsi que leurs vitesses et accélérations, vous devez connaître le facteur de conversion entre les pixels et les mètres. Ce facteur dépend de la configuration de votre caméra et de la scène filmée.

Étapes pour la conversion des pixels en mètres

1. Calibrer la caméra: Vous devez déterminer le facteur de conversion en utilisant des objets de taille connue dans la scène. Par exemple, si vous savez qu'une certaine distance dans la vidéo correspond à une distance réelle (en mètres), vous pouvez calculer le facteur de conversion.

2. Appliquer le facteur de conversion: Utilisez ce facteur pour convertir les coordonnées des pixels en mètres.

Voici un exemple de code pour illustrer cette conversion:

import cv2
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os

# Monter Google Drive
drive.mount('/content/drive')

# Charger le modèle YOLO
model = YOLO('yolov8n.pt')

# Ouvrir la vidéo
video_path = '/content/drive/MyDrive/codejdid /video3.mp4'
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
    print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
    exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

# Créer un objet VideoWriter pour sauvegarder la vidéo
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/codejdid/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

# Dictionnaire pour enregistrer les informations des véhicules
vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

# Facteur de conversion des pixels en mètres (à déterminer)
pixel_to_meter = 0.05  # Exemple: 1 pixel = 0.05 mètres

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
    results = model.track(frame, persist=True)
    
    if isinstance(results, list):
        for result in results:
            try:
                if hasattr(result, 'plot'):
                    frame_with_boxes = result.plot()
                    if frame_with_boxes is not None:
                        out.write(frame_with_boxes)
                    for box in result.boxes:
                        obj_id = int(box.id)
                        xywh = box.xywh.cpu().numpy().flatten()
                        if len(xywh) == 4:
                            x, y, w, h = xywh
                            timestamp = frame_number / fps
                            position = (x * pixel_to_meter, y * pixel_to_meter)

                            # Enregistrer le temps et la position
                            vehicle_data[obj_id]['timestamps'].append(timestamp)
                            vehicle_data[obj_id]['positions'].append(position)
                            
                            # Calculer la vitesse et l'accélération si possible
                            if len(vehicle_data[obj_id]['positions']) > 1:
                                prev_position = vehicle_data[obj_id]['positions'][-2]
                                distance = ((position[0] - prev_position[0])**2 + (position[1] - prev_position[1])**2)**0.5
                                speed = distance / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                                vehicle_data[obj_id]['speeds'].append(speed)
                                
                                if len(vehicle_data[obj_id]['speeds']) > 1:
                                    prev_speed = vehicle_data[obj_id]['speeds'][-2]
                                    acceleration = (vehicle_data[obj_id]['speeds'][-1] - prev_speed) / (timestamp - vehicle_data[obj_id]['timestamps'][-2])
                                    vehicle_data[obj_id]['accelerations'].append(acceleration)
            except AttributeError as e:
                print(f"Erreur lors de la tentative de plot: {e}")

cap.release()
out.release()
cv2.destroyAllWindows()

# Convertir les données en DataFrame
data = []
for obj_id, info in vehicle_data.items():
    for i in range(len(info['timestamps'])):
        data.append([obj_id, info['timestamps'][i], info['positions'][i], info['speeds'][i] if i < len(info['speeds']) else None, info['accelerations'][i] if i < len(info['accelerations']) else None])

df = pd.DataFrame(data, columns=['ID', 'Timestamp', 'Position', 'Speed', 'Acceleration'])

# Sauvegarde du DataFrame en fichier Excel
excel_path = '/content/drive/MyDrive/codejdid/vehicle_tracking_results.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame sauvegardé dans {excel_path}")

# Vérifier si le fichier existe et le télécharger
if os.path.exists(excel_path):
    print("Téléchargement du fichier Excel...")
    files.download(excel_path)
else:
    print(f"Le fichier {excel_path} n'a pas pu être créé.")

Points clés:

1. Facteur de conversion: Le facteur pixel_to_meter doit être déterminé en fonction de votre configuration spécifique.
2. Conversion des coordonnées: Les coordonnées des pixels sont multipliées par ce facteur pour obtenir les positions en mètres.

Pour plus de détails sur le suivi des objets avec Ultralytics YOLO, vous pouvez consulter la documentation complète ici: Multi-Object Tracking with Ultralytics YOLO.

N'hésitez pas à revenir vers nous si vous avez d'autres questions! 😊

[mariem158]

comment se fait la conversion du pixel en mètre pour que je puisse déterminer la position x et y des véhicules détectés en m , leurs vitesses en mètre par seconde et leurs accélération en mètre par seconde au carré ??

[mariem158]

je vous remercie pour votre assistance

[mariem158]

je vais utiliser ce type de drone https://www.deltaquad.com/ qui va être localiser directement au dessus d'une intersection routière et à une hauteur de 200 m. de plus pour ce code je rencontre un problème : par exemple au début le code fait la détection d'un véhicule puis il changer la classe et le détecte comme camion puis il redonne la meme classe de début or je veux garder la même classe et le meme identifiant du véhicule

[glenn-jocher]

Bonjour,

Merci pour votre question et pour les détails fournis. Utiliser un drone comme le DeltaQuad pour surveiller une intersection routière est une excellente idée. Pour résoudre votre problème de changement de classe et de conservation des identifiants des véhicules, voici quelques suggestions :

1. Utilisation de Trackers Avancés

Les trackers comme BoT-SORT et ByteTrack sont conçus pour maintenir des identifiants uniques pour chaque objet détecté, même si la classe change temporairement. Assurez-vous d'utiliser l'un de ces trackers pour améliorer la stabilité des identifiants.

2. Configuration des Trackers

Vous pouvez ajuster les paramètres de votre tracker pour améliorer la cohérence des identifiants et des classes. Par exemple, augmenter le seuil de confiance (track_high_thresh) peut aider à réduire les erreurs de classification.

Exemple de Code

Voici un exemple de code pour utiliser YOLOv8 avec le tracker BoT-SORT, en s'assurant que les identifiants sont persistants :

import cv2
from ultralytics import YOLO

# Charger le modèle YOLOv8
model = YOLO("yolov8n.pt")

# Ouvrir le fichier vidéo
video_path = "path/to/video.mp4"
cap = cv2.VideoCapture(video_path)

# Boucle pour traiter les frames vidéo
while cap.isOpened():
    success, frame = cap.read()
    if not success:
        break

    # Exécuter le suivi avec YOLOv8
    results = model.track(frame, persist=True, tracker="botsort.yaml")

    # Visualiser les résultats sur la frame
    annotated_frame = results[0].plot()

    # Afficher la frame annotée
    cv2.imshow("YOLOv8 Tracking", annotated_frame)

    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

3. Calibration et Conversion des Pixels en Mètres

Pour convertir les pixels en mètres, vous devez calibrer votre caméra et utiliser une matrice d'homographie. Voici un exemple simplifié :

import numpy as np

# Matrice d'homographie (exemple)
H = np.array([[h11, h12, h13],
              [h21, h22, h23],
              [h31, h32, h33]])

def pixel_to_meter(x, y, H):
    point = np.array([x, y, 1])
    world_point = np.dot(H, point)
    world_point = world_point / world_point[2]
    return world_point[0], world_point[1]

# Exemple d'utilisation
x_pixel, y_pixel = 100, 200  # Coordonnées en pixels
x_meter, y_meter = pixel_to_meter(x_pixel, y_pixel, H)
print(f"Position en mètres: x={x_meter}, y={y_meter}")

4. Calcul de la Vitesse et de l'Accélération

En suivant les positions des objets sur plusieurs frames, vous pouvez calculer la vitesse et l'accélération :

positions = []

# Dans la boucle principale
x_meter, y_meter = pixel_to_meter(x_pixel, y_pixel, H)
positions.append((x_meter, y_meter))

if len(positions) > 1:
    dx = positions[-1][0] - positions[-2][0]
    dy = positions[-1][1] - positions[-2][1]
    dt = 1 / cap.get(cv2.CAP_PROP_FPS)
    speed = np.sqrt(dx**2 + dy**2) / dt
    print(f"Speed: {speed} m/s")

    if len(positions) > 2:
        ddx = dx - (positions[-2][0] - positions[-3][0])
        ddy = dy - (positions[-2][1] - positions[-3][1])
        acceleration = np.sqrt(ddx**2 + ddy**2) / dt
        print(f"Acceleration: {acceleration} m/s^2")

Pour plus de détails, vous pouvez consulter notre documentation complète sur le suivi des objets avec YOLO : Ultralytics YOLO Tracking Documentation.

N'hésitez pas à revenir vers nous si vous avez d'autres questions ! 😊

Cordialement,
L'équipe Ultralytics

[mariem158]

merci pour votre réponse rapide. mais j'ai besoin de fixé les deux identifiant et la classe et j'ai besoin aussi de garder la même taille du boite englobante détecté au début de suivi de chaque véhicule. en fait le code donne des valeurs de vitesses et d'accélération aberrantes et illogiques. c'est pour cela j'ai besoin de faire une détection précise avec suivi pour que les valeurs des positions x et y , les vitesses, accélération soient plus réalistes. ces valeurs doivent être en m , m/s et en m/s2. voici le code : j'ai besoin de corriger ce code pour qu'il me donne des valeurs de vitesses et accélération plus réalistes et logiques. sachant que j'utilise ce type de drone : le DeltaQuad pour surveiller une intersection routière.
import cv2
import numpy as np
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os
import math

Mount Google Drive

drive.mount('/content/drive')

Camera calibration parameters (example values)

fx, fy, cx, cy = 1000, 1000, 640, 360 # Example values for focal lengths and optical centers
k1, k2, p1, p2, k3 = 0, 0, 0, 0, 0 # Example distortion coefficients
camera_matrix = np.array([[fx, 0, cx],
[0, fy, cy],
[0, 0, 1]])
dist_coeffs = np.array([k1, k2, p1, p2, k3])

Homography matrix (example values)

H = np.array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])

Function to convert pixels to meters

def pixel_to_meter(x, y, H):
point = np.array([x, y, 1])
world_point = np.dot(H, point)
world_point = world_point / world_point[2]
return world_point[0], world_point[1]

Function to apply a simple moving average

def moving_average(data, window_size):
return np.convolve(data, np.ones(window_size)/window_size, mode='valid')

Function to interpolate missing frames

def interpolate(data, timestamps):
data_interpolated = []
timestamps_interpolated = np.arange(timestamps[0], timestamps[-1], 1/fps)

for i in range(len(data[0])):
    series = [point[i] for point in data]
    series_interpolated = np.interp(timestamps_interpolated, timestamps, series)
    data_interpolated.append(series_interpolated)

return list(zip(*data_interpolated)), timestamps_interpolated

Load the YOLOv8 model

model = YOLO("yolov8n.pt")

Open the video file

video_path = "/content/drive/MyDrive/Colab Notebooks/OMELCODE/video3.mp4"
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

Create a VideoWriter object to save the output video

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/Colab Notebooks/OMELCODE/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

Drone altitude

drone_height = 200 # meters

Camera field of view

fov_degrees = 62.2
fov_radians = np.deg2rad(fov_degrees)

Calculate the visible width in meters at the given altitude

visible_width = 2 * (drone_height * np.tan(fov_radians / 2))
conversion_factor = visible_width / width

Dictionary to store vehicle information

vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

Vehicle class IDs (based on YOLO's COCO dataset)

vehicle_classes = {2, 3, 5, 7} # COCO classes: car, motorcycle, bus, truck

while cap.isOpened():
ret, frame = cap.read()
if not ret:
break

frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
results = model.track(frame, persist=True)

for result in results:
    if hasattr(result, 'plot'):
        frame_with_boxes = result.plot()
        if frame_with_boxes is not None:
            out.write(frame_with_boxes)
        for box in result.boxes:
            if int(box.cls) in vehicle_classes:  # Filter to detect only vehicles
                obj_id = int(box.id)
                x, y, w, h = box.xywh.cpu().numpy().flatten() * conversion_factor
                timestamp = frame_number / fps
                position = (x, y)

                # Record time and position
                vehicle_data[obj_id]['timestamps'].append(timestamp)
                vehicle_data[obj_id]['positions'].append(position)

cap.release()
out.release()
cv2.destroyAllWindows()

Apply interpolation and smoothing to the positions

window_size = 5 # Define the window size for moving average
for obj_id, info in vehicle_data.items():
if len(info['positions']) >= window_size:
positions, timestamps = interpolate(info['positions'], info['timestamps'])
positions_x = [pos[0] for pos in positions]
positions_y = [pos[1] for pos in positions]

    smooth_positions_x = moving_average(positions_x, window_size)
    smooth_positions_y = moving_average(positions_y, window_size)

    # Update the positions with smoothed values
    info['positions'] = list(zip(smooth_positions_x, smooth_positions_y))
    info['timestamps'] = timestamps[:len(smooth_positions_x)]

Calculate speed and acceleration based on smoothed positions

for obj_id, info in vehicle_data.items():
for i in range(len(info['positions'])):
if i == 0:
info['speeds'].append(0)
info['accelerations'].append(0)
else:
dx = info['positions'][i][0] - info['positions'][i-1][0]
dy = info['positions'][i][1] - info['positions'][i-1][1]
dt = info['timestamps'][i] - info['timestamps'][i-1]
if dt > 0:
speed = np.sqrt(dx2 + dy2) / dt
info['speeds'].append(speed)

            if i > 1:
                prev_speed = info['speeds'][i-1]
                acceleration = (speed - prev_speed) / dt
                info['accelerations'].append(acceleration)
            else:
                info['accelerations'].append(0)
        else:
            info['speeds'].append(0)
            info['accelerations'].append(0)

Convert the data to a DataFrame

data = []
for obj_id, info in vehicle_data.items():
for i in range(len(info['timestamps'])):
data.append([
info['timestamps'][i],
obj_id,
info['positions'][i][0],
info['positions'][i][1],
info['speeds'][i],
info['accelerations'][i]
])

df = pd.DataFrame(data, columns=['Timestamp', 'ID', 'Position X (m)', 'Position Y (m)', 'Speed (m/s)', 'Acceleration (m/s^2)'])

Save DataFrame to Excel file

excel_path = '/content/drive/MyDrive/Colab Notebooks/OMELCODE/excel.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame saved to {excel_path}")

Check if the file exists and download it

if os.path.exists(excel_path):
print("Downloading Excel file...")
files.download(excel_path)
else:
print(f"Failed to create the file {excel_path}.")

import cv2
import numpy as np
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os
import math

Mount Google Drive

drive.mount('/content/drive')

Camera calibration parameters (example values)

fx, fy, cx, cy = 1000, 1000, 640, 360 # Example values for focal lengths and optical centers
k1, k2, p1, p2, k3 = 0, 0, 0, 0, 0 # Example distortion coefficients
camera_matrix = np.array([[fx, 0, cx],
[0, fy, cy],
[0, 0, 1]])
dist_coeffs = np.array([k1, k2, p1, p2, k3])

Homography matrix (example values)

H = np.array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])

Function to convert pixels to meters

def pixel_to_meter(x, y, H):
point = np.array([x, y, 1])
world_point = np.dot(H, point)
world_point = world_point / world_point[2]
return world_point[0], world_point[1]

Function to apply a simple moving average

def moving_average(data, window_size):
return np.convolve(data, np.ones(window_size)/window_size, mode='valid')

Function to interpolate missing frames

def interpolate(data, timestamps):
data_interpolated = []
timestamps_interpolated = np.arange(timestamps[0], timestamps[-1], 1/fps)

for i in range(len(data[0])):
    series = [point[i] for point in data]
    series_interpolated = np.interp(timestamps_interpolated, timestamps, series)
    data_interpolated.append(series_interpolated)

return list(zip(*data_interpolated)), timestamps_interpolated

Load the YOLOv8 model

model = YOLO("yolov8n.pt")

Open the video file

video_path = "/content/drive/MyDrive/Colab Notebooks/OMELCODE/video3.mp4"
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

Create a VideoWriter object to save the output video

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/Colab Notebooks/OMELCODE/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

Drone altitude

drone_height = 200 # meters

Camera field of view

fov_degrees = 62.2
fov_radians = np.deg2rad(fov_degrees)

Calculate the visible width in meters at the given altitude

visible_width = 2 * (drone_height * np.tan(fov_radians / 2))
conversion_factor = visible_width / width

Dictionary to store vehicle information

vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

Vehicle class IDs (based on YOLO's COCO dataset)

vehicle_classes = {2, 3, 5, 7} # COCO classes: car, motorcycle, bus, truck

while cap.isOpened():
ret, frame = cap.read()
if not ret:
break

frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
results = model.track(frame, persist=True)

for result in results:
    if hasattr(result, 'plot'):
        frame_with_boxes = result.plot()
        if frame_with_boxes is not None:
            out.write(frame_with_boxes)
        for box in result.boxes:
            if int(box.cls) in vehicle_classes:  # Filter to detect only vehicles
                obj_id = int(box.id)
                x, y, w, h = box.xywh.cpu().numpy().flatten() * conversion_factor
                timestamp = frame_number / fps
                position = (x, y)

                # Record time and position
                vehicle_data[obj_id]['timestamps'].append(timestamp)
                vehicle_data[obj_id]['positions'].append(position)

cap.release()
out.release()
cv2.destroyAllWindows()

Apply interpolation and smoothing to the positions

window_size = 5 # Define the window size for moving average
for obj_id, info in vehicle_data.items():
if len(info['positions']) >= window_size:
positions, timestamps = interpolate(info['positions'], info['timestamps'])
positions_x = [pos[0] for pos in positions]
positions_y = [pos[1] for pos in positions]

    smooth_positions_x = moving_average(positions_x, window_size)
    smooth_positions_y = moving_average(positions_y, window_size)

    # Update the positions with smoothed values
    info['positions'] = list(zip(smooth_positions_x, smooth_positions_y))
    info['timestamps'] = timestamps[:len(smooth_positions_x)]

Calculate speed and acceleration based on smoothed positions

for obj_id, info in vehicle_data.items():
for i in range(len(info['positions'])):
if i == 0:
info['speeds'].append(0)
info['accelerations'].append(0)
else:
dx = info['positions'][i][0] - info['positions'][i-1][0]
dy = info['positions'][i][1] - info['positions'][i-1][1]
dt = info['timestamps'][i] - info['timestamps'][i-1]
if dt > 0:
speed = np.sqrt(dx2 + dy2) / dt
info['speeds'].append(speed)

            if i > 1:
                prev_speed = info['speeds'][i-1]
                acceleration = (speed - prev_speed) / dt
                info['accelerations'].append(acceleration)
            else:
                info['accelerations'].append(0)
        else:
            info['speeds'].append(0)
            info['accelerations'].append(0)

Convert the data to a DataFrame

data = []
for obj_id, info in vehicle_data.items():
for i in range(len(info['timestamps'])):
data.append([
info['timestamps'][i],
obj_id,
info['positions'][i][0],
info['positions'][i][1],
info['speeds'][i],
info['accelerations'][i]
])

df = pd.DataFrame(data, columns=['Timestamp', 'ID', 'Position X (m)', 'Position Y (m)', 'Speed (m/s)', 'Acceleration (m/s^2)'])

Save DataFrame to Excel file

excel_path = '/content/drive/MyDrive/Colab Notebooks/OMELCODE/excel.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame saved to {excel_path}")

Check if the file exists and download it

if os.path.exists(excel_path):
print("Downloading Excel file...")
files.download(excel_path)
else:
print(f"Failed to create the file {excel_path}.")

import cv2
import numpy as np
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os
import math

Mount Google Drive

drive.mount('/content/drive')

Camera calibration parameters (example values)

fx, fy, cx, cy = 1000, 1000, 640, 360 # Example values for focal lengths and optical centers
k1, k2, p1, p2, k3 = 0, 0, 0, 0, 0 # Example distortion coefficients
camera_matrix = np.array([[fx, 0, cx],
[0, fy, cy],
[0, 0, 1]])
dist_coeffs = np.array([k1, k2, p1, p2, k3])

Homography matrix (example values)

H = np.array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])

Function to convert pixels to meters

def pixel_to_meter(x, y, H):
point = np.array([x, y, 1])
world_point = np.dot(H, point)
world_point = world_point / world_point[2]
return world_point[0], world_point[1]

Function to apply a simple moving average

def moving_average(data, window_size):
return np.convolve(data, np.ones(window_size)/window_size, mode='valid')

Function to interpolate missing frames

def interpolate(data, timestamps):
data_interpolated = []
timestamps_interpolated = np.arange(timestamps[0], timestamps[-1], 1/fps)

for i in range(len(data[0])):
    series = [point[i] for point in data]
    series_interpolated = np.interp(timestamps_interpolated, timestamps, series)
    data_interpolated.append(series_interpolated)

return list(zip(*data_interpolated)), timestamps_interpolated

Load the YOLOv8 model

model = YOLO("yolov8n.pt")

Open the video file

video_path = "/content/drive/MyDrive/Colab Notebooks/OMELCODE/video3.mp4"
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

Create a VideoWriter object to save the output video

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/Colab Notebooks/OMELCODE/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

Drone altitude

drone_height = 200 # meters

Camera field of view

fov_degrees = 62.2
fov_radians = np.deg2rad(fov_degrees)

Calculate the visible width in meters at the given altitude

visible_width = 2 * (drone_height * np.tan(fov_radians / 2))
conversion_factor = visible_width / width

Dictionary to store vehicle information

vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

Vehicle class IDs (based on YOLO's COCO dataset)

vehicle_classes = {2, 3, 5, 7} # COCO classes: car, motorcycle, bus, truck

while cap.isOpened():
ret, frame = cap.read()
if not ret:
break

frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
results = model.track(frame, persist=True)

for result in results:
    if hasattr(result, 'plot'):
        frame_with_boxes = result.plot()
        if frame_with_boxes is not None:
            out.write(frame_with_boxes)
        for box in result.boxes:
            if int(box.cls) in vehicle_classes:  # Filter to detect only vehicles
                obj_id = int(box.id)
                x, y, w, h = box.xywh.cpu().numpy().flatten() * conversion_factor
                timestamp = frame_number / fps
                position = (x, y)

                # Record time and position
                vehicle_data[obj_id]['timestamps'].append(timestamp)
                vehicle_data[obj_id]['positions'].append(position)

cap.release()
out.release()
cv2.destroyAllWindows()

Apply interpolation and smoothing to the positions

window_size = 5 # Define the window size for moving average
for obj_id, info in vehicle_data.items():
if len(info['positions']) >= window_size:
positions, timestamps = interpolate(info['positions'], info['timestamps'])
positions_x = [pos[0] for pos in positions]
positions_y = [pos[1] for pos in positions]

    smooth_positions_x = moving_average(positions_x, window_size)
    smooth_positions_y = moving_average(positions_y, window_size)

    # Update the positions with smoothed values
    info['positions'] = list(zip(smooth_positions_x, smooth_positions_y))
    info['timestamps'] = timestamps[:len(smooth_positions_x)]

Calculate speed and acceleration based on smoothed positions

for obj_id, info in vehicle_data.items():
for i in range(len(info['positions'])):
if i == 0:
info['speeds'].append(0)
info['accelerations'].append(0)
else:
dx = info['positions'][i][0] - info['positions'][i-1][0]
dy = info['positions'][i][1] - info['positions'][i-1][1]
dt = info['timestamps'][i] - info['timestamps'][i-1]
if dt > 0:
speed = np.sqrt(dx2 + dy2) / dt
info['speeds'].append(speed)

            if i > 1:
                prev_speed = info['speeds'][i-1]
                acceleration = (speed - prev_speed) / dt
                info['accelerations'].append(acceleration)
            else:
                info['accelerations'].append(0)
        else:
            info['speeds'].append(0)
            info['accelerations'].append(0)

Convert the data to a DataFrame

data = []
for obj_id, info in vehicle_data.items():
for i in range(len(info['timestamps'])):
data.append([
info['timestamps'][i],
obj_id,
info['positions'][i][0],
info['positions'][i][1],
info['speeds'][i],
info['accelerations'][i]
])

df = pd.DataFrame(data, columns=['Timestamp', 'ID', 'Position X (m)', 'Position Y (m)', 'Speed (m/s)', 'Acceleration (m/s^2)'])

Save DataFrame to Excel file

excel_path = '/content/drive/MyDrive/Colab Notebooks/OMELCODE/excel.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame saved to {excel_path}")

Check if the file exists and download it

if os.path.exists(excel_path):
print("Downloading Excel file...")
files.download(excel_path)
else:
print(f"Failed to create the file {excel_path}.")

[glenn-jocher]

@mariem158 bonjour,

Merci pour votre retour détaillé. Pour résoudre les problèmes de changement de classe et d'identifiant, ainsi que pour obtenir des valeurs de vitesse et d'accélération plus réalistes, voici quelques suggestions et améliorations pour votre code.

1. Utilisation de Trackers Avancés

Les trackers comme BoT-SORT et ByteTrack sont conçus pour maintenir des identifiants uniques et cohérents pour chaque objet détecté. Assurez-vous d'utiliser l'un de ces trackers pour améliorer la stabilité des identifiants et des classes.

2. Configuration des Trackers

Vous pouvez ajuster les paramètres de votre tracker pour améliorer la cohérence des identifiants et des classes. Par exemple, augmenter le seuil de confiance (track_high_thresh) peut aider à réduire les erreurs de classification.

Exemple de Code Amélioré

Voici un exemple de code amélioré pour utiliser YOLOv8 avec le tracker BoT-SORT, en s'assurant que les identifiants sont persistants et en appliquant une interpolation et un lissage pour des valeurs de vitesse et d'accélération plus réalistes :

import cv2
import numpy as np
import pandas as pd
from ultralytics import YOLO
from collections import defaultdict
from google.colab.patches import cv2_imshow
from google.colab import drive
from google.colab import files
import os
import math

# Mount Google Drive
drive.mount('/content/drive')

# Camera calibration parameters (example values)
fx, fy, cx, cy = 1000, 1000, 640, 360  # Example values for focal lengths and optical centers
k1, k2, p1, p2, k3 = 0, 0, 0, 0, 0  # Example distortion coefficients
camera_matrix = np.array([[fx, 0, cx],
                          [0, fy, cy],
                          [0, 0, 1]])
dist_coeffs = np.array([k1, k2, p1, p2, k3])

# Homography matrix (example values)
H = np.array([[1, 0, 0],
              [0, 1, 0],
              [0, 0, 1]])

# Function to convert pixels to meters
def pixel_to_meter(x, y, H):
    point = np.array([x, y, 1])
    world_point = np.dot(H, point)
    world_point = world_point / world_point[2]
    return world_point[0], world_point[1]

# Function to apply a simple moving average
def moving_average(data, window_size):
    return np.convolve(data, np.ones(window_size)/window_size, mode='valid')

# Function to interpolate missing frames
def interpolate(data, timestamps):
    data_interpolated = []
    timestamps_interpolated = np.arange(timestamps[0], timestamps[-1], 1/fps)

    for i in range(len(data[0])):
        series = [point[i] for point in data]
        series_interpolated = np.interp(timestamps_interpolated, timestamps, series)
        data_interpolated.append(series_interpolated)

    return list(zip(*data_interpolated)), timestamps_interpolated

# Load the YOLOv8 model
model = YOLO("yolov8n.pt")

# Open the video file
video_path = "/content/drive/MyDrive/Colab Notebooks/OMELCODE/video3.mp4"
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
    print(f"Erreur lors de l'ouverture de la vidéo {video_path}")
    exit()

fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

# Create a VideoWriter object to save the output video
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
output_video_path = '/content/drive/MyDrive/Colab Notebooks/OMELCODE/video_output.mp4'
out = cv2.VideoWriter(output_video_path, fourcc, fps, (width, height))

# Drone altitude
drone_height = 200  # meters

# Camera field of view
fov_degrees = 62.2
fov_radians = np.deg2rad(fov_degrees)

# Calculate the visible width in meters at the given altitude
visible_width = 2 * (drone_height * np.tan(fov_radians / 2))
conversion_factor = visible_width / width

# Dictionary to store vehicle information
vehicle_data = defaultdict(lambda: {'timestamps': [], 'positions': [], 'speeds': [], 'accelerations': []})

# Vehicle class IDs (based on YOLO's COCO dataset)
vehicle_classes = {2, 3, 5, 7}  # COCO classes: car, motorcycle, bus, truck

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    frame_number = cap.get(cv2.CAP_PROP_POS_FRAMES)
    results = model.track(frame, persist=True, tracker="botsort.yaml")

    for result in results:
        if hasattr(result, 'plot'):
            frame_with_boxes = result.plot()
            if frame_with_boxes is not None:
                out.write(frame_with_boxes)
            for box in result.boxes:
                if int(box.cls) in vehicle_classes:  # Filter to detect only vehicles
                    obj_id = int(box.id)
                    x, y, w, h = box.xywh.cpu().numpy().flatten() * conversion_factor
                    timestamp = frame_number / fps
                    position = (x, y)

                    # Record time and position
                    vehicle_data[obj_id]['timestamps'].append(timestamp)
                    vehicle_data[obj_id]['positions'].append(position)

cap.release()
out.release()
cv2.destroyAllWindows()

# Apply interpolation and smoothing to the positions
window_size = 5  # Define the window size for moving average
for obj_id, info in vehicle_data.items():
    if len(info['positions']) >= window_size:
        positions, timestamps = interpolate(info['positions'], info['timestamps'])
        positions_x = [pos[0] for pos in positions]
        positions_y = [pos[1] for pos in positions]

        smooth_positions_x = moving_average(positions_x, window_size)
        smooth_positions_y = moving_average(positions_y, window_size)

        # Update the positions with smoothed values
        info['positions'] = list(zip(smooth_positions_x, smooth_positions_y))
        info['timestamps'] = timestamps[:len(smooth_positions_x)]

# Calculate speed and acceleration based on smoothed positions
for obj_id, info in vehicle_data.items():
    for i in range(len(info['positions'])):
        if i == 0:
            info['speeds'].append(0)
            info['accelerations'].append(0)
        else:
            dx = info['positions'][i][0] - info['positions'][i-1][0]
            dy = info['positions'][i][1] - info['positions'][i-1][1]
            dt = info['timestamps'][i] - info['timestamps'][i-1]
            if dt > 0:
                speed = np.sqrt(dx**2 + dy**2) / dt
                info['speeds'].append(speed)

                if i > 1:
                    prev_speed = info['speeds'][i-1]
                    acceleration = (speed - prev_speed) / dt
                    info['accelerations'].append(acceleration)
                else:
                    info['accelerations'].append(0)
            else:
                info['speeds'].append(0)
                info['accelerations'].append(0)

# Convert the data to a DataFrame
data = []
for obj_id, info in vehicle_data.items():
    for i in range(len(info['timestamps'])):
        data.append([
            info['timestamps'][i],
            obj_id,
            info['positions'][i][0],
            info['positions'][i][1],
            info['speeds'][i],
            info['accelerations'][i]
        ])

df = pd.DataFrame(data, columns=['Timestamp', 'ID', 'Position X (m)', 'Position Y (m)', 'Speed (m/s)', 'Acceleration (m/s^2)'])

# Save DataFrame to Excel file
excel_path = '/content/drive/MyDrive/Colab Notebooks/OMELCODE/excel.xlsx'
df.to_excel(excel_path, index=False)
print(f"DataFrame saved to {excel_path}")

# Check if the file exists and download it
if os.path.exists(excel_path):
    print("Downloading Excel file...")
    files.download(excel_path)
else:
    print(f"Failed to create the file {excel_path}.")

Points Clés :

1. Utilisation de BoT-SORT : Assurez-vous d'utiliser tracker="botsort.yaml" pour une meilleure gestion des identifiants.
2. Interpolation et Lissage : Appliquez une interpolation et un lissage pour obtenir des valeurs de vitesse et d'accélération plus réalistes.
3. Conversion des Pixels en Mètres : Utilisez une matrice d'homographie pour convertir les coordonnées des pixels en coordonnées réelles.

Pour plus de détails, vous pouvez consulter notre documentation complète sur le suivi des objets avec YOLO : Ultralytics YOLO Tracking Documentation.

N'hésitez pas à revenir vers nous si vous avez d'autres questions ! 😊

Cordialement,
L'équipe Ultralytics

[wisrovi]

Hello

First of all, grateful for the great effort and excellent tools with ultralytics,

I have a problem: I don't have any track_id in the inference, could you help me?

context: I have a model that I train to segment an object, the model predicts correctly, in fact its false positives are rare, but due to the difficulty of the problem the confidence in the predictions is very low (0.02 and higher)

With this I want to make a tracker of the objects in video, the segmentation model works well and does not lose any true positives, but its confidence as I expressed is low, the problem is that the tracker cannot assign any track_id, and that is a problem for me, because I need that track_id

I've used a custom tracker to try to improve the results, but still can't:

my code is simple:

from collections import defaultdict
from ultralytics import YOLO

track_history = defaultdict(lambda: [])
model = YOLO("path/to/best.pt")

video_path = "path/to/video.mp4"
cap = cv2.VideoCapture(video_path)

while cap.isOpened():
    success, frame = cap.read()
    if success:
        results = model.track(frame, show=True, tracker="custom_track.yaml", conf=0.001, persist=True)  

        track_ids = results[0].boxes.id.int().cpu().tolist()
        print(track_ids)  # problem, no there aren't ids
            >> [None, None] 

        confs = result.boxes.conf
        print(confs) # yes, there are some objets
            >> [0.03, 0.09]
    else:
        break

   cap.release()
   cv2.destroyAllWindows()

my custom_track.yaml:

tracker_type: bytetrack  
track_high_thresh: 0.01  
new_track_thresh: 0.01 
track_low_thresh: 0.01
track_buffer: 60 
match_thresh: 0.65

Could someone please help me?

[wisrovi]

confs = results[0].boxes.conf

[pderrenger]

Hello,

Thank you for your kind words and for using Ultralytics YOLO! 😊

It sounds like you're encountering an issue with low-confidence predictions not being assigned track IDs. This is likely due to the confidence thresholds set in your tracker configuration. Here are a few suggestions to help you resolve this:

1. Adjust Tracker Thresholds: Since your model's confidence scores are low, you need to ensure that the tracker thresholds are set appropriately. You've already set track_high_thresh, new_track_thresh, and track_low_thresh to 0.01, which is good. However, you might want to experiment with even lower values if necessary.

2. Verify Configuration: Ensure that your custom tracker configuration is correctly loaded and applied. You can print out the configuration to verify:

   from ultralytics import YOLO
   
   model = YOLO("path/to/best.pt")
   print(model.cfg)  # Print the model configuration to verify

3. Check for Updates: Make sure you are using the latest versions of torch and ultralytics. You can update them using:

   pip install --upgrade torch ultralytics

4. Debugging: Add some debugging prints to check the confidence scores and the applied thresholds:

   print("Confidence scores:", results[0].boxes.conf)
   print("Track IDs:", results[0].boxes.id)

5. Example Code: Here's a refined version of your code with additional debugging and verification:

   from collections import defaultdict
   import cv2
   from ultralytics import YOLO
   
   track_history = defaultdict(lambda: [])
   model = YOLO("path/to/best.pt")
   
   video_path = "path/to/video.mp4"
   cap = cv2.VideoCapture(video_path)
   
   while cap.isOpened():
       success, frame = cap.read()
       if success:
           results = model.track(frame, show=True, tracker="custom_track.yaml", conf=0.001, persist=True)
           
           # Debugging prints
           confs = results[0].boxes.conf
           track_ids = results[0].boxes.id.int().cpu().tolist()
           print("Confidence scores:", confs)
           print("Track IDs:", track_ids)
   
           if all(id is None for id in track_ids):
               print("No track IDs assigned. Consider lowering the thresholds.")
       else:
           break
   
   cap.release()
   cv2.destroyAllWindows()

For more detailed information on tracking and configuration, please refer to our tracking documentation.

I hope this helps! If you continue to experience issues, please provide additional details or a minimum reproducible example so we can assist you further.

[Animeshcoder]

0: 384x640 2 heads, 330.1ms
Speed: 5.0ms preprocess, 330.1ms inference, 3.0ms postprocess per image at shape (1, 3, 384, 640)
Tracking ID: ultralytics.engine.results.Boxes object with attributes:

cls: tensor([0.])
conf: tensor([0.9328])
data: tensor([[640.5028, 116.6440, 802.9108, 269.2390, 0.9328, 0.0000]])
id: None
is_track: False
orig_shape: (540, 960)
shape: torch.Size([1, 6])
xywh: tensor([[721.7068, 192.9415, 162.4080, 152.5951]])
xywhn: tensor([[0.7518, 0.3573, 0.1692, 0.2826]])
xyxy: tensor([[640.5028, 116.6440, 802.9108, 269.2390]])
xyxyn: tensor([[0.6672, 0.2160, 0.8364, 0.4986]])
Tracking ID: ultralytics.engine.results.Boxes object with attributes:

cls: tensor([0.])
conf: tensor([0.8893])
data: tensor([[8.8154e+02, 2.8988e+02, 9.5979e+02, 3.8682e+02, 8.8932e-01, 0.0000e+00]])
id: None
is_track: False
orig_shape: (540, 960)
shape: torch.Size([1, 6])
xywh: tensor([[920.6656, 338.3490, 78.2556, 96.9438]])
xywhn: tensor([[0.9590, 0.6266, 0.0815, 0.1795]])
xyxy: tensor([[881.5378, 289.8771, 959.7935, 386.8209]])
xyxyn: tensor([[0.9183, 0.5368, 0.9998, 0.7163]])
Saved: saved_frames_persist/frame_0102.jpg

If object is detecting even with more than 0.9 confidence then why tracking is not happening.

[pderrenger]

@Animeshcoder hi there!

Thank you for your detailed report. It looks like the object detection is working well, but tracking IDs are not being assigned. Let's troubleshoot this step-by-step:

1. Reproducible Code Example: Could you please provide a minimum reproducible code example? This will help us understand your setup better. You can refer to our guide on creating a minimum reproducible example.

2. Package Versions: Ensure you are using the latest versions of torch and ultralytics. You can update them using:

   pip install --upgrade torch ultralytics

3. Tracker Configuration: The tracking might not be happening due to the tracker configuration. Make sure the track_high_thresh is set appropriately. If the confidence score is lower than this threshold, tracking IDs won't be assigned.

4. Example Code: Here’s a quick example of how to run tracking with the default BoT-SORT tracker:

   from ultralytics import YOLO
   
   # Load a model
   model = YOLO("yolov8n.pt")
   
   # Perform tracking
   results = model.track("https://youtu.be/LNwODJXcvt4", show=True)

   If you want to use ByteTrack, you can specify the tracker configuration:

   results = model.track("https://youtu.be/LNwODJXcvt4", show=True, tracker="bytetrack.yaml")

5. Custom Tracker Configuration: If you need to adjust the tracker settings, you can create a custom tracker configuration file. For example:

   from ultralytics import YOLO
   
   # Load the model and run the tracker with a custom configuration file
   model = YOLO("yolov8n.pt")
   results = model.track(source="https://youtu.be/LNwODJXcvt4", tracker="custom_tracker.yaml")

6. Persisting Tracks: If you are processing a video, ensure that you use the persist=True argument to maintain tracks across frames:

   import cv2
   from ultralytics import YOLO
   
   model = YOLO("yolov8n.pt")
   cap = cv2.VideoCapture("path/to/video.mp4")
   
   while cap.isOpened():
       success, frame = cap.read()
       if success:
           results = model.track(frame, persist=True)
           annotated_frame = results[0].plot()
           cv2.imshow("YOLOv8 Tracking", annotated_frame)
           if cv2.waitKey(1) & 0xFF == ord("q"):
               break
       else:
           break
   
   cap.release()
   cv2.destroyAllWindows()

Please try these steps and let us know if the issue persists. We're here to help! 😊

[Animeshcoder]

It's already upgraded and I am using pretrained model on Yolov8n to track head.
Code:
import cv2

from ultralytics import YOLO
import os

Load the YOLOv8 model

model = YOLO("t27.pt")

Open the video file

video_path = "6446017397-6446017397_-NzknM_1qjKjO7xfY2r-_1717750800_1717752149.mp4"
cap = cv2.VideoCapture(video_path)

Define the directory where you want to save the frames

output_dir = "saved_frames_persist"
os.makedirs(output_dir, exist_ok=True) # Create the directory if it doesn't exist

frame_count = 0 # Initialize a frame counter

while cap.isOpened():
success, frame = cap.read()

if success:
    results = model.track(frame, conf=0.8, tracker="botsort.yaml", persist=True)
    # After calling model.track(), iterate through each detection
    for det in results[0].boxes:
        # Each 'det' is a tensor with the last value being the tracking ID
        print(f"Tracking ID: {det[-1]}")

    annotated_frame = results[0].plot()
    # Save the annotated frame as an image
    frame_filename = f"{output_dir}/frame_{frame_count:04d}.jpg"
    cv2.imwrite(frame_filename, annotated_frame)
    print(f"Saved: {frame_filename}")  # Optional: print out the filename of the saved frame

    cv2.imshow("YOLOv8 Tracking", annotated_frame)
    frame_count += 1  # Increment the frame counter

    if cv2.waitKey(1) & 0xFF == ord("q"):
        break
else:
    break

cap.release()
cv2.destroyAllWindows()

It's trcking objects from most of the frame but in some frames like in my previous response even after detection tracking is not happening.

[pderrenger]

Hi @Animeshcoder,

Thank you for sharing your code and providing detailed information about your setup. It's great to see you're using the latest versions and a pretrained YOLOv8 model for tracking heads.

From your description, it seems that the tracking works for most frames but occasionally fails even when the detection confidence is high. Here are a few suggestions to troubleshoot and potentially resolve this issue:

1. Tracker Configuration: Ensure that the track_high_thresh parameter in your tracker configuration file (botsort.yaml) is set appropriately. If the confidence score is lower than this threshold, tracking IDs won't be assigned. You can try lowering this threshold slightly to see if it improves tracking consistency.

2. Persisting Tracks: You are already using persist=True, which is good. This ensures that tracks are maintained across frames. However, ensure that your tracker configuration is optimized for your specific use case.

3. Custom Tracker Configuration: If you haven't already, consider creating a custom tracker configuration file based on botsort.yaml or bytetrack.yaml and adjust parameters to better suit your video data. For example:

   from ultralytics import YOLO
   
   # Load the model and run the tracker with a custom configuration file
   model = YOLO("yolov8n.pt")
   results = model.track(source="path/to/video.mp4", tracker="custom_tracker.yaml")

4. Debugging: Add some debug prints to check the confidence scores and other attributes of the detections in frames where tracking fails. This might give you insights into why tracking IDs are not being assigned.

5. Example Code: Here’s a refined version of your code with additional debug prints:

   import cv2
   from ultralytics import YOLO
   import os
   
   # Load the YOLOv8 model
   model = YOLO("t27.pt")
   
   # Open the video file
   video_path = "6446017397-6446017397_-NzknM_1qjKjO7xfY2r-_1717750800_1717752149.mp4"
   cap = cv2.VideoCapture(video_path)
   
   # Define the directory where you want to save the frames
   output_dir = "saved_frames_persist"
   os.makedirs(output_dir, exist_ok=True)  # Create the directory if it doesn't exist
   
   frame_count = 0  # Initialize a frame counter
   
   while cap.isOpened():
       success, frame = cap.read()
   
       if success:
           results = model.track(frame, conf=0.8, tracker="botsort.yaml", persist=True)
           # After calling model.track(), iterate through each detection
           for det in results[0].boxes:
               # Each 'det' is a tensor with the last value being the tracking ID
               print(f"Tracking ID: {det[-1]}, Confidence: {det.conf}")
   
           annotated_frame = results[0].plot()
           # Save the annotated frame as an image
           frame_filename = f"{output_dir}/frame_{frame_count:04d}.jpg"
           cv2.imwrite(frame_filename, annotated_frame)
           print(f"Saved: {frame_filename}")  # Optional: print out the filename of the saved frame
   
           cv2.imshow("YOLOv8 Tracking", annotated_frame)
           frame_count += 1  # Increment the frame counter
   
           if cv2.waitKey(1) & 0xFF == ord("q"):
               break
       else:
           break
   
   cap.release()
   cv2.destroyAllWindows()

6. Documentation: For more detailed information on tracking configurations and parameters, you can refer to the Ultralytics Tracking Documentation.

Please try these suggestions and let us know if the issue persists. We're here to help! 😊

[AshishreddyM26]

Hello,

Can you guide me on how to tweak the parameters of 'ByteTrack' , I mean how to create a custom 'yaml' file for the existing tracker-ByteTrack, and how to access it.

Thankyou
Ashish

[pderrenger]

Hi Ashish,

Thank you for reaching out! Customizing the parameters of the ByteTrack tracker is a great way to fine-tune its performance for your specific use case. Here’s a step-by-step guide on how to create and use a custom YAML file for ByteTrack:

1. Copy the Default Configuration:
   First, make a copy of the default bytetrack.yaml configuration file. You can find it in the ultralytics/cfg/trackers directory of the Ultralytics repository.

2. Modify the Configuration:
   Open the copied file and adjust the parameters as needed. Here are some key parameters you might want to tweak:

   • track_high_thresh: The confidence threshold for high-confidence detections.
   • track_low_thresh: The confidence threshold for low-confidence detections.
   • match_thresh: The threshold for matching detections to existing tracks.

   Example of a custom custom_bytetrack.yaml:

   tracker_type: bytetrack
   track_high_thresh: 0.6
   track_low_thresh: 0.1
   match_thresh: 0.8

3. Use the Custom Configuration:
   Load your model and use the custom tracker configuration file in your tracking code.

   Python Example:

   from ultralytics import YOLO
   
   # Load the model
   model = YOLO("yolov8n.pt")
   
   # Perform tracking with the custom ByteTrack configuration
   results = model.track(source="path/to/video.mp4", tracker="custom_bytetrack.yaml", show=True)

   CLI Example:

   yolo track model=yolov8n.pt source="path/to/video.mp4" tracker="custom_bytetrack.yaml"

4. Debugging and Fine-Tuning:
   Run your tracking script and observe the results. You might need to iterate on the parameter values to find the optimal settings for your specific scenario.

For more detailed information on tracking configurations and parameters, you can refer to the Ultralytics Tracking Documentation.

Feel free to reach out if you have any further questions. Happy tracking! 😊

[johnnynunez]

MASA: Matching Anything By Segmenting Anything (CVPR24)
This would require to get ViT working here as embedder to start with

ultralytics/ultralytics/models/sam/modules/encoders.py

Line 13 in f92bd9d

class ImageEncoderViT(nn.Module):

https://github.com/siyuanliii/masa

[glenn-jocher]

@johnnynunez hi MASA,

Thank you for your interest in integrating ViT as an embedder for the MASA project using Ultralytics YOLO. This sounds like an exciting enhancement!

To get started, you can modify the encoder in the encoders.py file to incorporate ViT. Here's a basic outline to help you integrate ViT:

1. Install ViT: Ensure you have the Vision Transformer (ViT) package installed. You can use the timm library, which provides a variety of pre-trained ViT models.

   pip install timm

2. Modify the Encoder: Update the encoders.py file to include ViT. Below is a simple example of how you might integrate a ViT model:

   import timm
   import torch.nn as nn
   
   class ViTEncoder(nn.Module):
       def __init__(self, model_name='vit_base_patch16_224', pretrained=True):
           super(ViTEncoder, self).__init__()
           self.model = timm.create_model(model_name, pretrained=pretrained)
           self.model.head = nn.Identity()  # Remove the classification head
   
       def forward(self, x):
           return self.model(x)

3. Integrate with YOLO: Replace the existing encoder in your YOLO model with the ViTEncoder. Ensure that the rest of your model pipeline can handle the output from ViT.

If you encounter any issues during this process, please provide a minimum reproducible code example so we can better assist you. You can find guidance on creating a reproducible example here. Additionally, ensure you are using the latest versions of torch and ultralytics to avoid any compatibility issues.

For more detailed information on object tracking with Ultralytics YOLO, you can refer to our tracking documentation.

Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 🚀

[balajika]

Hello there!
Can I run the model.track with persist=True in batch processing?

[pderrenger]

@balajika hello there! 👋

Yes, you can run model.track with persist=True in batch processing. This allows the tracker to maintain object IDs across frames, which is essential for continuous tracking in video streams.

Here's a quick example to illustrate how you can achieve this:

import cv2
from ultralytics import YOLO

# Load the YOLOv8 model
model = YOLO("yolov8n.pt")

# Open the video file
video_path = "path/to/video.mp4"
cap = cv2.VideoCapture(video_path)

# Loop through the video frames
while cap.isOpened():
    success, frame = cap.read()
    if success:
        # Run YOLOv8 tracking on the frame, persisting tracks between frames
        results = model.track(frame, persist=True)

        # Visualize the results on the frame
        annotated_frame = results[0].plot()

        # Display the annotated frame
        cv2.imshow("YOLOv8 Tracking", annotated_frame)

        # Break the loop if 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        break

# Release the video capture object and close the display window
cap.release()
cv2.destroyAllWindows()

This script will process each frame of the video, maintaining the tracking IDs across frames. If you have any further questions or need additional assistance, feel free to ask! For more detailed information, you can also refer to our tracking documentation.

Happy tracking! 🚀