Track - Ultralytics YOLOv8 Docs

[giscus[bot]]

Track - Ultralytics YOLOv8 Docs

Explore YOLOv8n-based object tracking with Ultralytics' BoT-SORT and ByteTrack. Learn configuration, usage, and customization tips.

https://docs.ultralytics.com/modes/track/

[Crear12]

How to input continuous frames one by one into model.track and annotate the results? Something like:

while True:
   frame = Q.get()
   if frame is not None:
      results = model.track(source=frame, tracker='custom_tracker.yaml')
      annotated_frame = results[0].plot()
   else:
      break

[convurt]

I've found that the tracker doesn't work great for many objects that are fast moving and cross each other. I've tried various settings and techniques over months. I've ended up having to determine when id's change on a timeline and change id's to continue following the object. That means I'm keeping up with ids and seconds in a csv and then remapping them post processing. I've not found a better way, if anyone has, please share. Hopefully some progress will be made with ReID soon and make tracking a strong component of YOLO.

[pderrenger]

Hey there! Thanks for sharing your experience with the tracking module and your innovative approach to handling ID changes for fast-moving, crossing objects. 🌟 Your method of managing IDs via a CSV and post-processing sounds like a smart workaround for specific scenarios.

Regarding your hope for improvements with ReID, we're continuously exploring ways to enhance our tracking capabilities, including the potential integration of robust ReID features. Keep an eye out for updates!

If anyone else has found alternative methods or improvements, feel free to chime in. Community insights are invaluable. 🚀 Happy tracking!

[RunsenXu]

Hi @pderrenger ,

Could you please give me some hints on my two questions? Thank you very much!

[pderrenger]

Hi there! 👋

Sure, I'd be happy to help with your questions about UltraLytics YOLO tracking! 🚀

For tracking a specific bounding box across frames, you'll want to initialize tracking with that particular detection. Currently, the track functionality processes full images and tracks all detected objects. For focusing on a specific object through initial bounding boxes, custom handling post-detection to filter for your object of interest would be needed.

Regarding resetting the tracker without reinitializing the model, there's no direct function like model.track.reset() available. A workaround could involve toggling the persist parameter or managing tracked objects' IDs at your application level to "reset" tracking contextually.

Your use case is very insightful! If there are specific attributes or methods you believe could enhance UltraLytics YOLO's functionality, consider sharing this feedback or contributing on GitHub. Your input is invaluable for growing and improving the tool. 🌟

[Hemilibeatriz]

I've found that the tracker doesn't work great for many objects that are fast moving and cross each other. I've tried various settings and techniques over months. I've ended up having to determine when id's change on a timeline and change id's to continue following the object. That means I'm keeping up with ids and seconds in a csv and then remapping them post processing. I've not found a better way, if anyone has, please share. Hopefully some progress will be made with ReID soon and make tracking a strong component of YOLO.

Hi @convurt

Can you explain what logic you used to improve tracking? I'm also having very poor results with a camera positioned on the ceiling of the room, with a top-down view.

[Crear12]

Memory leakage...? 2-min-video consumed 40GB.

[ashish-reddy26]

Thanks so much for the clarification and link for the MOTchallenge evaluator.

[pderrenger]

@AshishreddyM26 hey Ashish,

I'm thrilled to hear the MOTChallenge evaluator link was helpful! 🎉

Regarding your questions:

1. Tracking with persist=True: Yes, when you use persist=True, the default tracker (BoT-SORT) or the one you specify (like ByteTrack or BoT-SORT) maintains object IDs across frames. This is essential for consistent tracking in videos, especially with dense objects and a moving camera.

2. BoT-SORT vs. ByteTrack: The difference in performance between BoT-SORT and ByteTrack could be due to how each algorithm handles occlusions, interactions, and the dynamic nature of your scenes. BoT-SORT might be better at maintaining IDs because of its design to handle such complexities more effectively.

3. DeepSORT Implementation: Currently, Ultralytics YOLOv8 doesn't provide a direct .yaml configuration for DeepSORT. For integrating DeepSORT, you might need to manually combine its tracking logic with the detections from YOLOv8.

4. Generating Videos & Metrics: Your code for generating videos looks good! For metrics, after tracking, you'd typically compare your tracker's output against ground truth annotations using evaluation tools like the MOTChallenge evaluator. This comparison can give you insights into your tracker's performance in terms of accuracy, ID switches, etc.

For a simpler code snippet to generate videos, your approach is correct. Just ensure you're plotting (results[0].plot()) and writing (out.write()) each frame to your video writer within your loop.

I hope this clarifies your doubts! Keep experimenting, and don't hesitate to reach out if you have more questions. Happy tracking! 🚀

[RunsenXu]

Hi @pderrenger ,

If I pass an array of images to model.track and set persist=False, will the tracking maintain object IDs across frames?

Thank you very much!

[RunsenXu]

From my experiment, if I pass a list of [PIL.Image] to model.track and set persist=False, the tracking IDs will not maintain across frames, which means each image in the list is tracked separately (so if only one target object is in the image, the tracking ID will always be 1).

But it's interesting that if directly passing a mp4 video and persist=False, the tracking IDs will maintain across frames.

[pderrenger]

Hey there! 🌟

Indeed, when passing an array of images to model.track with persist=False, each image is considered independently, resetting the tracking IDs for each frame. Consequently, for standalone images, tracking IDs might not be consistently maintained across frames.

However, when processing a video directly (e.g., an MP4 file) with persist=False, tracking IDs can still maintain across frames due to the sequential nature and inherent temporal context in videos, which might not be explicitly present when processing an array of images.

For more advanced or specialized tracking needs like implementing DeepSORT, adapting the tracker to your specific requirements might involve more manual integration since the desired .yaml for DeepSORT is not directly provided.

Your observations on BoT-SORT and ByteTrack's performance are insightful. Different tracking algorithms have varying strengths and weaknesses depending on the scene complexity, object density, and camera movement. BoT-SORT's design might be better suited to your video's characteristics, leading to more stable tracking IDs compared to ByteTrack.

Here's a simplified example for generating a video output, similar to your detailed script:

results = model.track(source="video23.mp4", conf=0.5, iou=0.5, show=True, tracker='botsort.yaml')

To generate a video from this code, your approach of using cv2.VideoWriter as you've done in your detailed script is correct. Just ensure to grab frames from results[0].plot() for visualization and write them to your output video file.

Keep up the great work navigating these complex tracking scenarios! If you've got more questions or need further clarifications, feel free to reach out. Happy coding! 😊

[holy-chicken]

face_result = tracker.track(frame, tracker="bytetrack.yaml", iou=0.5, verbose=False, classes=0, persist=True)
for index, box in enumerate(face_result[0].boxes):
tracker_id = tracker.predictor.trackers[0].tracked_stracks[index].track_id

[holy-chicken]

box.id seems incorrect, not the tracking ID.Please try my code

[log901108]

what is tracker in your code?
how can I get tracker id from code below(which is Jie-Xio gave):
import cv2
from ultralytics import YOLO

cap = cv2.VideoCapture("/path/to/video")
model = YOLO("yolov8n.pt")
while True:
ret, frame = cap.read()
if not ret:
break
results = model.track(frame, persist=True)
boxes = results[0].boxes.xyxy.cpu().numpy().astype(int)
ids = results[0].boxes.id.cpu().numpy().astype(int)

for box, id in zip(boxes, ids):
    cv2.rectangle(frame, (box[0], box[1]), (box[2], box[3]), (0, 255, 0), 2)
    cv2.putText(
        frame,
        f"Id {id}",
        (box[0], box[1]),
        cv2.FONT_HERSHEY_SIMPLEX,
        1,
        (0, 0, 255),
        2,
    )
cv2.imshow("frame", frame)
if cv2.waitKey(1) & 0xFF == ord("q"):
    break

[holy-chicken]

this is the official method provided. previously, i was using the method mentioned above to obtain the tracker ID.

[YongerZhu]

with this 'ids = results[0].boxes.id.cpu().numpy().astype(int)',
given the wrong messages: ids = results[0].boxes.id.cpu().numpy().astype(int)
AttributeError: 'NoneType' object has no attribute 'cpu'
how to resolve?

[pderrenger]

@YongerZhu the error message you're seeing indicates that results[0].boxes.id is None, which means that no tracking IDs are available in the boxes object. This could happen if the tracking algorithm did not assign IDs to the detected objects, or if the track method was not called with the correct arguments to enable tracking.

To resolve this issue, ensure that you are using a model that supports tracking and that you have specified the correct tracker configuration file. Here's an updated version of your code with additional checks for the id attribute:

import cv2
from ultralytics import YOLO

# Load the model with tracking enabled
model = YOLO("yolov8n.pt", tracker="bytetrack.yaml")

# Open the video file
cap = cv2.VideoCapture("/path/to/video")

# Process the video frames
while True:
    ret, frame = cap.read()
    if not ret:
        break

    # Run tracking on the current frame
    results = model.track(frame, persist=True)

    # Check if tracking IDs are available
    if hasattr(results[0].boxes, 'id') and results[0].boxes.id is not None:
        boxes = results[0].boxes.xyxy.cpu().numpy().astype(int)
        ids = results[0].boxes.id.cpu().numpy().astype(int)

        # Draw boxes and IDs on the frame
        for box, id in zip(boxes, ids):
            cv2.rectangle(frame, (box[0], box[1]), (box[2], box[3]), (0, 255, 0), 2)
            cv2.putText(frame, f"Id {id}", (box[0], box[1]), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
    else:
        print("Tracking IDs not available.")

    # Display the frame
    cv2.imshow("frame", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

# Release resources
cap.release()
cv2.destroyAllWindows()

Make sure that the bytetrack.yaml file is present in the ultralytics/cfg/trackers directory and that it is correctly configured for tracking. If the issue persists, it may be helpful to check the documentation or seek support from the Ultralytics team for further assistance.

[yzhEric]

for i, r in enumerate(results):
    for index, box in enumerate(r.boxes):
            tracker_id = box.id

[pderrenger]

@yzhEric looks like you're iterating through the results to access each detection's tracker ID. If you're using YOLOv8 for object tracking, this snippet will work as long as results is a list of Results objects and tracking was enabled during prediction. Each box in r.boxes will have an associated id attribute if the tracker is properly configured. 👍

If you encounter any issues or have further questions, feel free to ask!

[alcarvalhox]

In the code : "from ultralytics import YOLO
model = YOLO('yolov8n.pt')
results = model.track(source="https://youtu.be/Zgi9g1ksQHc", conf=0.3, iou=0.5, show=True) "
How do I get a "results" report of counting the number of trackers related to each model prediction type?

[pderrenger]

@alcarvalhox to count the number of trackers related to each model prediction type, you can iterate through the results object and tally the occurrences of each class. Here's an example of how you might modify your code to achieve this:

from ultralytics import YOLO
from collections import Counter

# Load the model
model = YOLO('yolov8n.pt')

# Run tracking on the source
results = model.track(source="https://youtu.be/Zgi9g1ksQHc", conf=0.3, iou=0.5, show=True)

# Initialize a counter for all detected classes
class_counts = Counter()

# Iterate through the results and count each class
for result in results:
    # Get the class IDs for all detections in the frame
    class_ids = result.boxes.cls.cpu().tolist()
    
    # Update the class counts
    class_counts.update(class_ids)

# Convert class IDs to class names and print the counts
for class_id, count in class_counts.items():
    class_name = model.names[class_id]
    print(f"Class '{class_name}' was tracked {count} times.")

This script uses a Counter from the collections module to keep track of how many times each class is detected across all frames. After processing all frames, it prints out the count for each class. The model.names attribute is used to convert class IDs to human-readable class names.

Remember to install the necessary packages and ensure that the video source is accessible. If you're processing a video file or a live stream, you might want to accumulate counts over all processed frames. If stream=True is used, results will be a generator, and the loop will process frames as they are generated.

[uzumakinaruto19]

How do i specifically call out the Unique id?
ids=results[0].boxes.id.cpu().numpy()
is this the correct way?

[pderrenger]

@uzumakinaruto19 yes, that's correct! To access the unique IDs assigned to tracked objects, you can use the id attribute from the boxes object within the Results object. Your code snippet correctly retrieves the IDs and moves them to the CPU and converts them to a NumPy array. Here's the full usage in context:

from ultralytics import YOLO

# Load your model
model = YOLO('path/to/your/model.pt')

# Perform tracking
results = model.track('path/to/your/video.mp4')

# Access the unique IDs
ids = results[0].boxes.id.cpu().numpy()

This will give you an array of IDs for the detected objects in the first frame processed by the tracker. If you're processing multiple frames, you might want to collect IDs in a loop for each frame. 😊👍

[Wei0205]

I want to count how many objects are in each class. How do I do that

[tamara-laranga]

I'm also interested in this. does anyone have a clue to do that?

[pderrenger]

@Laranga22 to count the number of objects in each class after performing detection with YOLOv8, you can access the results object returned by the predict method. The results object contains the detected classes and their respective counts. Here's a concise example of how you can achieve this:

from ultralytics import YOLO

# Load your trained model
model = YOLO('path/to/your/model.pt')

# Perform prediction on an image or set of images
results = model.predict(source='path/to/your/image.jpg')

# Access the predicted classes and counts
detected_classes = results.pred[0][:, -1].int()  # Extracting the class IDs from the predictions
unique_classes, counts = detected_classes.unique(return_counts=True)  # Counting unique class IDs

# Print the count of each class
for cls, count in zip(unique_classes, counts):
    print(f"Class {cls.item()}: {count.item()} objects")

This code snippet will print the count of detected objects for each class in the provided image(s). Remember to replace 'path/to/your/model.pt' and 'path/to/your/image.jpg' with the actual paths to your model and image(s).

[ImSuMyatNoe]

I would like to know that , in order to use in the track model of yolov8, how can i train my custom model?
Although i train my custom model based on the yolov8x-seg.pt weight, but when i use this custom weight on the track function, there is no detection. Can someone please help me?

[pderrenger]

@ImSuMyatNoe hello! To train a custom model for tracking with YOLOv8, you'll need to follow these general steps:

1. Prepare Your Dataset: Ensure your dataset is properly annotated and structured. For tracking, you typically need sequential frames with consistent object IDs across frames.

2. Train Your Model: Use the train mode to train your model on your custom dataset. If you're starting from a pretrained segmentation model like yolov8x-seg.pt, make sure your dataset includes the necessary segmentation annotations.

3. Validate Your Model: After training, use the val mode to validate your model's performance on a separate validation set. This helps ensure that your model generalizes well to new data.

4. Use the Trained Model for Tracking: Once you have a trained and validated model, you can use it in the track mode. Make sure to load your custom trained weights when initializing the YOLO model for tracking.

Here's a concise example of how you might train and then use your model for tracking:

from ultralytics import YOLO

# Train your custom model
model = YOLO('yolov8x-seg.yaml')  # Use the appropriate model configuration file
results = model.train(data='your_dataset.yaml', epochs=50)  # Train on your custom dataset

# Use your trained model for tracking
model = YOLO('path/to/your_trained_weights.pt')  # Load your custom trained weights
results = model.track(source='path/to/your_video.mp4')  # Perform tracking on a video

If you're not getting any detections during tracking, consider the following:

• Check Annotations: Ensure your custom dataset annotations are correct and in the expected format.
• Model Configuration: Verify that the model configuration (yaml file) matches your dataset classes and structure.
• Thresholds: Adjust the confidence and IoU thresholds when using the track mode, as they might be too restrictive.
• Validation Performance: Review the validation results to confirm that your model is performing well on the validation set.

If you continue to face issues, you might want to share more details about your training process, dataset, and the exact commands you're using, so the community can provide more targeted assistance. 🛠️

[CloarecZ]

When tracking vehicles based on videos, how do I separate the cars into two groups indicating directions?

[pderrenger]

@CloarecZ to separate tracked vehicles into two groups indicating directions, you can analyze the trajectories of the detected vehicles over time. Here's a concise approach:

1. Track Vehicles: Use the track mode to track vehicles in the video. Ensure you have a trained model capable of detecting vehicles.

2. Analyze Trajectories: Post-process the tracking results to analyze the direction of movement. You can calculate the displacement of each vehicle's bounding box center between frames.

3. Define Direction Criteria: Establish criteria for direction, such as the change in the x-coordinate for left/right movement or the y-coordinate for up/down movement.

4. Group Vehicles: Based on the calculated trajectories and your criteria, assign each vehicle to a group representing its direction.

5. Visualize or Output: Optionally, visualize the results by drawing the trajectories in different colors for each group or output the grouped data for further analysis.

Remember to consider the orientation of the camera and the scene's layout when defining your direction criteria. 🚗🔍

[log901108]

how can I get result of tracking ID of each object?
and how can I get boxes information of each object by tracking ID?

[tztechno]

Add save_txt=True in the command line.
The result txt file will be generated in runs/detect/track/labels.
The last term in each line is tracking ID.

[log901108]

Thank You so much

[pderrenger]

@log901108 you're welcome! If you have any more questions or need further assistance, feel free to ask. Happy tracking! 😊🚀

[Tai-Pham-2002]

help me , i using ultralytics version 8.0.128

import lap
ModuleNotFoundError: No module named 'lap'

[log901108]

Try your python version with downgrading to 3.8

[Tai-Pham-2002]

unsolvable

[pderrenger]

@Tai-Pham-2002 it seems like you're encountering an issue with a missing module (lap) required for tracking. The lap module is a linear assignment problem solver used in some tracking algorithms. To resolve this issue, you should install the lap module. You can do this by running the following command in your terminal:

pip install lap

Make sure you are using the correct Python environment where YOLOv8 is installed. If you continue to face issues, please check the Track documentation for additional guidance on setting up tracking with YOLOv8. If the problem persists, feel free to open an issue on the repository with detailed information about your setup and the error message you're receiving. 🛠️

[Tai-Pham-2002]

import lap
ModuleNotFoundError: No module named 'lap'

[sr99622]

The official lap install is broken, there is the same package under another name that is fixed

pip install lap-sdist

[pioneerRick]

conda install -c conda-forge lap I think this is helpful

[RizwanMunawar]

You can install the lap package by using the provided command:

pip install lapx

Thank you.

[sammcdevitt]

How do I display the tail like shown in the image above?

[pderrenger]

@sammcdevitt to display tails or trails behind tracked objects, you can use the --tail argument when running the tracker from the command line. This will show the past positions of each object as a tail. If you're using the Python API, you can achieve this by setting the tail parameter in the track method.

Here's an example of how to use it in Python:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8n.pt')

# Track objects with tails
results = model.track(source='your_video.mp4', tail=30)  # tail length of 30 frames

Adjust the tail parameter to the desired length of the trail in frames. 🐍🔍

[aus10powell]

My setup:

• Have a trained object detection model that has around 0.70 mAP50-95
• Using model with track method:

    for result in model.track(
        source=video_path,
        show=True,
        stream=True,
        agnostic_nms=True,
        tracker="botsort.yaml",
    ):

• Here's my current settings for the "botsort.yaml":

tracker_type: botsort  # tracker type, ['botsort', 'bytetrack']
track_high_thresh: 0.2  # 0.6 default threshold for the first association
track_low_thresh: 0.1  # threshold for the second association
new_track_thresh: 0.1  # 0.6 default threshold for init new track if the detection does not match any tracks
track_buffer: 30  # buffer to calculate the time when to remove tracks
match_thresh: 0.1  # 0.8 default threshold for matching tracks
# min_box_area: 10  # threshold for min box areas(for tracker evaluation, not used for now)
# mot20: False  # for tracker evaluation(not used for now)

# BoT-SORT settings
cmc_method: sparseOptFlow  # method of global motion compensation
# ReID model related thresh (not supported yet)
proximity_thresh: 0.5
appearance_thresh: 0.25
with_reid: False

My problem:
I'm getting decent confidence on all images, but having inconsistent results with both ID'ing the object.

Things I've tried:

• Adjusting he above parameters to be surprisingly low
• Trying to get lower cls_loss to help with object id'ing (~2.8 logloss is best I've gotten. Not great but the video quality is not great)

My Question:

• Suggestions on most likely culprit to improve given that I likely will not be able to improve video quality

[pderrenger]

You're absolutely right! Currently, with_reid is not supported in BoT-SORT as per the implementation in bot_sort.py. This means tweaking the with_reid parameter won't have any effect right now. 😅 Your keen observation will surely help others avoid confusion. We're always working on enhancements, so stay tuned for updates. Your input is highly valued, and contributions are welcome if you have suggestions or improvements!

For implementing tracking without ReID support, focusing on adjustments to parameters like track_high_thresh, track_low_thresh, and track_buffer can be beneficial for optimizing tracking performance given the current capabilities. Here’s a quick snippet adjusting these settings for better tracking:

tracker_config = {
  "tracker_type": "botsort",
  "track_high_thresh": 0.3,
  "track_low_thresh": 0.2,
  "new_track_thresh": 0.3,
  "track_buffer": 20,
  "match_thresh": 0.2,
  "cmc_method": "sparseOptFlow",
}

Thank you for engaging with us, and we look forward to any more insights or questions you might have! 🚀

[RunsenXu]

Hi, @pderrenger ,
I am new to tracking and confused by the meanings of tracker arguments. Could you give me some references or explain what these arguments mean, especially the thresholds?

In my project, I encountered a problem where an object is not associated with the same object in the previous frame (which should be) or assigned as a new tracked object.

How to tune the arguments?

Thank you so much!

[pderrenger]

Hey there! 😊 Great to hear you're diving into object tracking with YOLOv8. Let's break down those tracking arguments a bit to help address your situation.

Thresholds play a big role in determining when a detection is considered the same object across frames or when to start tracking a new object. Here's a quick look at what each does:

• track_high_thresh: Detections with confidence scores above this value are considered for tracking. Raising it a bit might help avoid false tracks but could miss some true detections.
• track_low_thresh and new_track_thresh: These thresholds help in deciding when to continue tracking an existing object or start a new track. They're especially useful in tricky situations where the object slightly changes appearance or gets partially obscured.

In your project, if an object isn't consistently associated with its previous instance, tweaking these thresholds can be a starting point. For instance, increasing the track_high_thresh might reduce false positives, ensuring only high-confidence detections are tracked. Similarly, adjusting the track_low_thresh can help in maintaining the object's continuity even with lower confidence detections.

To fine-tune:

• Experiment by gradually adjusting each threshold.
• Watch for changes in tracking behavior to find a balance that reduces new object assignment for existing tracks.
• Remember, changes in video quality or scene complexity can necessitate different settings.

Without the ability to improve video quality, these parameter adjustments, along with ensuring your model is as accurate as possible with the given data, are your best bet. Tracking performance heavily depends on the quality and consistency of detection across frames, so focus on optimizing detection confidence and threshold parameters to mitigate issues with object ID consistency.

Hope this helps! Keep experimenting, and don't hesitate to ask more questions. Happy tracking! 🚀

[convurt]

One thing Ive always wondered. When I do a custom training, the parameter defaults to botsort when I view the yaml after training. If I'm using bytetrack, does changing the tracking in the training arguments make a difference?

[pderrenger]

Hey there! 👋 The tracking algorithm selected during training (botsort by default in your YAML) mainly prepares the model for a specific type of tracking approach. However, after training, during deployment or when actually tracking objects in a video stream, you're free to specify a different tracking algorithm (like bytetrack.yaml) without affecting the model's detection capabilities.

The key thing is: the choice of tracker during training pre-sets your YAML, but it's during the tracking (inference time) where specifying the tracker makes a practical difference. Therefore, changing to bytetrack during tracking if you feel it performs better for your specific use case is absolutely fine and encouraged if it yields better results!

For better continuity in tracking the same object across frames, tuning the tracker's parameters, like the association thresholds, might help more than the choice of tracker itself. Given that you've encountered some challenges, tweaking these parameters based on your video's specifics is a good path to explore. 🕵️‍♂️

# Example to switch tracker at inference time
results = model.track(source="your_video.mp4", show=True, tracker="bytetrack.yaml")

Hope this helps clear things up! Happy tracking! 🚀

[khanhdu1609]

can you tell me what "[0]" mean?

[mooend]

if you track many frames at a same time , then you will get a result list.
in the examples above ,there is only one frame in args "source", so , results[0], is just the thing you want to get .

[pderrenger]

@mooend the [0] you're referring to is a way to access the first element of a list in Python. When tracking multiple frames, the results variable becomes a list where each element corresponds to the tracking results of a single frame. By using results[0], you are selecting the tracking results of the first frame in the sequence. If you have multiple frames and you want to access the results of each one, you would iterate over the results list or access each frame's results using results[index] where index is the position of the frame in the list. 😊👍

[prashanthakkala]

I am unable to find the source code for Traffic Counting (In & Out) Using Ultralytics YOLOv8 🔥🔥🔥 🚀

[RizwanMunawar]

@prashanthakkala Thank you for getting in touch! The code for object counting is not currently available in the documentation. We are in the process of modifying the internal structure of the documentation, and the code for object counting and other tutorials will be made available soon.

[AISTALK]

Right now model in predict model see more objects than in a track mode, pre-trained model of any size in track stops seeing half vehicles, which it sees in predict mode. It's actual both for rtdetr, both for yolo

[RizwanMunawar]

@AISTALK, indeed, this is a possibility. The object tracking process may require a few frames to begin considering previously missed objects. To enhance results, you can experiment with the configuration parameters for tracking. Find the tracking configuration parameters here: BoT-SORT Tracker Configuration.

The basic overview of these parameters is mentioned:

• ✅tracker_type: botsort: Tracker type, with options ['botsort', 'bytetrack']. Choose between BoT-SORT or ByteTrack for object tracking.
• ✅track_high_thresh: 0.5: Threshold for the first association. It determines the confidence level for the initial track association.
• ✅track_low_thresh: 0.1: Threshold for the second association. This threshold is used for refining track associations.
• ✅new_track_thresh: 0.6: Threshold for initializing a new track if the detection doesn't match any existing tracks. This parameter influences the creation of new object tracks.
• ✅track_buffer: 30: Buffer to calculate the time when to remove tracks. It helps manage the retention period for tracks in the tracking system.
• ✅match_thresh: 0.8: Threshold for matching tracks. This parameter determines the level of similarity required for associating detections with existing tracks.
• ✅gmc_method: sparseOptFlow: BoT-SORT global motion compensation method. This specifies the method used for global motion compensation in BoT-SORT tracking.
• ✅proximity_thresh: 0.5: Threshold for proximity. It plays a role in determining the closeness criteria for objects or tracks.
• ✅appearance_thresh: 0.25: Threshold for appearance. This parameter relates to the confidence level in matching the appearance features of objects during tracking.
• ✅with_reid: False: Indicates whether ReID (Re-identification) is enabled (currently set to False). ReID is a technique used for identifying and distinguishing objects across different frames.

Hope this helps! Let me know if you have any additional questions.

Thanks
Ultralytics Team!

[AISTALK]

I tried both trackers, mod parameters slightly, also i am already tracking only classes=[2,5,7] and i don't see any diff in accuracy between changing params

[pderrenger]

@AISTALK, if adjusting the tracker parameters and class filtering hasn't resolved the discrepancy between predict and track modes, it might be beneficial to review the following:

1. Dataset Consistency: Ensure the dataset used for tracking is similar to the one used for prediction. Differences in data distribution can affect performance.
2. Model Warm-up: Trackers may require a warm-up period to stabilize and perform optimally. Consider running the tracker for a longer duration to observe if detection improves over time.
3. Frame Rate: Verify that the frame rate of the video source is consistent. A lower frame rate can impact the tracker's ability to maintain object identities.
4. Quality of Detections: The quality of initial detections plays a crucial role in tracking. Poor detection quality can lead to suboptimal tracking results.
5. Re-identification (ReID): Although currently disabled, enabling ReID can sometimes improve tracking by providing a more robust object association across frames.

If the issue persists, it may be helpful to share a minimal reproducible example with the Ultralytics team for further investigation. We're committed to improving the performance and accuracy of our models and appreciate your feedback.

Thank you for your patience and collaboration.

[thanhdatpham122]

I want to classify and count different object classes in the total number of ids, how should I do that?

[pderrenger]

@thanhdatpham122 to classify and count different object classes and their total number of unique IDs during tracking with YOLOv8, you can follow these steps:

1. Load the Model: Load your YOLOv8 model with the appropriate weights that have been trained for detection on your classes of interest.

2. Perform Tracking: Use the track mode to track objects in your video or image sequence. This will provide you with the tracked object IDs and their corresponding classes.

3. Classify and Count: After tracking, you can iterate through the tracked results to classify and count the occurrences of each unique object ID per class.

Here's a concise example in Python:

from ultralytics import YOLO
from collections import defaultdict

# Load your trained YOLOv8 model
model = YOLO('path/to/your/model.pt')

# Perform tracking on your source
tracked_objects = model.track(source='path/to/your/video.mp4')

# Initialize a dictionary to hold counts
class_counts = defaultdict(int)

# Iterate through tracked results and count classes
for obj in tracked_objects:
    class_counts[obj.cls] += 1

# Print the total counts per class
for cls, count in class_counts.items():
    print(f'Class {cls}: {count} objects')

This code will give you the total count of unique object IDs for each class detected and tracked in the video. Adjust the source parameter to point to your video file or image sequence, and ensure your model is correctly trained to detect the classes you're interested in. 🚀

For more details on tracking and the available options, you can refer to the Ultralytics Docs on Track mode.

[sorooshsorkhani]

Inconsistent Tracking with YOLOv8: Seeking Solutions

I'm using YOLOv8 for object counting and encountering inconsistent tracking results. Here's my setup:

model.track(source_video_path, imgsz=1280, classes=CLASS_ID, stream=True, vid_stride=2, persist=True)

Issue: When running the same code on the same video, the tracker_id assignment varies. Sometimes it's perfect, with each object getting a unique tracker_id. Other times, the same object receives multiple IDs, or an ID is assigned to multiple objects. This inconsistency persists across both botsort.yaml and bytetrack.yaml trackers.

Key Questions:

1. How can I ensure each object receives a unique tracker_id?
2. What steps can I take to achieve consistent tracking results?

Any insights or suggestions to address these issues would be greatly appreciated!

[pderrenger]

@sorooshsorkhani it's important to ensure that your tracking setup is optimized for the best performance. Here are a few suggestions to address the inconsistency in tracker_id assignments:

1. Stable Detection: Ensure that the detection model is stable and accurate. If the detections are inconsistent, the tracker will struggle to maintain consistent IDs. You might want to fine-tune your model on your specific dataset for better detection stability.

2. Tracker Hyperparameters: Experiment with the tracker hyperparameters in botsort.yaml or bytetrack.yaml. Adjusting parameters such as iou_threshold, time_window, and track_buffer can significantly impact tracking consistency.

3. Frame Rate: Consider not skipping frames (vid_stride=1) to provide the tracker with more information for maintaining identities. Skipping frames can lead to missed detections and ID switches.

4. Image Size: Ensure that imgsz is large enough to capture sufficient detail for tracking, but also consider the trade-off with processing speed.

5. Persistence: If you're using the persist flag, understand that it's designed to keep track identities longer even without detections, which can sometimes lead to incorrect ID assignments if not tuned properly.

6. Post-processing: Implement post-processing steps to handle ID switches and fragmentation. This can involve smoothing techniques or using additional filters based on object movement patterns.

7. Version Check: Ensure you are using the latest version of YOLOv8, as updates may include improvements to tracking algorithms.

8. Custom Tracker: If the provided trackers do not meet your needs, consider implementing a custom tracker or integrating a third-party tracking algorithm that's more suited to your specific use case.

For further guidance, you can refer to the Track documentation for more detailed information on tracker configuration and usage. If the issue persists, providing a reproducible example with your dataset can help the community to assist you better. 🛠️

[apiszcz]

i had similar results using yv8 on WSL2 on WINDOWS. No issues on linux.

…

On Wed, Dec 6, 2023 at 10:45 AM Soroosh Sorkhani ***@***.***> wrote: *Inconsistent Tracking with YOLOv8: Seeking Solutions* I'm using YOLOv8 for object counting and encountering inconsistent tracking results. Here's my setup: model.track(source_video_path, imgsz=1280, classes=CLASS_ID, stream=True, vid_stride=2, persist=True) *Issue:* When running the same code on the same video, the tracker_id assignment varies. Sometimes it's perfect, with each object getting a unique tracker_id. Other times, the same object receives multiple IDs, or an ID is assigned to multiple objects. This inconsistency persists across both botsort.yaml and bytetrack.yaml trackers. *Key Questions:* 1. How can I ensure each object receives a unique tracker_id? 2. What steps can I take to achieve consistent tracking results? Any insights or suggestions to address these issues would be greatly appreciated! — Reply to this email directly, view it on GitHub <#2687 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAK5KTOQCXUYTIGP4PQJ3ATYICHLFAVCNFSM6AAAAAAYF552DGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONZXHEYTQ> . You are receiving this because you commented.Message ID: ***@***.*** com>

[pderrenger]

@apiszcz it's great to hear that you've had success with YOLOv8 on Linux. Regarding the inconsistent tracking results on WSL2, tracking can indeed be sensitive to various factors, including the underlying system performance and configuration. Here are a few suggestions to help ensure more consistent tracking:

1. System Performance: Ensure that WSL2 has access to sufficient resources. Performance can impact tracking consistency, especially if the system is under heavy load or if there are differences in GPU acceleration between WSL2 and your native Linux setup.

2. Deterministic Behavior: Set deterministic=True in your tracking configuration to ensure reproducible results if you're not already doing so. This can sometimes help with consistency.

3. Hyperparameters: Experiment with the tracker hyperparameters in botsort.yaml or bytetrack.yaml. Adjusting parameters like iou_threshold, conf_thres, and frame_rate can influence tracker assignment.

4. Version Consistency: Make sure you're using the same version of YOLOv8 and dependencies on both WSL2 and Linux. Differences in versions can lead to different behaviors.

5. Debugging: Enable verbose output during tracking to get more insights into what's happening under the hood. This might provide clues as to why the tracking is inconsistent.

6. Data Preprocessing: Preprocess your video to ensure consistent frame rates and resolutions. Variations in these can affect tracking performance.

If you continue to experience issues, consider sharing a minimal reproducible example on the GitHub issues page for further assistance. The community and maintainers can often provide more targeted help when they can see the code and data you're working with. For more detailed guidance on tracking, you can refer to our Track documentation. Keep in mind that object tracking is a complex task and may require fine-tuning to achieve optimal results in different environments. 🚀

[DMEN073]

Is the id correspond to the result array arrangement?

[glenn-jocher]

@DMEN073 yes, the ID corresponds to the arrangement of the results array. Each detected object is assigned a unique ID that persists across frames, allowing you to track the object's movement over time. If you need a code example to clarify, here's a simple way to access the IDs and their corresponding bounding boxes:

from ultralytics import YOLO

# Load your model
model = YOLO('path/to/your/model.pt')

# Run tracking on your source
results = model.track(source='path/to/your/video.mp4')

# Accessing IDs and bounding boxes
for result in results:
    for id, box in zip(result.boxes.id, result.boxes.xyxy):
        print(f"ID: {id}, Bounding Box: {box}")

Hope this helps! Let me know if you have any more questions. 😊

[YongerZhu]

Can I use YOLO V8 to track the same object in different surveillance frames, and if so, how do I do it, and if not, what algorithms can I incorporate to do it?

[pderrenger]

Yes, you can use YOLOv8 for tracking objects across different surveillance frames! 🚀 YOLOv8 supports multi-object tracking with algorithms like BoT-SORT and ByteTrack. Here's a quick example on how to do it:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8n.pt')  # or your custom trained model

# Perform tracking on your video source
results = model.track(source="path/to/your/video.mp4", tracker="botsort.yaml")  # or "bytetrack.yaml"

Or using the CLI:

yolo track model=yolov8n.pt source="path/to/your/video.mp4" tracker="botsort.yaml"

This will track objects across frames, maintaining unique IDs for each detected object. If you're looking to incorporate other algorithms or customize tracking, you can modify the tracker configuration files as needed. Happy tracking! 🎥

[Mauricio566]

hello. Note that when making the inference the video is super slow, I don't think it's a question of my hardware or is it? My laptop has good resources. Or I wonder if the track method will have some parameter to solve this issue of slowness in the video. although the detections are going well. thank you

[glenn-jocher]

Hello! 😊

Thank you for reaching out. If your detections are running smoothly but the video is slow, there are a few parameters you might want to tweak to improve performance:

1. Confidence Threshold (conf): Lowering this can speed up processing but might reduce detection accuracy.
2. Intersection Over Union (iou): Adjusting this can help with performance.
3. Batch Size: Ensure you're not processing too many frames at once.

Here's an example of how to adjust these parameters in Python:

from ultralytics import YOLO

# Load your model
model = YOLO("yolov8n.pt")

# Adjust parameters and run tracking
results = model.track(source="your_video.mp4", conf=0.3, iou=0.5, show=True)

And via CLI:

yolo track model=yolov8n.pt source="your_video.mp4" conf=0.3 iou=0.5 show

If you continue to experience issues, consider checking your hardware utilization to ensure your resources are being fully leveraged. For more detailed configurations, you can refer to the tracking documentation.

Hope this helps! 🚀

[pedromoraesh]

I was wondering: can ByeTrack do matching ops using GPU? Can this improve overall performance?

[pderrenger]

Hello! Yes, ByteTrack can utilize GPU for matching operations, which can indeed enhance the overall performance by accelerating the computation process. Utilizing GPU resources effectively can lead to faster tracking and improved real-time capabilities, especially beneficial in scenarios requiring high frame rate processing. If you have further questions or need more detailed guidance, feel free to ask! 😊

[amirrezabsh]

Hi!
Is there any way to use the tracker independently?! I have a yolov8 deepsparse model which doesn't support the ultralytics yolo tracker and I want to use the object detection results of deepsparse model and feed the results to ultralytics tracker. Can anyone here help me with that?
something like the code below:

from deepsparse import Pipeline
from ultralytics.tracker.trackers.bot_sort import BOTSORT


model_path = "yolov8n.onnx"
yolo_pipeline = Pipeline.create(
    task="yolov8",
    model_path=model_path,
)
cap = cv2.VideoCapture(video_path)
tracker  = BOTSORT()
while True:
    ret, frame = cap.read()
    if not ret:
        break
    detections = yolo_pipeline([frame])
    tracks = tracker.update(detections)

[glenn-jocher]

@amirrezabsh hello!

Currently, our Ultralytics trackers, such as BoT-SORT and ByteTrack, are designed to work seamlessly with the YOLO models directly within our ecosystem. They are not set up out-of-the-box to accept external model outputs like those from a DeepSparse model.

However, you can adapt the tracker to work with your DeepSparse model outputs by ensuring that the detection outputs from DeepSparse are formatted correctly to match the expected input format of our trackers. This typically involves ensuring that the detection outputs are structured as [x1, y1, x2, y2, confidence, class] for each detection.

Here's a simplified approach:

1. Ensure your DeepSparse model outputs the detections in the required format.
2. Use these detections as inputs to the update() method of the tracker.

If you need further customization or integration help, consider exploring the source code of our trackers and modifying them as needed to suit your specific requirements.

Best of luck with your tracking implementation! If you have more questions or need further assistance, feel free to ask. 🚀

[ulziitogtoh]

Hi
Can i use ground truth data which is image sequences with annotation file instead of object detection model for ultralytics tracker?

[pderrenger]

@ulziitogtoh hi, Ultralytics YOLO trackers require an object detection model to generate the initial detections before tracking. Using only ground truth data with annotations is not supported for tracking. You can use a trained YOLO model to generate detections and then apply tracking. For more details, please refer to the Ultralytics tracking documentation.

[FayalaMohamed]

Hello I am using YOLOv8 tracking on a custom dataset and I want to save in a csv the latent space information for every tracked object and for every frame of the video. Can I get some help with this please ?

[glenn-jocher]

@FayalaMohamed to save latent space information for each tracked object, you'll need to modify the YOLO model to extract and store these features during tracking. This typically involves accessing intermediate layers of the model. For detailed guidance, you might want to explore PyTorch's hooks or similar methods. If you have further questions, feel free to ask!

[FayalaMohamed]

Thanks @glenn-jocher ! I have this code and I just wanted to make sure that is extracting all the latent space information. I am able to get back a huge vector for each object but wanted to make sure I am not missing anything.

class YOLOWithLatentSpace(YOLO):
    def extract_features(self, x):
        x = x.to(self.device)
        x = self.model.model[0:4](x)
        return self.model.model[4](x)

def track_with_latent_space(video_path, weights_path):
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    
    model = YOLOWithLatentSpace(weights_path)
    model.to(device)

    cap = cv2.VideoCapture(video_path)

    fps = int(cap.get(cv2.CAP_PROP_FPS))
    width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    
    fourcc = cv2.VideoWriter_fourcc(*'XVID')
    output_path = 'tracked_output.avi'
    out = cv2.VideoWriter(output_path, fourcc, fps, (width, height))

    csv_file = 'tracked_detections_with_latent.csv'
    detection_data = []

    frame_idx = 0
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break

        # Prepare frame for YOLO
        frame_resized = cv2.resize(frame, (640, 640))
        frame_tensor = torch.from_numpy(frame_resized).permute(2, 0, 1).unsqueeze(0).float() / 255.0
        frame_tensor = frame_tensor.to(device)

        # Extract latent features
        with torch.no_grad():
            latent_features = model.extract_features(frame_tensor)

        results = model.track(frame, persist=True)

        if results and results[0].boxes:
            boxes = results[0].boxes
            for i, box in enumerate(boxes):
                x1, y1, x2, y2 = box.xyxy[0].tolist()
                track_id = box.id.item() if box.id is not None else -1
                conf = box.conf.item()
                cls = box.cls.item()

                cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (255, 0, 0), 2)
                cv2.putText(frame, f'ID: {track_id}', (int(x1), int(y1) - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 0, 0), 2)

                # Extract corresponding latent feature
                feature_x = int(((x1 + x2) / 2 / 640) * latent_features.shape[3])
                feature_y = int(((y1 + y2) / 2 / 640) * latent_features.shape[2])
                feature_vector = latent_features[0, :, feature_y, feature_x].cpu().numpy()

                detection_data.append({
                    'frame': frame_idx,
                    'track_id': track_id,
                    'x1': x1,
                    'y1': y1,
                    'x2': x2,
                    'y2': y2,
                    'confidence': conf,
                    'class': cls,
                    'latent_feature': feature_vector.tolist()
                })
        
        out.write(frame)
        print(f"Processed frame {frame_idx}")
        frame_idx += 1

    cap.release()
    out.release()

    df = pd.DataFrame(detection_data)
    df.to_csv(csv_file, index=False)
    print(f"Tracking information with latent features saved to {csv_file}")

[glenn-jocher]

To ensure you're capturing all latent space information, verify that your extract_features method accesses the desired layers. You might want to explore PyTorch hooks for more granular control. If you need further assistance, feel free to ask!

[apiszcz]

https://docs.ultralytics.com/modes/track/#persisting-tracks-loop

…

On Mon, Sep 30, 2024 at 6:01 AM FayalaMohamed ***@***.***> wrote: Hello I am using YOLOv8 tracking on a custom dataset and I want to save the latent space information for every tracked object and for every frame of the video. Can I get some help with this please ? — Reply to this email directly, view it on GitHub <#2687 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAK5KTI72XZ4GBR23A53WXLZZEOQPAVCNFSM6AAAAAAYF552DGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANZZGYZTONA> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[glenn-jocher]

@apiszcz to save latent space information for each tracked object, you'll need to modify the model's forward pass to extract and store these features. Unfortunately, this requires custom code adjustments beyond standard usage. For detailed guidance, please refer to the Ultralytics documentation or community forums.

[WangCarl77424]

hello i use yolov8 tracking. I want to keep the tracking ID when frames are lost. I have tried to change the self.max_time_lost parameter in byte_tracker.py but the effect is not as expected. Which file or program can I change?
I want the object to have an ID.Even if you disappear for a long time and then reappear on the screen, there will be no new ID

[glenn-jocher]

@WangCarl77424 to maintain consistent IDs across long occlusions in YOLOv8 tracking, create a custom tracker configuration file by copying bytetrack.yaml from the Ultralytics tracker configurations directory. Increase the track_buffer parameter (which controls how many frames to keep lost tracks) and use it with model.track(tracker='custom_bytetrack.yaml'). This approach preserves your modifications across updates better than editing source files directly.

[ryouchinsa]

RectLabel is an offline image annotation tool for object detection and segmentation.
Although this is not an open source program, with RectLabel, you can save a track id for each box, using an attribute of "Text input" type and combine with the object name such as "sneakers-1", the attribute name should be set as "track id" on the setting.
For the YOLO Detection Track format, see the CVAT document.

class_index center_x center_y width height track_id
0 0.464615 0.594724 0.680000 0.769784 1

yolo_track_id.mp4

[glenn-jocher]

@ryouchinsa thanks for sharing RectLabel's tracking capabilities! For Ultralytics YOLO tracking implementations with persistent IDs, see our detailed Multi-Object Tracking guide at https://docs.ultralytics.com/modes/track. Community integrations like this are always interesting to see! 🚀