Save new video that only shows detections on filtered classes

[courtneywhelan]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

How would I save a new video that only shows the detection on the specific classes that I filter on. I run detect.py with the --class flag and see the detections are filtered but the output video still shows the detections from all of the classes opposed to just the classes I want to see.

Additional

No response

[github-actions]

👋 Hello @courtneywhelan, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

Hello,

Thank you for reaching out and for your detailed question! It sounds like you're on the right track using the --class flag to filter detections. However, the issue you're encountering with the output video still showing all detections could be due to how the video is being processed and saved.

To ensure that only the filtered classes are shown in the output video, you can modify the detect.py script slightly. Here's a step-by-step guide to help you achieve this:

1. Ensure you're using the latest versions:

   • Make sure you have the latest version of YOLOv5 from the repository.
   • Update your torch package to the latest version.

2. Modify detect.py:

   • Open the detect.py script and locate the section where the results are being processed and saved.
   • Add a filter to ensure only the desired classes are included in the output.

Here's an example modification you can make to the detect.py script:

# Assuming you have already parsed the --class argument and have a list of class indices to filter
filtered_classes = [0, 1, 2]  # Example class indices you want to keep

# Inside the loop where detections are processed
for i, det in enumerate(pred):  # detections per image
    if len(det):
        # Apply NMS
        det[:, :4] = scale_coords(im0.shape[2:], det[:, :4], im0.shape).round()

        # Filter detections by class
        det = det[det[:, 5].isin(filtered_classes)]

        # Proceed with saving the filtered detections
        for *xyxy, conf, cls in reversed(det):
            if save_txt:  # Write to file
                xywh = (xyxy2xywh(torch.tensor(xyxy).view(1, 4)) / gn).view(-1).tolist()  # normalized xywh
                line = (cls, *xywh, conf) if save_conf else (cls, *xywh)  # label format
                with open(txt_path + '.txt', 'a') as f:
                    f.write(('%g ' * len(line)).rstrip() % line + '\n')

            if save_img or view_img:  # Add bbox to image
                label = f'{names[int(cls)]} {conf:.2f}'
                plot_one_box(xyxy, im0, label=label, color=colors[int(cls)], line_thickness=3)

3. Run the script:
   • Use the --class flag to specify the classes you want to filter.
   • Run the modified detect.py script to process your video and save the output with only the filtered classes.

If you encounter any issues or need further assistance, please provide a minimum reproducible example of your code and the exact command you are using to run the script. This will help us better understand the problem and provide a more accurate solution. You can refer to our minimum reproducible example guide for more details.

Thank you for your cooperation, and I hope this helps! 😊

[courtneywhelan]

Thanks! I am getting the error IndexError: tuple index out of range.

I run the command python detect_filter.py --weights yolov5x.pt --source ~/file --class 0 2 5 7

Here is the modified code. Any thoughts on why I am getting this error?

      # Process predictions
    for i, det in enumerate(pred):  # per image
        seen += 1
        if webcam:  # batch_size >= 1
            p, im0, frame = path[i], im0s[i].copy(), dataset.count
            s += f"{i}: "
        else:
            p, im0, frame = path, im0s.copy(), getattr(dataset, "frame", 0)
        if len(det):
         # Apply NMS
            det[:, :4] = scale_boxes(im0.shape[2:], det[:, :4], im0.shape).round()
            det = det[det[:, 5].isin(opt.classes)]
        p = Path(p)  # to Path
        save_path = str(save_dir / p.name)  # im.jpg
        txt_path = str(save_dir / "labels" / p.stem) + ("" if dataset.mode == "image" else f"_{frame}")  # im.txt
        s += "%gx%g " % im.shape[2:]  # print string
        gn = torch.tensor(im0.shape)[[1, 0, 1, 0]]  # normalization gain whwh
        imc = im0.copy() if save_crop else im0  # for save_crop
        annotator = Annotator(im0, line_width=line_thickness, example=str(names))
        if len(det):
            # Rescale boxes from img_size to im0 size
            det[:, :4] = scale_boxes(im.shape[2:], det[:, :4], im0.shape).round()

            # Print results
            for c in det[:, 5].unique():
                n = (det[:, 5] == c).sum()  # detections per class
                s += f"{n} {names[int(c)]}{'s' * (n > 1)}, "  # add to string

            # Write results
            for *xyxy, conf, cls in reversed(det):
                c = int(cls)  # integer class
                label = names[c] if hide_conf else f"{names[c]}"
                confidence = float(conf)
                confidence_str = f"{confidence:.2f}"

                if save_csv:
                    write_to_csv(p.name, label, confidence_str)

                if save_txt:  # Write to file
                    xywh = (xyxy2xywh(torch.tensor(xyxy).view(1, 4)) / gn).view(-1).tolist()  # normalized xywh
                    line = (cls, *xywh, conf) if save_conf else (cls, *xywh)  # label format
                    with open(f"{txt_path}.txt", "a") as f:
                        f.write(("%g " * len(line)).rstrip() % line + "\n")

                if save_img or view_img:  # Add bbox to image
                    label = f'{names[int(cls)]} {conf:.2f}'
                    plot_one_box(xyxy, im0, label=label, color=colors[int(cls)], line_thickness=3)

[glenn-jocher]

@courtneywhelan hello,

Thank you for sharing the details of your issue and the modified code. The IndexError: tuple index out of range typically indicates that the code is trying to access an index that doesn't exist in a tuple or list. Let's address this step-by-step.

Steps to Troubleshoot:

1. Ensure Latest Versions:

   • First, please make sure you're using the latest versions of YOLOv5 and torch. You can update YOLOv5 by pulling the latest changes from the repository and update torch using:

     pip install --upgrade torch

2. Minimum Reproducible Example:

   • To help us investigate further, could you please provide a minimum reproducible example? This includes a snippet of your code that can be run independently and reproduces the error. You can refer to our minimum reproducible example guide for more details.

3. Code Review:

   • From the code snippet you provided, it seems like there might be an issue with how the det tensor is being accessed. Specifically, the line det = det[det[:, 5].isin(opt.classes)] might be causing the issue if det does not have the expected dimensions.

Suggested Code Fix:

Here's a revised version of your code snippet with added checks and corrections:

# Process predictions
for i, det in enumerate(pred):  # per image
    seen += 1
    if webcam:  # batch_size >= 1
        p, im0, frame = path[i], im0s[i].copy(), dataset.count
        s += f"{i}: "
    else:
        p, im0, frame = path, im0s.copy(), getattr(dataset, "frame", 0)
    
    if len(det):
        # Apply NMS
        det[:, :4] = scale_boxes(im0.shape[2:], det[:, :4], im0.shape).round()
        
        # Ensure det has the expected dimensions
        if det.shape[1] > 5:
            det = det[det[:, 5].isin(opt.classes)]
        else:
            print(f"Warning: det does not have the expected dimensions: {det.shape}")
            continue

    p = Path(p)  # to Path
    save_path = str(save_dir / p.name)  # im.jpg
    txt_path = str(save_dir / "labels" / p.stem) + ("" if dataset.mode == "image" else f"_{frame}")  # im.txt
    s += "%gx%g " % im.shape[2:]  # print string
    gn = torch.tensor(im0.shape)[[1, 0, 1, 0]]  # normalization gain whwh
    imc = im0.copy() if save_crop else im0  # for save_crop
    annotator = Annotator(im0, line_width=line_thickness, example=str(names))
    
    if len(det):
        # Rescale boxes from img_size to im0 size
        det[:, :4] = scale_boxes(im.shape[2:], det[:, :4], im0.shape).round()

        # Print results
        for c in det[:, 5].unique():
            n = (det[:, 5] == c).sum()  # detections per class
            s += f"{n} {names[int(c)]}{'s' * (n > 1)}, "  # add to string

        # Write results
        for *xyxy, conf, cls in reversed(det):
            c = int(cls)  # integer class
            label = names[c] if hide_conf else f"{names[c]} {conf:.2f}"
            confidence = float(conf)
            confidence_str = f"{confidence:.2f}"

            if save_csv:
                write_to_csv(p.name, label, confidence_str)

            if save_txt:  # Write to file
                xywh = (xyxy2xywh(torch.tensor(xyxy).view(1, 4)) / gn).view(-1).tolist()  # normalized xywh
                line = (cls, *xywh, conf) if save_conf else (cls, *xywh)  # label format
                with open(f"{txt_path}.txt", "a") as f:
                    f.write(("%g " * len(line)).rstrip() % line + "\n")

            if save_img or view_img:  # Add bbox to image
                label = f'{names[int(cls)]} {conf:.2f}'
                plot_one_box(xyxy, im0, label=label, color=colors[int(cls)], line_thickness=3)

Next Steps:

1. Run the updated script and check if the issue persists.
2. Provide a minimum reproducible example if the issue continues, so we can further investigate.

Thank you for your patience and cooperation. We're here to help! 😊