Need help understanding how v8DetectionLoss works

[rootdrew27]

Hi.

I'm trying to modify the source code and I am having issues understanding how the 'batch' dictionary works. Specifically in relation to the 'targets' variable.

ultralytics/ultralytics/utils/loss.py

Line 223 in edca88d

targets = torch.cat((batch["batch_idx"].view(-1, 1), batch["cls"].view(-1, 1), batch["bboxes"]), 1)

I expected 'targets' to be a tensor containing indexes (i.e. information regarding what image in the batch it corresponds to), the ground truth class label, as well as the ground truth bboxes. Unfortunately, I was unable to relate these bbox coordinates to the batch['im_file'] images and I was unable to plot the ground truth bboxes (btw I am aware that I must resize the coordinates).

Sorry if this is a loaded question, I really appreciate any help though. Thanks!

[glenn-jocher]

@rootdrew27 hi there!

You're on the right track with your understanding. The 'batch' dictionary indeed contains the necessary information for processing targets. The 'targets' tensor is structured to include the image index, class labels, and bounding box coordinates. The image index helps in associating each target with the corresponding image in the batch.

To plot the ground truth bounding boxes, ensure that you correctly resize the coordinates to match the original image dimensions. If you're having trouble, double-check the preprocessing steps and scaling factors applied to the bounding boxes.

If the issue persists, try updating to the latest version of the Ultralytics package to ensure compatibility and bug fixes. Feel free to ask if you need further clarification!

Best of luck with your modifications!

[rootdrew27]

Thanks Glenn. I am still having trouble, so let me walkthrough what I'm doing in more detail:

I start debugging, my breakpoint hits at line 223 in loss.py (as referenced above). I step forward to line 225 (thus 'targets' has been preprocessed). The shape of 'targets' is (16, 3, 5) which I presume is because I have a batch_size of 16, a single class, and this batch has a max of 3 objects.

Anyways, I take the first image path from batch['im_file'] and I use opencv to display it, after adding a rectangle using the coordinates from the first (3,5) tensor in 'targets'.

image = cv2.resize(image, (640,640))

x1, y1, x2, y2 = (580.9771,  31.3396, 594.7000,  37.8399)

cv2.rectangle(image, (int(x1), int(y1)), (int(x2), int(y2)), (0,0,255), 2)

cv2.imshow('Image with Bounding Box', image)
cv2.waitKey(0)
cv2.destroyAllWindows()

I verified that the images are being resized to (640, 640) and this makes sense with the current bounding box values.

The issue is of course that the displayed bounding box is nowhere near the object. And I did verify that the corresponding label file, does produce the correct bounding box.

Thank you again for your help.

[rootdrew27]

Also, I'd like to mention that I have set augment = false in my data.yaml file and I verified that args.multi_scale is false

ultralytics/ultralytics/models/yolo/detect/train.py

Line 60 in edca88d

if self.args.multi_scale:

I am assuming that some sort of augmentation is occurring but maybe this is an erroneous assumption.

[glenn-jocher]

Thanks for the additional details! Given that you've set augment = false and verified args.multi_scale is false, it seems augmentation shouldn't be affecting your bounding boxes. However, ensure you're using the latest version of the Ultralytics package to rule out any bugs. If the issue persists, double-check the preprocessing steps and scaling factors applied to the bounding boxes. If you need further assistance, feel free to ask!

[rootdrew27]

I tested the code that I copied (and slightly modified) and it worked fine. I might've been making a silly mistake. Thanks for your speedy replies. Appreciate ya.

[glenn-jocher]

Glad to hear it worked out for you! It's easy to overlook small details, especially when modifying code. If you run into any more issues or have further questions, feel free to reach out. Happy coding!