A question about validation set drawing image results

[Love-syntacticSugar]

Search before asking

• I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

When I used the validation method on the DOTAv1.0 validation set, I got incorrect image results either on my own computer or on the kaggle cloud server. Here's my running code:

from ultralytics import YOLO
model = YOLO('yolov8l-obb.pt', task='obb')
results = model.val(data='datav1.yaml', batch=2, device='cpu', imgsz=1024, split='val')

With batch_size of 2, this is what I get:

After looking at the v8 source code in detail, I used pycharm to debug, placing breakpoints at key code points, as shown below:

After debugging, I found that the result was surprisingly normal! As shown in the picture below:

Why did this situation occur? After thinking for a long time, I found that the plot_images method used the decorator @threaded in the plot_val_samples method. I attributed the problem to thread-related reasons, so I passed the threaded=False parameter to the plot_val_samples method but I didn't do the same thing for plot_predictions method. I found that the result did become correct！！It seems that dual threading works correctly (one thread for the main method and another for the plot_predictions method). However, with three threads (main + plot_val_samples + plot_predictions), errors occur.But fortunately, the results are normal now.

However, due to my limited knowledge of thread-related issues, I am not sure what consequences this approach will bring. It is well known that the introduction of threads can ensure the safety of the program. I also understand that the introduction of threads here is to separate the main program from the drawing of images, and to speed up the execution of the main program. But when I set the threaded parameter to False, it will share a thread. Besides the slowdown, could there be potential security issues? I don't care about the speed issue, but I care about the security issue.

In addition, besides setting threaded=False, are there any other methods to solve this problem?

Additional

Furthermore, it is worth mentioning that: In order to produce the desired image results, I made some modifications to the v8 source code. However, I also tested on the original v8 source code, and encountered the same issue mentioned above. So, I believe the problem is not caused by my personal code modifications.

[github-actions]

👋 Hello @Love-syntacticSugar, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@Love-syntacticSugar hello! It's great to see you've done a thorough investigation into the threading issue with the plot_images method. 👍

Setting the threaded=False parameter indeed forces the plotting to run on the main thread, which can help avoid issues related to thread safety and data consistency, especially in environments where the Python GIL (Global Interpreter Lock) might cause unexpected behavior when using multiple threads.

Regarding security concerns, using a single thread by setting threaded=False generally doesn't introduce security risks by itself. It primarily affects performance due to sequential execution. The main risk in multithreading environments typically involves data corruption or race conditions, not security vulnerabilities.

If you prefer not to modify the threading behavior, another approach could be to ensure that all data passed between threads is properly synchronized or that thread-safe data structures are used. However, this might require more in-depth changes to the codebase.

If the current solution of setting threaded=False works for you without significant performance drawbacks, it's a valid approach to stick with it, especially if it maintains the stability of your validation outputs.

Keep exploring and feel free to reach out if you have more questions! 🚀

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐