Issue when try to validate openvino format model

[FrancoArtale]

Search before asking

• I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

Validation

Bug

The next script, to validate a trained yolov5 works well:
!python ./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/val.py --weights yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/runs/train/exp5/weights/best.pt --imgsz 1280 --data "./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/100KBDD/data.yaml"

To convert to onnx format i use (works well):
!python ./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/export.py --weights ./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/runs/train/exp5/weights/best.pt --include onnx --imgsz 736 1280 --data ./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/100KBDD/data.yaml --batch-size 1

To validate the onnx i used:
!python ./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/val.py --weights yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/runs/train/exp5/weights/best.onnx --imgsz 1280 --data "./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/100KBDD/data.yaml" --batch-size 1

here there is a problem:
Traceback (most recent call last):
File "c:\Users\franco\OneDrive\Escritorio\Final_project\yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\val.py", line 438, in
main(opt)
File "c:\Users\franco\OneDrive\Escritorio\Final_project\yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\val.py", line 409, in main
run(**vars(opt))
File "c:\Users\franco\OneDrive\Escritorio\Final_project\project\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "c:\Users\franco\OneDrive\Escritorio\Final_project\yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\val.py", line 236, in run
preds, train_out = model(im) if compute_loss else (model(im, augment=augment), None)
File "c:\Users\franco\OneDrive\Escritorio\Final_project\project\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "c:\Users\franco\OneDrive\Escritorio\Final_project\project\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "c:\Users\franco\OneDrive\Escritorio\Final_project\yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\models\common.py", line 666, in forward
y = self.session.run(self.output_names, {self.session.get_inputs()[0].name: im})
File "c:\Users\franco\OneDrive\Escritorio\Final_project\project\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 220, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: images for the following indices
index: 2 Got: 1280 Expected: 736
Please fix either the inputs/outputs or the model.

But val.py doesn't accept --rect parameter. How do i fix this?

For last, i used openvino to quantized my model, nncf, etc.
Then when I tried to validate this new model int8, we have:

!python ./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/val.py --weights yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/runs/train/exp5/weights/int8_openvino_model --imgsz 1280 --data "./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/100KBDD/data.yaml" --batch-size 1

The problem here is different:
val: data=./yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/100KBDD/data.yaml, weights=['yolov5_train100kbdd/yolov5s_originsize_300epochs_lr_001/runs/train/exp5/weights/int8_openvino_model'], batch_size=1, imgsz=1280, conf_thres=0.001, iou_thres=0.6, max_det=300, task=val, device=, workers=8, single_cls=False, augment=False, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=False, project=yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\runs\val, name=exp, exist_ok=False, half=False, dnn=False
YOLOv5 v7.0-294-gdb125a20 Python-3.10.9 torch-2.2.2+cpu CPU

Loading yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\runs\train\exp5\weights\int8_openvino_model for OpenVINO inference...
Traceback (most recent call last):
File "c:\Users\franco\OneDrive\Escritorio\Final_project\yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\val.py", line 438, in
main(opt)
File "c:\Users\franco\OneDrive\Escritorio\Final_project\yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\val.py", line 409, in main
run(**vars(opt))
File "c:\Users\franco\OneDrive\Escritorio\Final_project\project\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "c:\Users\franco\OneDrive\Escritorio\Final_project\yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\val.py", line 165, in run
model = DetectMultiBackend(weights, device=device, dnn=dnn, data=data, fp16=half)
File "c:\Users\franco\OneDrive\Escritorio\Final_project\yolov5_train100kbdd\yolov5s_originsize_300epochs_lr_001\models\common.py", line 643, in init
if names[0] == "n01440764" and len(names) == 1000: # ImageNet
TypeError: 'NoneType' object is not subscriptable

There is a issue similar, #10180.

But my data.yaml worked well in the first case, why does it happen?

data.yaml:
#test: 100KBDD/test/images
train: 100KBDD/train/images
val: 100KBDD/valid/images

names:
0: 0
1: 1
2: 2
3: 3
4: 4
5: 5
6: 6
7: 7
nc: 8
#roboflow:

license: CC BY 4.0

project: car_part2

url: https://universe.roboflow.com/carpart2-gj01d/car_part2/dataset/1

version: 1

workspace: carpart2-gj01d

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[glenn-jocher]

@FrancoArtale hello! It looks like you've encountered a couple of issues during your validation process after converting to ONNX and OpenVINO formats.

1. ONNX Model Image Size Mismatch:
   Your trace log indicates an image dimension mismatch during validation. When exporting the model using the export.py script to ONNX, you've defined the image size as 736x1280. However, during validation, you are attempting to use an image size of 1280x1280. Ensure the image dimensions match throughout the conversion and validation processes to resolve this.

2. OpenVINO Model Validation Issue:
   For the OpenVINO model, the error related to "NoneType object is not subscriptable" typically points towards an issue with loading the class names or other essential data properties. Ensure that the configuration in data.yaml is correctly loaded and accessible by the model during validation. Double-check the paths and formatting within your data.yaml file.

If these tips don't resolve the issues, providing a more detailed error output or further context may help pinpoint the specific cause. Thanks for your detailed inquiry, and good luck with your further YOLOv5 deployments! 😊

[FrancoArtale]

1. I'm using images of 1280x736, the images are okey:
   
   The problem is in the command, I used --imgsz 1280 and i didn't add 736 because it's not possible. There is no --rect parameter in val.py, so it's not possible put --imgsz 736 1280.

If you see the next line from val.py, it's only accept one value:
parser.add_argument("--imgsz", "--img", "--img-size", type=int, default=640, help="inference size (pixels)")

2. My data.yaml is:

#test: 100KBDD/test/images
train: 100KBDD/train/images
val: 100KBDD/valid/images

names:
0: 0
1: 1
2: 2
3: 3
4: 4
5: 5
6: 6
7: 7
nc: 8
#roboflow:
#license: CC BY 4.0
#project: car_part2
#url: https://universe.roboflow.com/carpart2-gj01d/car_part2/dataset/1
#version: 1
#workspace: carpart2-gj01d

It worked in other cases:
In training of the model.
In the validation of the original model.

[glenn-jocher]

Hey @FrancoArtale! Thanks for the additional details. 😊

1. Image Size in Validation: The val.py script indeed only accepts a single integer for --imgsz, which sets both width and height to the same value. If your model was trained or exported with non-square dimensions (like 736x1280), you'll need to modify the validation script to accept two dimensions or adjust your model to work with square input sizes. This limitation in val.py is by design to simplify the input size handling.

2. Data.yaml Usage: It's great to hear that your data.yaml works well in other scenarios. If it's failing in specific cases (like with OpenVINO), the issue might be related to how the model or the validation script handles the loaded configuration. Ensure that the paths and format are consistently correct across different environments or setups.

For handling different aspect ratios during validation without modifying the script, consider resizing your images to square dimensions before validation as a workaround. Keep up the great work with YOLOv5! 🚀