some errors for yolov8

[meaquanana]

Here is ultralytics export demo : yolo export model=yolov8n.pt format=openvino int8=True.
So i used this by got some errors :

OpenVINO: starting export with openvino 2024.1.0-15008-f4afc983258-releases/2024/1...
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, onnx, openvino
OpenVINO: collecting INT8 calibration images from 'data=/root/ultralytics-main_old/dataset/data.yaml'
Scanning /root/obb/labels/val... 182 images, 1 backgrounds, 0 corrupt: 100%|██████████| 182/182 [00:00<00:00, 574.48it/s]
New cache created: /root/obb/labels/val.cache
OpenVINO: WARNING ⚠️ >300 images recommended for INT8 calibration, found 182 images.
INFO:nncf:11 ignored nodes were found by types in the NNCFGraph
INFO:nncf:Not adding activation input quantizer for operation: 171 /model.22/Sigmoid
INFO:nncf:Not adding activation input quantizer for operation: 182 Multiply_4113
193 /model.22/Mul

INFO:nncf:Not adding activation input quantizer for operation: 192 /model.22/Sigmoid_1
INFO:nncf:Not adding activation input quantizer for operation: 251 /model.22/Sub_1
258 /model.22/Div

INFO:nncf:Not adding activation input quantizer for operation: 212 /model.22/Mul_1
INFO:nncf:Not adding activation input quantizer for operation: 213 /model.22/Mul_4
INFO:nncf:Not adding activation input quantizer for operation: 214 /model.22/Mul_2
INFO:nncf:Not adding activation input quantizer for operation: 215 /model.22/Mul_3
INFO:nncf:Not adding activation input quantizer for operation: 223 /model.22/Sub_2
INFO:nncf:Not adding activation input quantizer for operation: 259 /model.22/Mul_5
OpenVINO: export failure ❌ 3.0s: list index out of range
Traceback (most recent call last):
  File "export.py", line 9, in <module>
    model.export(format='openvino',int8=True,data='/root/ultralytics-main_old/dataset/data.yaml')
  File "/root/ultralytics-main/ultralytics/engine/model.py", line 539, in export
    return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model)
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/root/ultralytics-main/ultralytics/engine/exporter.py", line 279, in __call__
    f[3], _ = self.export_openvino()
  File "/root/ultralytics-main/ultralytics/engine/exporter.py", line 136, in outer_func
    raise e
  File "/root/ultralytics-main/ultralytics/engine/exporter.py", line 131, in outer_func
    f, model = inner_func(*args, **kwargs)
  File "/root/ultralytics-main/ultralytics/engine/exporter.py", line 458, in export_openvino
    quantized_ov_model = nncf.quantize(
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/quantization/quantize_model.py", line 151, in quantize
    return quantize_impl(
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/openvino/quantization/quantize_model.py", line 326, in quantize_impl
    return quantize_fn(
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/telemetry/decorator.py", line 72, in wrapped
    retval = fn(*args, **kwargs)
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/openvino/quantization/quantize_model.py", line 143, in native_quantize_impl
    quantized_model = quantization_algorithm.apply(model, graph, dataset=calibration_dataset)
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/quantization/algorithms/post_training/algorithm.py", line 112, in apply
    return self._pipeline.run_from_step(model, dataset, graph, 0, step_index_to_statistics)
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/quantization/algorithms/pipeline.py", line 160, in run_from_step
    statistic_points = self.get_statistic_points_for_step(step_index, step_model, step_graph)
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/quantization/algorithms/pipeline.py", line 185, in get_statistic_points_for_step
    for statistic_points in algorithm.get_statistic_points(model, graph).values():
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/quantization/algorithms/min_max/algorithm.py", line 938, in get_statistic_points
    stat_collector = self._get_stat_collector(
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/quantization/algorithms/min_max/algorithm.py", line 420, in _get_stat_collector
    shape = self._backend_entity.get_target_point_shape(graph, node, target_point)
  File "/root/miniconda3/envs/yolov8/lib/python3.8/site-packages/nncf/quantization/algorithms/min_max/openvino_backend.py", line 145, in get_target_point_shape
    return nncf_graph.get_input_edges(node)[target_point.port_id].tensor_shape
IndexError: list index out of range

[alexsu52]

@daniil-lyakhov, could you take a look?

[daniil-lyakhov]

Hello @meaquanana,

I could not reproduce your error with the latest ultralytics update:

yolo export model=yolov8n.pt format=openvino int8=True
Ultralytics YOLOv8.2.10 🚀 Python-3.8.10 torch-2.1.2+cu121 CPU (Intel Core(TM) i9-10980XE 3.00GHz)
WARNING ⚠️ INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco8.yaml'.
YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs

PyTorch: starting from 'yolov8n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (6.2 MB)

OpenVINO: starting export with openvino 2024.1.0-15008-f4afc983258-releases/2024/1...
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino
OpenVINO: collecting INT8 calibration images from 'data=coco8.yaml'
Scanning /home/dlyakhov/Projects/datasets/coco8/labels/val.cache... 4 images, 0 backgrounds, 0 corrupt: 100%|██████████| 4/4 [00:00<?, ?it/s]
OpenVINO: WARNING ⚠️ >300 images recommended for INT8 calibration, found 4 images.
INFO:nncf:24 ignored nodes were found by patterns in the NNCFGraph
INFO:nncf:1 ignored nodes were found by types in the NNCFGraph
INFO:nncf:Not adding activation input quantizer for operation: 162 __module.model.22.dfl/aten::view/Reshape
INFO:nncf:Not adding activation input quantizer for operation: 163 __module.model.22/aten::sigmoid/Sigmoid
INFO:nncf:Not adding activation input quantizer for operation: 173 __module.model.22.dfl/aten::transpose/Transpose
INFO:nncf:Not adding activation input quantizer for operation: 182 __module.model.22.dfl/aten::softmax/Softmax
INFO:nncf:Not adding activation input quantizer for operation: 190 __module.model.22.dfl.conv/aten::_convolution/Convolution
INFO:nncf:Not adding activation input quantizer for operation: 189 __module.model.22.dfl/aten::view/Reshape_1
INFO:nncf:Not adding activation input quantizer for operation: 203 __module.model.22/aten::sub/Subtract
INFO:nncf:Not adding activation input quantizer for operation: 204 __module.model.22/aten::add/Add
INFO:nncf:Not adding activation input quantizer for operation: 212 __module.model.22/aten::add/Add_1
221 __module.model.22/aten::div/Divide

INFO:nncf:Not adding activation input quantizer for operation: 213 __module.model.22/aten::sub/Subtract_1
INFO:nncf:Not adding activation input quantizer for operation: 229 __module.model.22/aten::mul/Multiply
Statistics collection ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 4/4 • 0:00:00 • 0:00:00
Applying Fast Bias correction ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 63/63 • 0:00:02 • 0:00:00
OpenVINO: export success ✅ 7.4s, saved as 'yolov8n_int8_openvino_model/' (3.6 MB)

Export complete (8.8s)
Results saved to /home/dlyakhov/Projects/tmp
Predict:         yolo predict task=detect model=yolov8n_int8_openvino_model imgsz=640 int8 
Validate:        yolo val task=detect model=yolov8n_int8_openvino_model imgsz=640 data=coco.yaml int8 
Visualize:       https://netron.app
💡 Learn more at https://docs.ultralytics.com/modes/export

Versions:

ultralytics==8.2.10
openvino==2024.1.0
nncf==2.10.0

Please try to update your environment:

pip install -U "ultralytics[export]" nncf

[Whitefish-by]

I got some errors when I run this commend model.export(format="openvino", imgsz=[832,1248], rect=True, dynamic=True ,half=True, simplify=True, int8=int8) which the model to export is a Mono classification model. But when I try exporting a muti classification model, the code runs successfully.
Appreciate for helping me!

$ python  pt2OpenVINO.py --model_path "runs/0823/finetune_mouse_bite_v1_new/weights/epoch20.pt" --int8
/home/um202276680/yolov10/yolov10/ultralytics/nn/tasks.py:733: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  ckpt = torch.load(file, map_location="cpu")
Ultralytics YOLOv8.1.34 🚀 Python-3.9.19 torch-2.4.0+cu121 CPU (Intel Xeon Gold 6330 2.00GHz)
YOLOv10m summary (fused): 369 layers, 16451542 parameters, 0 gradients

PyTorch: starting from 'runs/0823/finetune_mouse_bite_v1_new/weights/epoch20.pt' with input shape (1, 3, 832, 1248) BCHW and output shape(s) (1, 300, 6) (190.1 MB)

OpenVINO: starting export with openvino 2024.3.0-16041-1e3b88e4e3f-releases/2024/3...
OpenVINO: WARNING ⚠️ INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco128.yaml'.
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, onnx, openvino
OpenVINO: collecting INT8 calibration images from 'data=coco128.yaml'
Scanning /home/um202276680/yolov10/datasets/coco128/labels/train2017.cache... 126 images, 2 backgrounds, 0 corrupt: 100%|██████████| 128/128 [
OpenVINO: WARNING ⚠️ >300 images recommended for INT8 calibration, found 128 images.
OpenVINO: export failure ❌ 5.8s: No matches for ignored patterns ['.*model.23/.*/Div*'] in the NNCFGraph. Refer to the original_graph.dot to discover the operationsin the model currently visible to NNCF and specify the ignored/target scopes in terms of the names there.
Traceback (most recent call last):
  File "/home/um202276680/yolov10/yolov10/pt2OpenVINO.py", line 44, in <module>
    res = model.export(format="openvino", imgsz=[832,1248], rect=True, dynamic=True ,half=True, simplify=True, int8=int8)
  File "/home/um202276680/yolov10/yolov10/ultralytics/engine/model.py", line 590, in export
    return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model)
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/um202276680/yolov10/yolov10/ultralytics/engine/exporter.py", line 292, in __call__
    f[3], _ = self.export_openvino()
  File "/home/um202276680/yolov10/yolov10/ultralytics/engine/exporter.py", line 138, in outer_func
    raise e
  File "/home/um202276680/yolov10/yolov10/ultralytics/engine/exporter.py", line 133, in outer_func
    f, model = inner_func(*args, **kwargs)
  File "/home/um202276680/yolov10/yolov10/ultralytics/engine/exporter.py", line 487, in export_openvino
    quantized_ov_model = nncf.quantize(
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/quantization/quantize_model.py", line 98, in quantize
    return quantize_impl(
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/openvino/quantization/quantize_model.py", line 349, in quantize_impl
    return quantize_fn(
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/telemetry/decorator.py", line 72, in wrapped
    retval = fn(*args, **kwargs)
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/openvino/quantization/quantize_model.py", line 164, in native_quantize_impl
    quantized_model = quantization_algorithm.apply(model, graph, dataset=calibration_dataset)
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/quantization/algorithms/post_training/algorithm.py", line 112, in apply
    return self._pipeline.run_from_step(model, dataset, graph, 0, step_index_to_statistics)
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/quantization/algorithms/pipeline.py", line 160, in run_from_step
    statistic_points = self.get_statistic_points_for_step(step_index, step_model, step_graph)
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/quantization/algorithms/pipeline.py", line 185, in get_statistic_points_for_step
    for statistic_points in algorithm.get_statistic_points(model, graph).values():
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/quantization/algorithms/min_max/algorithm.py", line 859, in get_statistic_points
    quantization_target_points, _ = self._get_quantization_target_points(model, graph)
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/quantization/algorithms/min_max/algorithm.py", line 641, in _get_quantization_target_points
    quantizer_setup = self._get_quantizer_setup(nncf_graph, inference_nncf_graph, hw_patterns, ignored_patterns)
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/quantization/algorithms/min_max/algorithm.py", line 497, in _get_quantizer_setup
    ignored_names = self._get_ignored_names(nncf_graph, inference_nncf_graph, ignored_patterns)
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/quantization/algorithms/min_max/algorithm.py", line 404, in _get_ignored_names
    user_ignored_names = get_ignored_node_names_from_ignored_scope(
  File "/home/um202276680/anaconda3/envs/yolov10/lib/python3.9/site-packages/nncf/scopes.py", line 136, in get_ignored_node_names_from_ignored_scope
    raise RuntimeError(f"No matches for ignored patterns {not_matched_patterns} in the NNCFGraph. " + error_msg)
RuntimeError: No matches for ignored patterns ['.*model.23/.*/Div*'] in the NNCFGraph. Refer to the original_graph.dot to discover the operationsin the model currently visible to NNCF and specify the ignored/target scopes in terms of the names there.

[daniil-lyakhov]

Hello, @Whitefish-by, could you please specify the model name and version of the ultralytics?
First of all, please try to update ultralitics version.
The error is coming from this part of the code https://github.com/ultralytics/ultralytics/blob/main/ultralytics/engine/exporter.py#L494-L501, it is either wrong names in the model or wrong ignored scope patters

[daniil-lyakhov]

Could not reproduce:

$yolo export model=yolov8n.pt format=openvino int8=True imgsz=[832,1248] rect=True dynamic=True half=True simplify=True
Ultralytics YOLOv8.2.82 🚀 Python-3.8.10 torch-2.4.0+cu121 CPU (Intel Core(TM) i9-10980XE 3.00GHz)
WARNING ⚠️ half=True and int8=True are mutually exclusive, setting half=False.
WARNING ⚠️ INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco8.yaml'.
YOLOv8n summary (fused): 168 layers, 3,151,904 parameters, 0 gradients, 8.7 GFLOPs

PyTorch: starting from 'yolov8n.pt' with input shape (1, 3, 832, 1248) BCHW and output shape(s) (1, 84, 21294) (6.2 MB)

OpenVINO: starting export with openvino 2024.3.0-16041-1e3b88e4e3f-releases/2024/3...
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, openvino
OpenVINO: collecting INT8 calibration images from 'data=coco8.yaml'
Scanning /home/dlyakhov/Projects/nncf/datasets/coco8/labels/val.cache... 4 images, 0 backgrounds, 0 corrupt: 100%|██████████| 4/4 [00:00<?, ?it/s]
OpenVINO: WARNING ⚠️ >300 images recommended for INT8 calibration, found 4 images.
INFO:nncf:24 ignored nodes were found by patterns in the NNCFGraph
INFO:nncf:1 ignored nodes were found by types in the NNCFGraph
INFO:nncf:Not adding activation input quantizer for operation: 166 __module.model.22.dfl/aten::view/Reshape
INFO:nncf:Not adding activation input quantizer for operation: 167 __module.model.22/aten::sigmoid/Sigmoid
INFO:nncf:Not adding activation input quantizer for operation: 180 __module.model.22.dfl/aten::transpose/Transpose
INFO:nncf:Not adding activation input quantizer for operation: 191 __module.model.22.dfl/aten::softmax/Softmax
INFO:nncf:Not adding activation input quantizer for operation: 201 __module.model.22.dfl.conv/aten::_convolution/Convolution
INFO:nncf:Not adding activation input quantizer for operation: 208 __module.model.22.dfl/aten::view/Reshape_1
INFO:nncf:Not adding activation input quantizer for operation: 225 __module.model.22/aten::sub/Subtract
INFO:nncf:Not adding activation input quantizer for operation: 226 __module.model.22/aten::add/Add_6
INFO:nncf:Not adding activation input quantizer for operation: 235 __module.model.22/aten::add/Add_7
244 __module.model.22/aten::div/Divide

INFO:nncf:Not adding activation input quantizer for operation: 236 __module.model.22/aten::sub/Subtract_1
INFO:nncf:Not adding activation input quantizer for operation: 214 __module.model.22/aten::mul/Multiply_3
Statistics collection ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 2/2 • 0:00:00 • 0:00:00
Applying Fast Bias correction ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 63/63 • 0:00:02 • 0:00:00
OpenVINO: export success ✅ 7.1s, saved as 'yolov8n_int8_openvino_model/' (3.5 MB)

Export complete (8.7s)
Results saved to /home/dlyakhov/tmp/tmp
Predict:         yolo predict task=detect model=yolov8n_int8_openvino_model imgsz=832,1248 int8 
Validate:        yolo val task=detect model=yolov8n_int8_openvino_model imgsz=832,1248 data=coco.yaml int8 WARNING ⚠️ non-PyTorch val requires square images, 'imgsz=[832, 1248]' will not work. Use export 'imgsz=1248' if val is required.
Visualize:       https://netron.app
💡 Learn more at https://docs.ultralytics.com/modes/export

Versions:

ultralytics==8.2.82
openvino==2024.3.0
nncf==2.12.0

Please update your environment by

pip install -U ultralytics openvino nncf

[Whitefish-by]

@daniil-lyakhov Thank you very much! I upgrade the ultralitics to the latest version and my the code runs successfully!