AttributeError: 'NoneType' object has no attribute 'serialize'

[cloudrivers]

Hi~
I use https://github.com/ultralytics/yolov3 to trained a 1 class model. when use onnx_to tensorrt.py, meet below error.

ubuntu:~/tensorrt_demos/yolov3_onnx$ python3 onnx_to_tensorrt.py
Loading ONNX file from path yolov3-416.onnx...
Beginning ONNX file parsing
Completed parsing of ONNX file
Building an engine; this may take a while...
[TensorRT] ERROR: ../builder/cudnnBuilderWeightConverters.cpp (555) - Misc Error in operator(): 1 (Weights are outside of fp16 range. A possible fix is to retrain the model with regularization to bring the magnitude of the weights down.)
[TensorRT] ERROR: ../builder/cudnnBuilderWeightConverters.cpp (555) - Misc Error in operator(): 1 (Weights are outside of fp16 range. A possible fix is to retrain the model with regularization to bring the magnitude of the weights down.)
Completed creating engine
Traceback (most recent call last):
File "onnx_to_tensorrt.py", line 119, in
main()
File "onnx_to_tensorrt.py", line 115, in main
_ = build_engine(onnx_file_path, engine_file_path, args.verbose)
File "onnx_to_tensorrt.py", line 99, in build_engine
f.write(engine.serialize())
AttributeError: 'NoneType' object has no attribute 'serialize'

[cloudrivers]

Hi~
I have disabled builder.fp16_mode = True, then the onnx can be converted to trt. but when i execute trt_yolov3.py, meet below errors:

Traceback (most recent call last):
File "trt_yolov3.py", line 96, in
main()
File "trt_yolov3.py", line 88, in main
loop_and_detect(cam, trt_yolov3, conf_th=0.3, vis=vis)
File "trt_yolov3.py", line 56, in loop_and_detect
boxes, confs, clss = trt_yolov3.detect(img, conf_th)
File "/home/michael/tensorrt_demos/utils/yolov3.py", line 473, in detect
in zip(trt_outputs, self.output_shapes)]
File "/home/michael/tensorrt_demos/utils/yolov3.py", line 472, in
trt_outputs = [output.reshape(shape) for output, shape
ValueError: cannot reshape array of size 3042 into shape (1,255,13,13)

[jkjung-avt]

Please refer to: http://disq.us/p/26kit21

It looks like your custom YOLOv3 model is only detecting 1 class of object. So you should modify "category_num" to 1, and 255 in "output_shapes" to 18.

p.s. (1 + 5) * 3 = 18

[cloudrivers]

Hi Jung, Thank you for the quick reply. you are correct. after change output_shapes to 18, it works. However i meet another issue as below. can you help on this?

demos$ python3 trt_yolov3.py --model yolov3-416 --image --filename 002.jpg
[array([-inf, -inf, -inf, ..., -inf, -inf, -inf], dtype=float32), array([nan, nan, nan, ..., nan, nan, nan], dtype=float32), array([nan, nan, nan, ..., nan, nan, nan], dtype=float32)]
/home/michael/tensorrt_demos/utils/yolov3.py:261: RuntimeWarning: invalid value encountered in greater_equal
pos = np.where(box_class_scores >= conf_th)
[array([-inf, -inf, -inf, ..., -inf, -inf, -inf], dtype=float32), array([nan, nan, nan, ..., nan, nan, nan], dtype=float32), array([nan, nan, nan, ..., nan, nan, nan], dtype=float32)]
[array([-inf, -inf, -inf, ..., -inf, -inf, -inf], dtype=float32), array([nan, nan, nan, ..., nan, nan, nan], dtype=float32), array([nan, nan, nan, ..., nan, nan, nan], dtype=float32)]
[array([-inf, -inf, -inf, ..., -inf, -inf, -inf], dtype=float32), array([nan, nan, nan, ..., nan, nan, nan], dtype=float32), array([nan, nan, nan, ..., nan, nan, nan], dtype=float32)]

[jkjung-avt]

Have you tested your model with darknet first? Does it produce the correct result?

It appears that the optimized TensorRT engine outputs -inf (infinite) and nan (not a number -> out of float32 range) with 002.jpg as the input image.

[cloudrivers]

the model is trained with pytorch and it can be used to inference with pytorch env. but have not check with darknet

[jkjung-avt]

So how did you convert the pytorch model to onnx? Were you able to verify the conversion is correct?

[cloudrivers]

ohh, I just use the code in your repo. yolov3_to_onnx.py and onnx_to_tensorrt.py.

[cloudrivers]

transfer model can't be used with this repo?

[jkjung-avt]

1. "yolov3_to_onnx.py" can only handle darknet models (.cfg & .weights files). In other words, it cannot handle pytorch models. (You said your model was trained with pytorch?)
2. If you train a custom YOLOv3 model with darknet, you should be able to use most of the code in this repository. I think you'd only need to modify "category_num", "output_shapes" and "output_tensor_dims" as I've listed in the Disqus post: http://disq.us/p/26kit21

[cloudrivers]

Hi Jung，Thank you so much for the kindly explainition. darknet model works well with your repo.

[jkjung-avt]

I've added a "--category_num" command-line option to make it easier to adapt my TensorRT YOLOv3 code to custom trained models. Please check out my blog post TensorRT YOLOv3 For Custom Trained Models for details.