226-yolov7-optimization on Ubuntu

[bbartling]

When I run this notebook on Ubuntu with a successful setup of the virtual env and requirements.txt install...the kernel dies on my machine half way through every time...would you have tips to try?

Its this block of code towards the end...where it does run I can see the process go from 0 to 100% but after a 100% is met the Kernel dies and I cant make it any further.

mp, mr, map50, map, maps, num_images, labels = test(data=data, model=compiled_model, dataloader=dataloader, names=NAMES)
# Print results
s = ('%20s' + '%12s' * 6) % ('Class', 'Images', 'Labels', 'Precision', 'Recall', 'mAP@.5', 'mAP@.5:.95')
print(s)
pf = '%20s' + '%12i' * 2 + '%12.3g' * 4  # print format
print(pf % ('all', num_images, labels, mp, mr, map50, map))

Any options to try greatly appreciated.

[eaidova]

@bbartling which ubuntu and python version you use?
it looks like some problem with memory consumption on your machine which does not enough to finish metric calculation

could you please try to reduce number of samples?

replace

mp, mr, map50, map, maps, num_images, labels = test(data=data, model=compiled_model, dataloader=dataloader, names=NAMES)

with

mp, mr, map50, map, maps, num_images, labels = test(data=data, model=compiled_model, dataloader=dataloader, names=NAMES, num_samples=100)

[eaidova]

@akashAD98 which matplotlib version do you have?

[akashAD98]

@eaidova its 3.5.3

[eaidova]

@akashAD98 could you please update it? it looks like some internal bug in matplotlib of specific version. I tested with 3.6.2 and do not see this porblem

[bbartling]

Hi @eaidova

Using the mp, mr, map50, map, maps, num_images, labels = test(data=data, model=compiled_model, dataloader=dataloader, names=NAMES, num_samples=100) suggestion I get an error:

TypeError: test() got an unexpected keyword argument 'num_samples'

Should I add anything to the test function?

def test(data,
         model: Model,
         dataloader: torch.utils.data.DataLoader,
         conf_thres: float = 0.001,
         iou_thres: float = 0.65,  # for NMS
         single_cls: bool = False,
         v5_metric: bool = False,
         names: List[str] = None
        ):

[akashAD98]

@akashAD98 could you please update it? it looks like some internal bug in matplotlib of specific version. I tested with 3.6.2 and do not see this porblem

it solved my issue, thanks, Also is there any script for doing inference on video & webcam using .XML file? or converted format

[eaidova]

@bbartling ok, now I know that you use develop branch:) please switch to main, I introduced this functionality on it

[eaidova]

@akashAD98 glad to know that you solve the issue. Inside notebook you can see inference code for single image, which include pre processing, inference and postprocessing code. I think it should not be an issue just replace image reading with video capturing, right?

[akashAD98]

@eaidova
thre is detect function() ,how can i do process_image part for video ? its taking the number of frames .

    img = np.array(Image.open(image_path))
    preprocessed_img, orig_img = preprocess_image(img)

still let me try ,also Nncf is not supported, my purpose of using openvino to get higher FPS on CPU for video/webcam not for image, that's main part I'm missing here in this tutorial, NOT able to compare speed

[eaidova]

@eaidova thre is detect function() ,how can i do process_image part for video ? its taking the number of frames .

    img = np.array(Image.open(image_path))
    preprocessed_img, orig_img = preprocess_image(img)

still let me try ,also Nncf is not supported, my purpose of using openvino to get higher FPS on CPU for video/webcam not for image, that's main part I'm missing here in this tutorial, NOT able to compare speed

@akashAD98
video is a sequence of images so you need just replace image reading with video capturing and run process_image for each frame. We have not target to run it on video in this tutorial, because video reading is not our product specific (it is just utility how your obtain input data, it does not depend from itself model and inference engine). I believe you enough mature engineer to modify provided scripts and obtain result and measure preformance on own side (or at least to find any tutorials how to read video and combain with represented, for exampe this one):)
It is just tutorial, just for demonstration basic concepts, so it is NOT our target to get you ready to use solution for ANY cases.

What do you mean under nncf unsupported? It is tool which allow you to get quantized model representation, moving to low precision helps to make model more hardware-friendly and speedup it, so I think it can help get higher FPS on CPU no matter video or image you use as input

[akashAD98]

@eaidova yes I can do it, but just saying if you add this it would be easy for others. yesterday my NNCF import was not working, not its working fine. thanks for your quick reply

[eaidova]

@akashAD98 thank you for your feedback, possibly we consider to adding video inference in the tutorial in future (or having separated demo for it)

[akashAD98]

@eaidova im getting this issue while doing inference

[eaidova]

@akashAD98 are you sure that you specify postprocessing correctly?
It looks like shape of output does not correspond requested for reshape in your code.
How you export model to onnx?
If you follow tutorial in our repo, your model should contains --grid parameter whcih lead to injection of Detect layer from pytorch and model has only 1 output where this part of postprocessing already done.

[akashAD98]

@eaidova i tried to export it with --grid parameter only, still not able to do the process further.

[eaidova]

@akashAD98 I mean if you use --grid parameter, you have wrong postprocessing in your code, it assume that you have 3 outputs, while you have only one
if you use --grid parameter, please carefully look on detect function in the tutorial, it got only 1 output and then apply nms, no any addiitional actions like reshape, appling sigmod and get boxes from grid using anchors does not required, but you do it in your code

[eaidova]

@akashAD98 why you do not use prepare_input_tensor for video frames, you missing normalization input tensors, that is why you have no detections. Also each frame from video capture is BGR, you need to convert it do RGB in the same way how you do it in process_image.

can you highlight what things ii need to drop from yolov7.py to get it work?

Sorry, I have a lot of work and I think this is an inappropriate request to rewire someone else's code for making it working... especially, if it is not connected with tutorial content. Probably, your code works with your onnx model which was exported in another way, different from demonstrated in our tutorial, if you convert your model to IR using mo, your code will work with your model in this format, but from model provided in tutorial (no mattter onnx or xml) required different postprocessing

[akashAD98]

@eaidova i converted to rgb & also used prepare_input_tensor(), then what is wrong here

[eaidova]

@akashAD98 but it is too late

you run inference using async infer queue inside process_video
infer_queue.start_async({input_layer.any_name: img_batch}, (src_img_list, src_size))
in this case model get image without preprocessing
and process_video does not have return value, so input tensor is empty in your case here:
input_tensor = prepare_input_tensor(video_process)