[Detector Support]: Can my CPU use OpenVino?

[zeroarst]

Describe the problem you are having

Hi guys, I'm new to Frigate and currently still learning how to configure the detectors. I looked at the Object Detectors documentation and found that there are some options besides CPU. But it seems that my CPU and GPU are not supported.

I'm running Home Assistant on an Asus F555L laptop, which comes with an i5-5200U (GPU: Intel® HD Graphics 5500, Instruction Set Extensions Intel® SSE4.1, Intel® SSE4.2, Intel® AVX2 ) and Nvidia GeForce 920M.

I checked the OpenVino page, and it has CPU and GPU tabs. On the CPU tab, it requires 6th to 13th generation Intel Core processors. Althought the i5-5200U is 5th generation, it has AVX2? As I saw some people mention it requires AVAX so I am thinking maybe my CPU would work? Also, on the GPU tab, it says Intel® HD Graphics. Does this mean I can use OpenVinow ith integrated GPU in the CPU regardess what my CPU is?

Also, for using the Nvidia TensorRT Detector, it states that the CPU must support Compute Capability of 5.0 or greater, and by looking at this GPU Compute Capability, the 920M is 3.2. That means no chance at all to use it? It's also weird that the 910M is 5.2 whereas the 920M is 3.2.

I'm asking because although it seems that neither my CPU nor GPU supports OpenVino or Nvidia TensorRT, I still have a little hope it might be possible with some ways.

Thank you for any suggestions.

Version

0.13.2

Frigate config file

mqtt:
  enabled: False

cameras:
  reolink:
    enabled: True
    ffmpeg:
      inputs:
        - path: "rtsp://admin:xxxxxxx@192.168.1.104:554/Preview_01_main"
          roles:
            - record
            - detect
    detect:
      enabled: True
    record:
      enabled: True
      retain:
        days: 7
        mode: all
      events:
        retain:
          default: 30
          mode: motion
 detectors:
   ov:
     type: openvino
     device: AUTO
     model:
       path: /openvino-model/ssdlite_mobilenet_v2.xml

docker-compose file or Docker CLI command

NA

Relevant log output

NA

Operating system

HassOS

Install method

HassOS Addon

Coral version

CPU (no coral)

Any other information that may be helpful

No response

[NickM-27]

Some users have had openvino working with 5th Gen so might as well configure it and see if it works

[zeroarst]

That is good to know. I will give it a go...

[zeroarst]

I just added ov detector...

mqtt:
  enabled: False
cameras:
  reolink:
    enabled: True
    ffmpeg:
      inputs:
        - path: "rtsp://admin:xxxxx@192.168.1.104:554/Preview_01_main"
          roles:
            - record
            - detect
    detect:
      enabled: True
    record:
      enabled: True
      retain:
        days: 7
        mode: all
      events:
        retain:
          default: 30
          mode: motion
detectors:
   ov:
     type: openvino
     device: AUTO
     model:
       path: /openvino-model/ssdlite_mobilenet_v2.xml

There is an error..

2024-05-02 12:24:14.221287417  [2024-05-02 22:24:14] frigate.detectors.plugins.openvino INFO    : Model Input Shape: [1,300,300,3]
2024-05-02 12:24:14.221928435  [2024-05-02 22:24:14] frigate.detectors.plugins.openvino INFO    : Model Output-0 Shape: [1,1,100,7]
2024-05-02 12:24:14.223125354  [2024-05-02 22:24:14] frigate.detectors.plugins.openvino INFO    : Model has 1 Output Tensors
2024-05-02 12:24:14.245745662  Process detector:ov:
2024-05-02 12:24:14.323777427  Traceback (most recent call last):
2024-05-02 12:24:14.324259530    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-05-02 12:24:14.324263817      self.run()
2024-05-02 12:24:14.324266255    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-05-02 12:24:14.324272932      self._target(*self._args, **self._kwargs)
2024-05-02 12:24:14.324276442    File "/opt/frigate/frigate/object_detection.py", line 125, in run_detector
2024-05-02 12:24:14.324310153      detections = object_detector.detect_raw(input_frame)
2024-05-02 12:24:14.324313452    File "/opt/frigate/frigate/object_detection.py", line 75, in detect_raw
2024-05-02 12:24:14.324316737      return self.detect_api.detect_raw(tensor_input=tensor_input)
2024-05-02 12:24:14.324319424    File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 84, in detect_raw
2024-05-02 12:24:14.324321660      infer_request.infer([tensor_input])
2024-05-02 12:24:14.324324404    File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 152, in infer
2024-05-02 12:24:14.324359910      return super().infer(normalize_inputs(self, dict(enumerate(inputs))))
2024-05-02 12:24:14.324362944  RuntimeError: The input blob size is not equal to the network input size: got 307200 expecting

[NickM-27]

you are missing half of the configuration from the docs https://docs.frigate.video/configuration/object_detectors#openvino-detector

[zeroarst]

Thanks for that! Looks like it is working? I don't see erorrs in the log. If so that is great!! I actually tired that earlier and saw the same error. Thank you for pointing that out...I missed the model settings..

2024-05-02 12:33:57.163217787  [INFO] Preparing Frigate...
2024-05-02 12:33:57.182947490  [INFO] Starting Frigate...
2024-05-02 12:33:59.086691097  [2024-05-02 22:33:59] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
2024-05-02 12:33:59.086838970  [2024-05-02 22:33:59] frigate.app                    INFO    : Creating directory: /tmp/cache
2024-05-02 12:34:00.714706578  [2024-05-02 22:34:00] frigate.config                 WARNING : Customizing more than a detector model path is unsupported.
2024-05-02 12:34:00.730473632  [2024-05-02 22:34:00] peewee_migrate.logs            INFO    : Starting migrations
2024-05-02 12:34:00.739893578  [2024-05-02 22:34:00] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-05-02 12:34:00.747587061  [2024-05-02 22:34:00] frigate.app                    INFO    : Recording process started: 311
2024-05-02 12:34:00.753096265  [2024-05-02 22:34:00] frigate.app                    INFO    : go2rtc process pid: 89
2024-05-02 12:34:00.781785317  [2024-05-02 22:34:00] frigate.app                    INFO    : Output process started: 322
2024-05-02 12:34:00.783629233  [2024-05-02 22:34:00] detector.ov                    INFO    : Starting detection process: 320
2024-05-02 12:34:00.810803558  [2024-05-02 22:34:00] frigate.app                    INFO    : Camera processor started for reolink: 327
2024-05-02 12:34:00.818007522  [2024-05-02 22:34:00] frigate.app                    INFO    : Capture process started for reolink: 328
2024-05-02 12:34:01.176923651  [2024-05-02 22:34:01] frigate.detectors.plugins.openvino INFO    : Model Input Shape: [1,300,300,3]
2024-05-02 12:34:01.177392088  [2024-05-02 22:34:01] frigate.detectors.plugins.openvino INFO    : Model Output-0 Shape: [1,1,100,7]
2024-05-02 12:34:01.177397212  [2024-05-02 22:34:01] frigate.detectors.plugins.openvino INFO    : Model has 1 Output Tensors

[NickM-27]

yes, you should check the system page to see what the inference times are

[zeroarst]

So is it fair to say not necessarily need to be 6th generaiton Intel Core processors? As long as it supports AVAX? Not quiet sure if this is the key factor.

[NickM-27]

it depends on a lot of factors, that is a generalization of what is guaranteed by intel to work, more things outside of that work (including non intel cpus like AMD in cpu mode)