Is a i7-6700T CPU too old to use with OpenVINO ?

[noliveleger]

I've installed immich (1.106.4) and set the image learning container (IMO) correctly.

I have errors while looking at logs in the image learning container.

immich_machine_learning  | [06/15/24 17:25:11] INFO     Starting gunicorn 22.0.0
immich_machine_learning  | [06/15/24 17:25:11] INFO     Listening at: http://[::]:3003 (8)
immich_machine_learning  | [06/15/24 17:25:11] INFO     Using worker: app.config.CustomUvicornWorker
immich_machine_learning  | [06/15/24 17:25:11] INFO     Booting worker with pid: 9
immich_machine_learning  | [06/15/24 17:25:17] INFO     Started server process [9]
immich_machine_learning  | [06/15/24 17:25:17] INFO     Waiting for application startup.
immich_machine_learning  | [06/15/24 17:25:17] INFO     Created in-memory cache with unloading after 300s
immich_machine_learning  |                              of inactivity.
immich_machine_learning  | [06/15/24 17:25:17] INFO     Initialized request thread pool with 8 threads.
immich_machine_learning  | [06/15/24 17:25:17] INFO     Application startup complete.
immich_machine_learning  | [06/15/24 17:26:43] INFO     Setting 'ViT-B-32__openai' execution providers to
immich_machine_learning  |                              ['OpenVINOExecutionProvider',
immich_machine_learning  |                              'CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_machine_learning  | [06/15/24 17:26:43] INFO     Loading visual model 'ViT-B-32__openai' to memory
immich_machine_learning  | [06/15/24 17:28:59] INFO     Setting 'buffalo_l' execution providers to
immich_machine_learning  |                              ['OpenVINOExecutionProvider',
immich_machine_learning  |                              'CPUExecutionProvider'], in descending order of
immich_machine_learning  |                              preference
immich_machine_learning  | [06/15/24 17:28:59] INFO     Loading detection model 'buffalo_l' to memory
immich_machine_learning  | [06/15/24 17:29:01] ERROR    Exception in ASGI application
immich_machine_learning  |
immich_machine_learning  |                              ╭─────── Traceback (most recent call last) ───────╮
immich_machine_learning  |                              │ /usr/src/app/main.py:151 in predict             │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   148 │   │   inputs = text                     │
immich_machine_learning  |                              │   149 │   else:                                 │
immich_machine_learning  |                              │   150 │   │   raise HTTPException(400, "Either  │
immich_machine_learning  |                              │ ❱ 151 │   response = await run_inference(inputs │
immich_machine_learning  |                              │   152 │   return ORJSONResponse(response)       │
immich_machine_learning  |                              │   153                                           │
immich_machine_learning  |                              │   154                                           │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/main.py:174 in run_inference       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   171 │   │   response[entry["task"]] = output  │
immich_machine_learning  |                              │   172 │                                         │
immich_machine_learning  |                              │   173 │   without_deps, with_deps = entries     │
immich_machine_learning  |                              │ ❱ 174 │   await asyncio.gather(*[_run_inference │
immich_machine_learning  |                              │   175 │   if with_deps:                         │
immich_machine_learning  |                              │   176 │   │   await asyncio.gather(*[_run_infer │
immich_machine_learning  |                              │   177 │   if isinstance(payload, Image):        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/main.py:169 in _run_inference      │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   166 │   │   │   │   message = f"Task {entry[' │
immich_machine_learning  |                              │       output of {dep}"                          │
immich_machine_learning  |                              │   167 │   │   │   │   raise HTTPException(400,  │
immich_machine_learning  |                              │   168 │   │   model = await load(model)         │
immich_machine_learning  |                              │ ❱ 169 │   │   output = await run(model.predict, │
immich_machine_learning  |                              │   170 │   │   outputs[model.identity] = output  │
immich_machine_learning  |                              │   171 │   │   response[entry["task"]] = output  │
immich_machine_learning  |                              │   172                                           │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/main.py:187 in run                 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   184 │   if thread_pool is None:               │
immich_machine_learning  |                              │   185 │   │   return func(*args, **kwargs)      │
immich_machine_learning  |                              │   186 │   partial_func = partial(func, *args, * │
immich_machine_learning  |                              │ ❱ 187 │   return await asyncio.get_running_loop │
immich_machine_learning  |                              │   188                                           │
immich_machine_learning  |                              │   189                                           │
immich_machine_learning  |                              │   190 async def load(model: InferenceModel) ->  │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/lib/python3.10/concurrent/futures/thread.p │
immich_machine_learning  |                              │ y:58 in run                                     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/base.py:61 in predict       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    58 │   │   self.load()                       │
immich_machine_learning  |                              │    59 │   │   if model_kwargs:                  │
immich_machine_learning  |                              │    60 │   │   │   self.configure(**model_kwargs │
immich_machine_learning  |                              │ ❱  61 │   │   return self._predict(*inputs, **m │
immich_machine_learning  |                              │    62 │                                         │
immich_machine_learning  |                              │    63 │   @abstractmethod                       │
immich_machine_learning  |                              │    64 │   def _predict(self, *inputs: Any, **mo │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/facial_recognition/detectio │
immich_machine_learning  |                              │ n.py:37 in _predict                             │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   34 │   def _predict(self, inputs: NDArray[np. │
immich_machine_learning  |                              │      FaceDetectionOutput:                       │
immich_machine_learning  |                              │   35 │   │   inputs = decode_cv2(inputs)        │
immich_machine_learning  |                              │   36 │   │                                      │
immich_machine_learning  |                              │ ❱ 37 │   │   bboxes, landmarks = self._detect(i │
immich_machine_learning  |                              │   38 │   │   return {                           │
immich_machine_learning  |                              │   39 │   │   │   "boxes": bboxes[:, :4].round() │
immich_machine_learning  |                              │   40 │   │   │   "scores": bboxes[:, 4],        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/facial_recognition/detectio │
immich_machine_learning  |                              │ n.py:45 in _detect                              │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   42 │   │   }                                  │
immich_machine_learning  |                              │   43 │                                          │
immich_machine_learning  |                              │   44 │   def _detect(self, inputs: NDArray[np.u │
immich_machine_learning  |                              │      NDArray[np.float32]]:                      │
immich_machine_learning  |                              │ ❱ 45 │   │   return self.model.detect(inputs)   │
immich_machine_learning  |                              │   46 │                                          │
immich_machine_learning  |                              │   47 │   def configure(self, **kwargs: Any) ->  │
immich_machine_learning  |                              │   48 │   │   self.model.det_thresh = kwargs.pop │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning  |                              │ ace/model_zoo/retinaface.py:224 in detect       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   221 │   │   det_img = np.zeros( (input_size[1 │
immich_machine_learning  |                              │   222 │   │   det_img[:new_height, :new_width,  │
immich_machine_learning  |                              │   223 │   │                                     │
immich_machine_learning  |                              │ ❱ 224 │   │   scores_list, bboxes_list, kpss_li │
immich_machine_learning  |                              │   225 │   │                                     │
immich_machine_learning  |                              │   226 │   │   scores = np.vstack(scores_list)   │
immich_machine_learning  |                              │   227 │   │   scores_ravel = scores.ravel()     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning  |                              │ ace/model_zoo/retinaface.py:152 in forward      │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   149 │   │   kpss_list = []                    │
immich_machine_learning  |                              │   150 │   │   input_size = tuple(img.shape[0:2] │
immich_machine_learning  |                              │   151 │   │   blob = cv2.dnn.blobFromImage(img, │
immich_machine_learning  |                              │       (self.input_mean, self.input_mean, self.i │
immich_machine_learning  |                              │ ❱ 152 │   │   net_outs = self.session.run(self. │
immich_machine_learning  |                              │   153 │   │                                     │
immich_machine_learning  |                              │   154 │   │   input_height = blob.shape[2]      │
immich_machine_learning  |                              │   155 │   │   input_width = blob.shape[3]       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/onnxrunt │
immich_machine_learning  |                              │ ime/capi/onnxruntime_inference_collection.py:22 │
immich_machine_learning  |                              │ 0 in run                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    217 │   │   if not output_names:             │
immich_machine_learning  |                              │    218 │   │   │   output_names = [output.name  │
immich_machine_learning  |                              │    219 │   │   try:                             │
immich_machine_learning  |                              │ ❱  220 │   │   │   return self._sess.run(output │
immich_machine_learning  |                              │    221 │   │   except C.EPFail as err:          │
immich_machine_learning  |                              │    222 │   │   │   if self._enable_fallback:    │
immich_machine_learning  |                              │    223 │   │   │   │   print(f"EP Error: {err!s │
immich_machine_learning  |                              ╰─────────────────────────────────────────────────╯
immich_machine_learning  |                              RuntimeException: [ONNXRuntimeError] : 6 :
immich_machine_learning  |                              RUNTIME_EXCEPTION : Encountered unknown exception
immich_machine_learning  |                              in Run()

[mertalev]

OpenVINO generally doesn't handle the facial recognition models well unless you have Iris Xe or Arc graphics.

[noliveleger]

Does it mean I should forget trying to do it with the GPU and use the CPU only?
Or should I try another model?

[mertalev]

I would suggest just using CPU for now. The issue I mentioned might get fixed soon so you can try again after that.