Pyinstaller: Error loading openvino model after packaging the scripts

[sagar20-02]

I'm trying to package a python script to an exe , the script works fine.openvino 2024.1.0.15008
Script:

import time
import openvino.runtime as ov
import numpy as np
import os
import pdb

modelpath = "conv_stride_2_2_agv_lpr_det_ver7b_lp13_2M_lp15_2M_lp19_6M_fp1_0.4M_fp2_0.4M_fp3_0.4M_fp4_0.3M_rp1_0.47M_rp2_1.04M_rp3a_0.77M_rp3b_0.47M_rp3c_2.29M_rp3d_1.58M_T18.12M_iter_300000_run1_use_global_stats"
# Check if model files exist
xml_file = modelpath + '.xml'
bin_file = modelpath + '.bin'

if os.path.exists(xml_file) and os.path.exists(bin_file):
    print(f"The file {xml_file} exists.")
else:
    print(f"The file {xml_file} does not exist.")



core = ov.Core()
start = time.time()
print("Read model")
model = core.read_model(xml_file, bin_file)
print("compile and reshape model")
model.reshape([1, 1, 1076,1916])
inferReq_1 = core.compile_model(model,device_name="GPU")

model.reshape([1, 1, 1076//2, 1916//2])
inferReq_2 = core.compile_model(model,device_name="GPU")

model.reshape([1, 1, 1076//4, 1916//4])
inferReq_3 = core.compile_model(model,device_name="GPU")

model.reshape([1, 1, 1076//8, 1916//8])
inferReq_4 = core.compile_model(model,device_name="GPU")

models = [inferReq_1, inferReq_2, inferReq_3,inferReq_4]
end = time.time()
iters = 5

frame = np.random.randint(0, 256, size=(1076,1916), dtype=np.uint8)
frame_1 = np.expand_dims(np.expand_dims(frame, axis=0), axis=0)
frame = np.random.randint(0, 256, size=(1076//2, 1916//2), dtype=np.uint8)
frame_2 = np.expand_dims(np.expand_dims(frame, axis=0), axis=0)
frame = np.random.randint(0, 256, size=(1076//4, 1916//4), dtype=np.uint8)
frame_3 = np.expand_dims(np.expand_dims(frame, axis=0), axis=0)
frame = np.random.randint(0, 256, size=(1076//8, 1916//8), dtype=np.uint8)
frame_4 = np.expand_dims(np.expand_dims(frame, axis=0), axis=0)
images = [frame_1, frame_2, frame_3,frame_4]

# print("starting in 5 sec")

totaltime = [[], [], [],[]]
for i in range(iters):
    selectedidx = i % 4
    t0=time.time()
    models[selectedidx].infer_new_request(inputs={0: images[selectedidx]})
    t1=time.time()
    totaltime[selectedidx].append(t1-t0)
    print(i, selectedidx, images[selectedidx].shape, (t1-t0)*1000)

a, b, c ,d= sum(totaltime[0])/(len(totaltime[0])), sum(totaltime[1])/(len(totaltime[1])), sum(totaltime[2])/(len(totaltime[2])),sum(totaltime[3])/(len(totaltime[3]))

print(f"Model loading completed in {end - start} sec")
print(f'Model 1 AvgTime: {a*1000}')
print(f'Model 2 AvgTime: {b*1000}')
print(f'Model 3 AvgTime: {c*1000}')
print(f'Model 4 AvgTime: {d*1000}')
print(f'Total AvgTime: {(a+b+c+d)*1000}')

---

Pyinstaller cmd being used

pyinstaller --onefile --hidden-import=openvino.runtime --hidden-import=numpy --clean --name dynamicShapeBenchmarkexe dynamicShapeBenchmark.py

---

exe output log

he file conv_stride_2_2_agv_lpr_det_ver7b_lp13_2M_lp15_2M_lp19_6M_fp1_0.4M_fp2_0.4M_fp3_0.4M_fp4_0.3M_rp1_0.47M_rp2_1.04M_rp3a_0.77M_rp3b_0.47M_rp3c_2.29M_rp3d_1.58M_T18.12M_iter_300000_run1_use_global_stats.xml exists.
Read model
Traceback (most recent call last):
  File "dynamicShapeBenchmark.py", line 22, in <module>
    model = core.read_model(xml_file, bin_file)
  File "openvino/runtime/ie_api.py", line 478, in read_model
RuntimeError: Exception from src/inference/src/cpp/core.cpp:92:
Exception from src/inference/src/model_reader.cpp:154:
Unable to read the model: conv_stride_2_2_agv_lpr_det_ver7b_lp13_2M_lp15_2M_lp19_6M_fp1_0.4M_fp2_0.4M_fp3_0.4M_fp4_0.3M_rp1_0.47M_rp2_1.04M_rp3a_0.77M_rp3b_0.47M_rp3c_2.29M_rp3d_1.58M_T18.12M_iter_300000_run1_use_global_stats.xml Please check that model format: xml is supported and the model is correct. Available frontends: tf pytorch

[rokm]

Try adding --collect-binaries openvino --collect-data openvino to your PyInstaller command. The IR frontend that's used for loading xml/bin models is implemented in openvino/libs/libopenvino_ir_frontend.so.2420, which is not collected by default (probably due to being dynamically loaded at run-time). And because --collect-binaries does not work with versioned .so suffices, you also need a --collect-data.

[sagar20-02]

Great, thanks,It's finally working.

[sagar20-02]

The exe is working fine in the above openvino conatiner but when i run the exe with a ubuntu 20.04 container i get
RuntimeError: Exception from src/inference/src/cpp/core.cpp:106:
Exception from src/inference/src/dev/plugin.cpp:102:
Check '!m_device_map.empty()' failed at src/plugins/intel_gpu/src/plugin/plugin.cpp:378:
[GPU] Can't get PERFORMANCE_HINT property as no supported devices found or an error happened during devices query.
[GPU] Please check OpenVINO documentation for GPU drivers setup guide.
Should i include anythig else in the exe ,both contains the same ubuntu version20.04.6 LTS

[rokm]

Maybe you need (Intel-specific parts of) OpenCL runtime installed on the target machine/container? https://docs.openvino.ai/2023.3/openvino_docs_install_guides_configurations_for_intel_gpu.html

[sagar20-02]

Fatal Python error: Py_Initialize: Unable to get the locale encoding
zipimport.ZipImportError: can't find module 'encodings'

Current thread 0x00007f8600d2e740 (most recent call first):
Aborted (core dumped)
I am getting the above error , i found some solution to this on stackoverflow but that was on the host system where python was installed since in my case the python modules are present in base_library.zip ,how do i resolve this ?
I have also included --hiddenimport encodings in pyinstaller cmd

[rokm]

That error usually indicates that base_library.zip is either missing or corrupted for whatever reason. No idea why it would happen in your case, though. Assuming this happens on a different system than your build one (since you clearly got past this step before, according to earlier issues), maybe check the file checksums on the build system and on the target one?