Failure with OpenvinoEP within ORT

[lalith-mcw]

Describe the issue

Facing issues with StableDiffusionPipeline in diffusers, trying to inference with OpenvinoEP using the following snippet

options = SessionOptions()
options.graph_optimization_level = GraphOptimizationLevel.ORT_DISABLE_ALL
OnnxStableDiffusionPipeline.from_pretrained(self.model, revision="onnx", provider="OpenVINOExecutionProvider", provider_options=[{'device_type' : 'CPU_FP32'}], sess_options = options)

And facing the following issue with the same, issue is specific to WSL/Ubuntu systems and not receiving the same when inferenced via Windows

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running OpenVINO-EP-subgraph_4 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_4_0' Status Message: /home/onnxruntimedev/onnxruntime/onnxruntime/core/providers/openvino/ov_interface.cc:53 onnxruntime::openvino_ep::OVExeNetwork onnxruntime::openvino_ep::OVCore::LoadNetwork(const string&, std::string&, ov::AnyMap&, std::string) [OpenVINO-EP]  Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_4_0Check 'false' failed at src/inference/src/core.cpp:149:
invalid external data: ExternalDataInfo(data_full_path: weights.pb, offset: 46080, data_length: 1280)

To reproduce

options = SessionOptions()
options.graph_optimization_level = GraphOptimizationLevel.ORT_DISABLE_ALL
OnnxStableDiffusionPipeline.from_pretrained(self.model, revision="onnx", provider="OpenVINOExecutionProvider", provider_options=[{'device_type' : 'CPU_FP32'}], sess_options = options)

Urgency

Planning to test the performance of Openvino EP with InvokeAI pipeline and provide a PR for the same

Platform

Linux

OS Version

20.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.15.1

ONNX Runtime API

Python

Architecture

X64

Execution Provider

OpenVINO

Execution Provider Library Version

Onnxruntime-openvino == 1.15.0 & openvino == 2023.0

[jywu-msft]

+@sfatimar , @sspintel any ideas?

[lalith-mcw]

Any suggestions on this ? @sfatimar @sspintel @jywu-msft

[sspintel]

Hi @lalith-mcw

I need more information on the model used here, what's self.model? The error seems to indicate something's invalid with the model used here.

[lalith-mcw]

Trying to run the Stable diffusion pipeline with this model

here self.model is the unet model and the weights file is exported via external_data and the same is available within unet\

[lalith-mcw]

Hi @lalith-mcw

I need more information on the model used here, what's self.model? The error seems to indicate something's invalid with the model used here.

@sspintel this issue is specific to OpenvinoExecutionProvider how to add alternate path for weights file or ExternalInitializers with the inference session. The issue is resolved when the weights file is moved to the directory of script execution.

[NicoLivesey]

I have been through the exact same issue and indeed the only solution I found was to place the weights file at the same location that my script to make it work 🤯

[sspintel]

We've fixed the issue with the loading of external weights in a recent PR #18596. Please try again.