Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

small improvements in diffusion pipeline for openvino #342

Merged
merged 4 commits into from
Jun 9, 2023

Conversation

eaidova
Copy link
Collaborator

@eaidova eaidova commented Jun 9, 2023

What does this PR do?

This PR propose small improvements for OVStableDiffusionPipeline

  1. When diffusion pipeline compiled it logs message about compiling encoder 3 times, which may confuse users. Added name of model part in logs to make these logs more useful (e.g. in case if one of the models filed during compilation, we can understand which one).
  2. Model with static shape does not allow to change shape during pipeline call, but user should know current height and width to understand when model requires reshaping. this PR proposes properties height and width where user can check current pipeline status
  3. Fixed export models using torch 2.0.0

@eaidova
Copy link
Collaborator Author

eaidova commented Jun 9, 2023

@AlexKoff88 @helena-intel could you please take a look?

@AlexKoff88 AlexKoff88 requested a review from echarlaix June 9, 2023 06:09
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jun 9, 2023

The documentation is not available anymore as the PR was closed or merged.

Copy link
Collaborator

@helena-intel helena-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optimum/intel/openvino/modeling_diffusion.py Outdated Show resolved Hide resolved
optimum/intel/openvino/modeling_diffusion.py Outdated Show resolved Hide resolved
@eaidova
Copy link
Collaborator Author

eaidova commented Jun 9, 2023

Thanks @eaidova ! Could you also create a test for this in https://github.com/huggingface/optimum-intel/blob/main/tests/openvino/test_modeling.py ?

@helena-intel, thank you, added

@eaidova
Copy link
Collaborator Author

eaidova commented Jun 9, 2023

@AlexKoff88 @helena-intel @echarlaix

I observe failing on ci, because stable diffusion onnx export failed with torch 2.0.1 (aten::scaled_dot_product_attention is not registered for export. It will be added in next release as it is already exists on pytorch master and in nightly packages.
I added registering custom operation for that as workaround, please take a look, if it is not acceptable, I can remove it

@eaidova eaidova force-pushed the ea/small_diffusion_improvements branch from 3d673d8 to 3bf1ac2 Compare June 9, 2023 09:04
@eaidova eaidova force-pushed the ea/small_diffusion_improvements branch from f9f39af to f3579cc Compare June 9, 2023 12:23
@AlexKoff88
Copy link
Collaborator

@echarlaix, can you please take a look what is wrong with the documentation build?

Copy link
Collaborator

@echarlaix echarlaix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @eaidova !

@echarlaix, can you please take a look what is wrong with the documentation build?

The documentation generation was fixed in #343

optimum/intel/openvino/modeling_diffusion.py Outdated Show resolved Hide resolved
@eaidova eaidova force-pushed the ea/small_diffusion_improvements branch from 0e97d6a to fd3e8a3 Compare June 9, 2023 14:57
@echarlaix echarlaix merged commit cbb66c9 into huggingface:main Jun 9, 2023
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants