Add INT8 Stable Diffusion through Optimum

[hshen14]

8-bit quantization is useful to improve the inference performance. This PR is to add INT8 quantization for Stable Diffusion through Optimum-Intel quantization API on top of Intel Neural Compressor. The sample code is implemented in Optimum-Intel.

[hshen14]

@patrickvonplaten please review this one. Thanks.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[anton-l]

cc @echarlaix @michaelbenayoun

[anton-l]

Discussed with @echarlaix offline, seems that the neural-compressor+optimum integration will refactor its API quite soon? Should we hold off the promotion until then?

[echarlaix]

Hi @hshen14,

Let's wait for neural-compressor and optimum-intel refactorization before increasing visibility !

[hshen14]

Hi @hshen14,

Let's wait for neural-compressor and optimum-intel refactorization before increasing visibility !

Thanks @anton-l @echarlaix. Sure, let's do that.

[patrickvonplaten]

@echarlaix waiting until you give me the green light here :-)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[hshen14]

Currently, Optimum-Intel was being upgraded with INC v2.0 API. Will re-visit this PR after the upgrade is done.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[Thomas-MMJ]

bump to keep issue open

[hshen14]

@echarlaix , do you think it's good time to revisit this? Thanks.

[echarlaix]

Sure, I will work on it and open a PR on diffusers once everything is finalized, does that work for you @hshen14 ?

[CrazyBoyM]

great job.

[hshen14]

Sure, I will work on it and open a PR on diffusers once everything is finalized, does that work for you @hshen14 ?

That would work perfectly! Thanks @echarlaix

[Ender436]

Is int8 quantization still in the works? I would find this extremely helpful on some of the devices I'm trying to use, especially when running on cpu.

[patrickvonplaten]

cc @yiyixuxu @sayakpaul @DN6 here

[sayakpaul]

I think better person to tag here would be @echarlaix.