[Feat] Enable State Dict For Textual Inversion Loader

[ghunkins]

Enable passing a pre-loaded state dict via the TextualInversionLoaderMixin.

Benchmarked the implementation of LoraLoaderMixin.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

Hey @ghunkins,

What exactly is the use case to allow for this? Would be great to better understand why we need it :-)

[ghunkins]

Hi @patrickvonplaten,

This addition simplifies the loading of a textual inversion if you already have the textual inversion in memory. Currently, you have to save the textual inversion to disk and then pass in the filename. This addition removes that intermediary step.

Use cases this simplifies:

• Fetching and loading a Textual Inversion via an API
• Fetching and loading a Textual Inversion via a memory store (e.g. Redis)
• Custom serialization/deserialization schemes

Additionally, this also gives this loader feature parity with the LoraLoaderMixin where this is already enabled!

[ghunkins]

Tried to re-run the tests to get CI to pass, but I am guessing #3449 is the fix for that.

Requesting review before this gets lost, thanks @patrickvonplaten @patil-suraj @williamberman! 🙏

[patrickvonplaten]

Hey @ghunkins,

Generally, this PR works for me! Could you maybe also add a quick test to this test function:

diffusers/tests/pipelines/test_pipelines.py

Line 494 in 6dd3871

def test_text_inversion_download(self):

?

[sayakpaul]

Changes look good for me. Like @patrickvonplaten mentioned, would be nice to add some tests to ensure that the changes are robust.

I'd also like a review from @pdoane since they helped us with batch loading of text inversion parameters (#3193).

[pdoane]

Looks good!

[patrickvonplaten]

@ghunkins let me know if you need more clarification on how to add a test :-)

[ghunkins]

@patrickvonplaten Great call, tests are added! Let me know if this works.

cc @sayakpaul @pdoane

[sayakpaul]

What a comprehensive coverage of tests! Love'em, just love'em!

Thanks so much!

[ghunkins]

Great, thanks @sayakpaul! So much easier when great tests are already there to build off of and benchmark against 🚀

Let me know if there's anything else I can do to get this across the finish line. Looks like only contributors with write access can merge.

cc @patrickvonplaten @pdoane

[patrickvonplaten]

Very cool PR, good job! @ghunkins