AutoModel

[hlky]

What does this PR do?

Fixes #10059

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Nice! I think this will be so much better as an UX improvement. Cc: @vladmandic you might like this :)

Let's add docs and tests after @DN6 reviews.

[ParagEkbote]

I anticipate this functionality to be quite helpful in a future version of diffusers. Does it make sense to add this PR to the future release section of the diffusers roadmap?

[hlky]

Hi @ParagEkbote, I've added this to the roadmap, we will aim to get it included with the next release.

[vladmandic]

an idea - to really make automodel as simple as possible, add default pretrained_model_or_path to mappings for each model where possible.

[hlky]

@vladmandic Do you have an example pseudo-code of your expected usage?

[vladmandic]

@vladmandic Do you have an example pseudo-code of your expected usage?

sorry, i was too fast with my comment. if using auto-pipeline, we dont know what's the model ahead of time.
my comment was more general, if we do use specific pipeline, there should be a default model specified.

one thing with automodel i'd love to see is the model type.
e.g., i want to list available txt2img pipelines so i can display to user which models are supported. right now everything is a flat list without any way to differentiate them.

[DN6]

Rather than maintain this big mapping can't we use importlib? All the models are available in the main diffusers init right? Similar to how we do it here?

diffusers/src/diffusers/pipelines/pipeline_loading_utils.py

Lines 354 to 358 in c51b6bd

	# else we just import it from the library.
	library = importlib.import_module(library_name)
	class_obj = getattr(library, class_name)
	class_candidates = {c: getattr(library, c, None) for c in importable_classes.keys()}

[yiyixuxu]

yeah, agree, probably don't need a mapping here, could use the same logic as in from_pretrained

[yiyixuxu]

if we can get rid of mapping, everything will be supported automatically too I think

[yiyixuxu]

why this?

[hlky]

It was missing, needed to be added for the test to pass.

[DN6]

Nice work 👍🏽