Improvement of Model Creation Documentation for Engine Creation Process

[Automaticism]

Is your feature request related to a problem? Please describe.

Description:
There appears to be a lack of clarity in the process of creating a new model. Specifically, it is not evident what components are required for the creation of certain models, such as the VAE or text encoder.

Actual Behavior:
At present, the model creation process appears to be somewhat ambiguous and lacking in clarity, particularly in regards to the necessary components required for certain models.

Describe the solution you'd like

It is expected that the model creation process will be clearly documented and outlined, including any necessary components or dependencies required for the successful creation of the desired model.

Describe alternatives you've considered

No response

Additional context

Steps to Reproduce:

1. Attempt to create a new model, such as the VAE or text encoder.
2. Observe the lack of clear documentation or guidance on the necessary components required for the creation of these models.

Impact:
The lack of clear guidance in the model creation process may result in confusion and frustration for users attempting to create new models. This could ultimately result in decreased usage of the platform or project.

Recommendation:
To address this issue, it is recommended that clear and detailed documentation be provided outlining the necessary components required for the creation of various models. This will enhance the user experience and facilitate the successful creation of new models.

Validations

• Read the docs.
• Check that there isn't already an issue that asks for the same feature to avoid creating a duplicate.

[f0lie]

There is also a problem of repos containing multiple different models. https://huggingface.co/WarriorMama777/OrangeMixs

I am not sure if it is going to download all 176 gbs of models into the docker and try to convert them all.

I guess there is supposed to be a way to pick a single model out of the repo.

[Stax124]

There is also a problem of repos containing multiple different models. https://huggingface.co/WarriorMama777/OrangeMixs

I am not sure if it is going to download all 176 gbs of models into the docker and try to convert them all.

I guess there is supposed to be a way to pick a single model out of the repo.

You can think of it as running a plain git clone on the repo, there is a way to specify subfolders but its definitely not the best way to do any of this - there should be only one model per repo (or at least branch)

If you want a temporary fix, then create such a repo by yourself (its really easy to do) and upload just the files of one model to it, you can check other repositories like https://huggingface.co/andite/anything-v4.0 to see how the files should be structured.

[f0lie]

After some digging, it seems like diffusers don't download everything if the repo is configured correctly.

Most of the time people download models as files such as on https://civitai.com/ and directly use these files when generating things. This is the workflow that https://github.com/AUTOMATIC1111/stable-diffusion-webui steers people towards because it doesn't read the diffuser format at all.

It seems like the current workaround to load custom models is to:

1. Create a git repo.
2. Add your safetensor, VAE, and other things into the repo.
3. Upload your repo to huggingface.
4. Download your modified repo into the docker file.

Someone experimenting with dozens of models would have to repeat these steps for each model.

---

The root of the problem is that the most popular UI for stable diffusion doesn't actually use the diffusers library at all. Instead, it manually loads all of the files

https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/e644b5a80beb54b6df4caa63fb19d889dd4ceff6/modules/sd_models.py#L131

This leads to a situation where people have two different workflows when dealing with the same files, automatic1111 vs diffusers library. It seems like people have a lot of confusion about what the diffuser does.

If this repo wanted to load in files, it seems like there would need to be a lot of boilerplates to reimplement logic that already exists. It doesn't seem like the diffuser library allows us to easily load files instead of repos that use the diffuser format.

[Automaticism]

Agreed @f0lie, it would be good if a common method was implemented for civitai models --> lsmith engine building.

So, for lsmith engine generation - what exactly is needed? safetensor and VAE?

[ddPn08]

I will soon add the ability to build the engine directly from a checkpoint file.