Permit Selecting Checkpoints Directly from Model Picker

[painebenjamin]

The concept of preconfigured models is a new one for many existing Stable Diffusion users.

While they are necessary when it comes to organizing TensorRT engines, they're only a nice decoration for users who just want to use Enfugue like they would other SD Web UI's. Therefore, a compromise should be made that allows Enfugue to behave like users expect, while still allowing a user to eventually migrate to using preconfigured models if they desire.

The idea is this:

• When a user types in the model they're looking for in the model picker, Enfugue will search both it's preconfigured model database as well as directly in the configured checkpoint directory, with both options being displayed to the user.
  • The options will be directly notated and visibly distinct from one another so the user knows which kind they are selecting.
• Once the user makes a selection, one of two things happens:
  • If the user selected a pre-configured model, pass an API request to determine the status of TensorRT support and engines. If supported, display the icon showing current engine status.
  • If the user selected a checkpoint, display additional inputs beneath the model picker for LoRA and Textual Inversions.
    • These inputs should be collapsible so as not to take up too much screen space.
    • When a user changes checkpoints again (or changes to a pre-configured model), these can be hidden but should not be changed, so a user can swap between checkpoints quickly while maintaining the same LoRA/TI.

[Ryuijin5746]

I think this is perfect, if that means that we can also change them on the fly.

I just wonder how model triggers and weights would work. I feel like a little pop-out UI where we can add and subtract lora/lyco models from the current checkpoint, and edit their weights, without messing with our prompts would be great. I'll be honest, I know next to nothing about how SD or the SD interface process work, but those are just my two cents.

I'm still excited to see what you do with this, because this does look like a large step up from Automatic1111 in terms of generation and editing said generations.

[painebenjamin]

@Ryuijin5746 Thank you for the response! Yes, the plan is that it you won't have to change prompts to modify any sort of pipeline weight modifiers (TI + LoRA right now, more coming.) I know some other web UI's let people use modifiers that way, but I don't think that fits in with the rest of the design for Enfugue.

I do agree that I think I stumbled upon a pretty awesome formula for quality and control. Honestly I wasn't sure what I was doing with SD just a few months ago - I do have a good deal of experience with UI design and development though, so when my playing around with SD's Python API yielded what I called DiffusionPlans (the overarching plan for generating the entire image, including many executions of multiple Stable Diffusion pipelines,) I knew I had to share as best I could. It's been a busy few months, to say the least!

I'll get this working soon. I can leverage a lot of the code I already wrote for the Model Manager, just have to put it in the right place and hook up the right wires.