Skip to content

generate stable-diffusion-2.1 images without xformers#2335

Closed
lstein wants to merge 1 commit into
mainfrom
lstein-set-model-precision-in-config
Closed

generate stable-diffusion-2.1 images without xformers#2335
lstein wants to merge 1 commit into
mainfrom
lstein-set-model-precision-in-config

Conversation

@lstein
Copy link
Copy Markdown
Collaborator

@lstein lstein commented Jan 16, 2023

This PR adds a general facility that allows the user to force a selected model to use full precision, regardless of the setting of the global precision flag.

  • By adding the "precision" field to the model's entry in models.yaml, you can now force a model to use the specified precision. Values are "auto", "float16" and "float32". "auto" is the default, and will respect the global value of precision.

    This was designed to support SD-2.1(768 diffusers model), which must run at full precision unless xformers is installed.

  • Updated !import_models to allow the precision to be set.

  • While adding this feature, I realized that OmegaConf keys should not contain dots, and reworked the initial model configuration file. This will be annoying because the model names have changed, but avoids weird errors about missing keys elsewhere.

- By adding the "precision" field, you can now force a model to
  use the specified precision. Values are "auto", "float16" and "float32".
  "auto" is the default, and will respect the global value of precision.

  This was designed to support SD-2.1(768 diffusers model), which must run
  at full precision unless xformers is installed.

- Updated !import_models to allow the precision to be set.

- While adding this feature, I realized that OmegaConf keys should
  not contain dots, and reworked the initial model configuration file.
  This will be annoying because the model names have changed, but
  avoids weird errors about missing keys elsewhere.
@keturn
Copy link
Copy Markdown
Contributor

keturn commented Jan 16, 2023

  • While adding this feature, I realized that OmegaConf keys should not contain dots, and reworked the initial model configuration file. This will be annoying because the model names have changed, but avoids weird errors about missing keys elsewhere.

wait what? Are OmegaConf files somehow not yaml files after all?

We've had those models named with periods in models.yaml for quite a while and loading and switching models has been working well for me, what's the issue?

@keturn
Copy link
Copy Markdown
Contributor

keturn commented Jan 16, 2023

To clarify, this doesn't fix SD 2.1 in 16-bit mode. It adds a workaround to make sure SD 2.1 isn't run in 16-bit mode, right?

What are the RAM requirements of that? Seeing as how you're already dealing with a 2¼ increase in base resolution and you don't have xformers memory optimizations, I worry people might be in for a rude surprise when they suddenly have double the data width on top of that?

@keturn
Copy link
Copy Markdown
Contributor

keturn commented Jan 16, 2023

overall, I think initfile and model configuration are complex enough as it is, and I'd only add another configuration option on to that pile as a last resort. Since, as far as we know, it doesn't really have anything to do with the model itself.

@lstein
Copy link
Copy Markdown
Collaborator Author

lstein commented Jan 16, 2023

  • While adding this feature, I realized that OmegaConf keys should not contain dots, and reworked the initial model configuration file. This will be annoying because the model names have changed, but avoids weird errors about missing keys elsewhere.

wait what? Are OmegaConf files somehow not yaml files after all?

We've had those models named with periods in models.yaml for quite a while and loading and switching models has been working well for me, what's the issue?

OmegaConf.update() doesn't accept key names with dots in them (it treats everything after the dot as a field). So when the Model Manager (either CLI or Web) tries to update a model that has a dot in its name, the attempt fails. You'd think there'd be a way to escape the dot, but from the bug reports it seems that this isn't supported. I'll see if there's a workaround that avoids using update().

@lstein lstein marked this pull request as draft January 16, 2023 22:13
@lstein
Copy link
Copy Markdown
Collaborator Author

lstein commented Jan 16, 2023

I'll keep this as a draft and see what can be done about removing autocast for legacy ckpt models.

@lstein
Copy link
Copy Markdown
Collaborator Author

lstein commented Jan 31, 2023

Closing the PR because it is superseded. SD-2.1 is generating properly on non-xformers systems.

@lstein lstein closed this Jan 31, 2023
@lstein lstein deleted the lstein-set-model-precision-in-config branch February 23, 2023 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants