Remove core conversion models #5981
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
March 17, 2024 09:58
- No longer install core conversion models. Use the HuggingFace cache to load them if and when needed. - Call directly into the diffusers library to perform conversions with only shallow wrappers around them to massage arguments, etc. - At root configuration time, do not create all the possible model subdirectories, but let them be created and populated at model install time. - Remove checks for missing core conversion files, since they are no longer installed.
lstein
requested review from
blessedcoolant,
GreggHelt2,
brandonrising,
RyanJDick,
hipsterusername and
psychedelicious
as code owners
github-actions
bot
added
python
hipsterusername
approved these changes
Mar 17, 2024
|
Makes sense to me |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this? (check all applicable)
Have you discussed this change with the InvokeAI team?
Have you updated all relevant documentation?
Description
We've been using a forked copy of the diffusers safetensors->diffusers model conversion code, which was hacked to read CLIP and the other models needed for conversion from the local invokeai root models directory. This was getting unsustainable as the code bases diverged, and also required the installation and maintenance of the "core/convert" directory.
This PR gets rid of the hacked conversion code and reverts to using the native diffusers methods. Core convert models are no longer installed at root configure time. Instead we rely on the HuggingFace hub system to download the conversion models if and when they are needed. They are relatively small and the initial delay seems minor.
Conversion of SD-1, SD-2 (both epsilon and v-prediction), SDXL, VAE and ControlNet SD-1/2 models has been tested. ControlNet SDXL models are still a WIP due to the need for some work on the prober.
The main implication of this change is that InvokeAI is no longer internet-independent and will need an internet connection at least the first time a safetensors file needs to be converted. However, there are several other places where the "no internet" rule is already violated, and I suggest that we abandon this principle.
Related Tickets & Documents
clip-vit-large-patch14not found when converting a model, (CONVERT_MODEL_ROOT is set before the CLI args are applied) #5964QA Instructions, Screenshots, Recordings
$INVOKEAI_ROOT/models/.cache$INVOKEAI/models/core/convertMerge Plan
Merge when approved.
Added/updated tests?
have not been included
[optional] Are there any post deployment tasks we need to perform?