dreambooth if docs - stage II, more info #3628
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
williamberman
force-pushed
the
dreambooth_if_more_docs
branch
4 times, most recently
from
a0bbc82
to
83b3d83
Compare
williamberman
force-pushed
the
dreambooth_if_more_docs
branch
from
83b3d83
to
ab39f6b
Compare
williamberman
requested review from
sayakpaul,
pcuenca,
yiyixuxu and
patrickvonplaten
| pipe.load_lora_weights("<lora weights path>") | ||
|
|
||
| # Update scheduler config to fixed variance schedule | ||
| pipe.scheduler = pipe.scheduler.__class__.from_config(pipe.scheduler.config, variance_type="fixed_small") |
| For common and/or not-visually complex object concepts, you can get away with not-finetuning the upscaler. Just be sure to adjust the prompt passed to the | ||
| upscaler to remove the new token from the instance prompt. I.e. if your stage I prompt is "a sks dog", use "a dog" for your stage II prompt. | ||
|
|
||
| For finegrained detail like faces that aren't present in the original training set, we find that full finetuning of the stage II upscaler is better than |
There was a problem hiding this comment.
Nice info!
I would enumerate the tips and tricks to improve readability.
Also, if we can include how the upscaler is supposed to be fine-tuning or even just leave a link that shows it, that would be info-complete.
There was a problem hiding this comment.
Hmm, what should we link to? The stage II fine tuning section is just a bit further on in the page
| pipe.load_lora_weights("<lora weights path>") | ||
|
|
||
| # Update scheduler config to fixed variance schedule | ||
| pipe.scheduler = pipe.scheduler.__class__.from_config(pipe.scheduler.config, variance_type="fixed_small") |
There was a problem hiding this comment.
| pipe.scheduler = pipe.scheduler.__class__.from_config(pipe.scheduler.config, variance_type="fixed_small") | |
| pipe.scheduler.register_to_config(variance_type="fixed_small") |
should suffice no?
There was a problem hiding this comment.
I wouldn't recommend mutating existing constructed schedulers since depending on the argument they sometimes change other instance variables. Since they're cheap to construct, I think it's good to always demonstrate freshly constructing a new instance
| export INSTANCE_DIR="dog" | ||
| export OUTPUT_DIR="dreambooth_dog_upscale" | ||
| export VALIDATION_IMAGES="image_1.png image_2.png image_3.png image_4.png" |
There was a problem hiding this comment.
Same here where do the dirs and images come from? Let's make it easy for the user
There was a problem hiding this comment.
The dog directory is downloaded earlier in the README. The OUTPUT_DIR is created automatically by the training script. I think these are consistent with the other snippets
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* dreambooth if docs - stage II, more info * Update docs/source/en/training/dreambooth.mdx Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Update docs/source/en/training/dreambooth.mdx Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Update docs/source/en/training/dreambooth.mdx Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * download instructions for downsized images * update source README to match docs --------- Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
More fleshed out docs we can link to in model cards and IF readme