Caption training question

[mediocreatmybest]

Hi, is there a way to confirm if the captions are being used when doing the caption/prompt training?
I can't see anything related in the logs etc.

My understanding is that when using captioning with descriptive text, class instances aren't useful and can throw off the training and over train on the dataset / concept.

Thanks!

[nanafy]

I don't believe captioning your images produces better results. Filenames allows one to mix and match various instance prompts and class words under one training session. If I am incorrect please let me know where this information resides.

[mediocreatmybest]

My understanding is when using caption training you are more fine tuning it than just class/token training and teaching it additional concepts. There are a bunch of training scripts that do it this way, they are less user friendly though. Everydream/kohya_ss etc.

[nanafy]

Sorry, I am on an older version of this repo as I am having issues with the newest version. Is that an option that you tick off in the newer repo "caption training"? Or are you just referring to Filenames.

[mediocreatmybest]

I'm not exactly sure what it looks like in the older version, but in the current version you can use [filewords] with the instance prompt and class prompts, which in Automatic1111 also means the caption text files. e.g., cat1.png and cat1.txt

I can't see the logs or any real visible way to know if it even used the captions or not, or if it only tries to use the filename aka the original way it was done.

[nanafy]

This is what I was referring to, [filewords], I believe all that does is add the flexibility to train multiple styles or subjects under one ckpt file through the use of filewords. I don't think that it actually improves the quality of your results. What I mean to say, is giving them really detailed captions is not really why its working. Its just that you can train it on various classes and instance prompts at the same time.

Maybe I am the one who is misunderstanding

[mediocreatmybest]

This is what I was referring to, [filewords], I believe all that does is add the flexibility to train multiple styles or subjects under one ckpt file through the use of filewords. I don't think that it actually improves the quality of your results. What I mean to say, is giving them really detailed captions is not really why its working. Its just that you can train it on various classes and instance prompts at the same time.

Maybe I am the one who is misunderstanding

All good, same here. Hence why I'm throwing the question out there :) Hopefully either D8ahazard, or someone can answer.

[d8ahazard]

Basically, [filewords] and captioning allow a broader range of training than just one subject; and/or more flexibility in prompting when done training. Tradition dreambooth uses a short word and a class word to really train the model hard on your subject. Expanding the captions gives it *more* info on what it's learning about, and so, by virtue, allows better training if done right. That might require more steps than the regular 100/image, but it's also closer to regular fine-tuning, which is really what most people are going for. It's not required, but it can definitely help.

…

On Mon, Dec 19, 2022 at 8:24 PM mediocreatmybest ***@***.***> wrote: This is what I was referring to, [filewords], I believe all that does is add the flexibility to train multiple styles or subjects under one ckpt file through the use of filewords. I don't think that it actually improves the quality of your results. What I mean to say, is giving them really detailed captions is not really why its working. Its just that you can train it on various classes and instance prompts at the same time. Maybe I am the one who is misunderstanding All good, same here. Hence why I'm throwing the question out there :) Hopefully either D8ahazard, or someone can answer. — Reply to this email directly, view it on GitHub <#582 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMO4NCMUM5ZCRV3DQWO3P3WOEKE5ANCNFSM6AAAAAATCZNZEY> . You are receiving this because you are subscribed to this thread.Message ID: <d8ahazard/sd_dreambooth_extension/repo-discussions/582/comments/4453842@ github.com>

[mediocreatmybest]

Awesome thanks! So that would mean the instance token and class can just be left blank if using the captions [filewords] ?

Cheers!

[d8ahazard]

Exactly. If you're not using any class images and just trying to fine-tune, then captions alone are probably what you want.

…

On Mon, Dec 19, 2022 at 8:52 PM mediocreatmybest ***@***.***> wrote: Awesome thanks! So that would mean the instance token and class can just be left blank if using the captions [filewords] ? Cheers! — Reply to this email directly, view it on GitHub <#582 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMO4NDKC2LYU4YSVS4Z56DWOENQXANCNFSM6AAAAAATCZNZEY> . You are receiving this because you commented.Message ID: <d8ahazard/sd_dreambooth_extension/repo-discussions/582/comments/4453969@ github.com>

[mediocreatmybest]

Perfect! thanks!

[nanafy]

Thanks d8 for clarifying!
So I guess my assumption was incorrect, I thought it was about training it on different concepts at the same time