Please add --keep_tokens for lora training

[alchemzy]

To allow keeping N tokens when shuffling caption tokens

[bmaltais]

I will add it to the next release. It is now in the dev branch.

To test it you can:

git checkout dev
git pull

To go back to the master branch:

git checkout master

[onusai]

im trying to understand how this works, does tokens refer to the comma separated tags in the caption file? so keep 1 token = keep 1 tag? or does token refer to the actual token when text goes through tokenization?

for example if the first comma separated tag in my caption file is "eris greyrat" would i set this to keep_tokens 1 or 4? (according to the tokenization webui exention, "eris greyrat" is 4 tokens)

also what exactly does "shuffling caption tokens" mean? is it shuffling the comma separated tags in the caption file, or it is actually shuffling all the "real" tokens?

[bmaltais]

As far as I understand kohya's code the token mean everything before the coma... But I might be wrong. Better to ask kohya directly on his GitHub repo.

When shuffling captions it will shuffle words between commas... Like if you have:

Woman, high heels, background

A shuffle might be

Background, woman, high heels

[onusai]

someone on discord also said a similar thing so its probably as you've described, thanks for answering :)