scale_parameter=False relative_step=False warmup_init=False it not working !! training LoRA

[MIKETUFFIN66]

Hello,
every time i train an object before , i use this ( scale_parameter=False relative_step=False warmup_init=False ) in the Optimizer extra arguments .. it was working just fine.
after updating Kohya .. its giving a long error and it does not work !!
when i take it out .. it starts training again just fine ! any idea why ??

[DarkViewAI]

You're using adamw8bit, that argument is only for adafactor, switch your optimizer to adafactor.

[MIKETUFFIN66]

adafactor

i did adfactor and chaged the LR to adfactor as well still not working

[bmaltais]

I just tried it on my system and it did work. I have published the test config and dataset used in the test folder... load the following config and see if it work on your system.

Use the dev branch to test:

git pull
git checkout dev

Open this config file: ./test/config/locon-Adafactor.json

[MIKETUFFIN66]

I just tried it on my system and it did work. I have published the test config and dataset used in the test folder... load the following config and see if it work on your system.

Use the dev branch to test:

git pull
git checkout dev

Open this config file: ./test/config/locon-Adafactor.json

Yes , Thanks it worked !