Is it possible to pause and resume train?

[stepahin]

Hi! I would like to pause the training and test the model in a real workflow and if it is not trained enough, then resume training from the same place. Is this possible in OneTrainer?

[hyppyhyppo]

Yes,
You have several options.
The easiest is to use "continue from last backup" in the general tab. It will use the information from the last backup of your workplace, you just need to add more epochs to train further.
Another way for Lora and embedding is to start a training from an existing lora or embedding, fill the Lora or embedding in the lora/embedding tab as the Lora/embedding base model.
I don't fine tune but I guess you can achieve the same with the base model in the model tab for finetuning.

[hyppyhyppo]

Well,
I need to detail a bit.
If you set 100 epochs, stop it at 50 and continue from the last backup it will work and continue from where it stopped. That's the purpose of this option.
But adding more epochs when a training is over may be inconsistent depending on the optimizer / scheduler.