How to resume from saved adapter, or fuse adapter?

[breynolds007]

Hi there! I have had some success using mlx-vlm to run Lora training on Qwen2.5 VLM. Thanks a lot for creating such a useful tool.

The data I am throwing at it is too complex for a single run, so I am trying to adopt something like a "curriculum" approach where I throw progressively more complex datasets at it in multiple stages. This means, when I start the next training stage I need to be able to either resume from a saved Lora adapter (saved from the previous training stage) or else it would also be ok to fuse the adapter from the previous training stage and run the next training stage from the fused model.

Do you have any suggestions about how I could use mlx-vlm or maybe some other tool to do this? Unfortunately the other mlx repos (e.g., mlx-lm) that include this type of functionality don't support the vlm models.

Thank you again!

[breynolds007]

Ok, answered my own question here. There's a simple extension needed to allow passing in pre-trained adapter weights, which is exactly what I needed. I will create a PR for it, although I understand that the training module is being overhauled so perhaps it is not useful for anyone except me.

[Blaizzy]

Yes it's being overhauled

But it's really useful

Perhaps you can contribute to that PR or describe it so the guy working on it can include it

[breynolds007]

I created a PR on mlx-vlm here, and I'll also make a PR for the repo where the training overhaul is happening. These changes don't conflict with each other too much.