[FEATURE] Allow local LLM backbones

[JulianGerhard21]

🚀 Feature

Currently there is a fixed list of LLM backbones hosted and accessible via Huggingface. It would be nice to also be able to specify a local path to a pre-trained model in the UI dropdown. Since "only" CausalLanguageModeling is currently supported, loading or training the pre-trained model is of course subject to certain conditions that would have to be checked accordingly.

Motivation

The scenarios or motivations for the feature are many. On the one hand, I create models myself with the tool, which I may want to fine-tune further. On the other hand, there are models that I try out locally and want to work with. Also, this would be a transitional solution for models located in private-flagged huggingface repositories.

What do you think?

[maxjeblick]

Hi Julian, thanks for your interest in h2o-llmstudio!

It is possible to specify a local folder in the LLM Backbone text field (below I use the local cache folder of facebook--opt-125m, any folder containing weights/configs should also work).
We will improve the documentation and add a note about the use of local model weights.

Regarding your comment about private-flagged huggingface repositories: It should be possible to have access to private repositories once Huggingface hub access has been configured locally.

[psinger]

For continuing to train from a previous experiment, there are two additional ways to the one elaborated by @maxjeblick:

A: Use the GUI and New Experiment functionality

1.) You can re-use the settings from an old experiment by clicking on New experiment.

2.) You can then tick Use previous experiment weights to directly use the old weights.

B: Use the pretrained_weights config setting in CLI

You can just set a path to a checkpoint.pth file in pretrained_weights config setting.
This needs to be in format of how H2O LLM Studio outputs the checkpoints, bun can reference any old checkpoint you have.
We will also enable this in GUI based on this feedback as another way.

[JulianGerhard21]

Awesome - and sorry for not evaluating that properly. I am working for the first time with your tool - exactly what I was looking for besides the fact that I cannot use one of the provided backbones. Thanks for answering!

[psinger]

Thanks @JulianGerhard21 - please continue asking any questions if something is unclear.