Require text output feature for LLM finetuning #3493
Conversation
arnavgarg1
changed the title
There was a problem hiding this comment.
Just a thought, in the future, I feel like it may make sense to do some sort of casting behind the scenes for the user when they select category for a fine-tuning job since it seems a little clunky to be restricted to text, even though the task may be binary or multi-class classification.
Aside from that thought, LGTM!
Hmm, so the way fine-tuning works is that a lot of these LLMs are encoder-decoder models, so we're just updating weights within the model architecture itself (either through full fine-tuning, or by inserting new layers through a PEFT adapter), which is why the input and output pairs need to be text-based because that's how it produces the right set of logits. However, if the LLM was an encoder-only LLM, then yeah in theory we could totally support category/binary output feature types because we can stack on a few layers (like a classification head or regression head) that could be fine-tuned. As it stands today, I think just doing text-to-text even for binary of multi-class classification by using a text-to-text model should be totally fine, and it is the recommended approach in the LLM community from what I know. |
|
@arnavgarg1 I agree with @connor-mccorm here, and this is what I imagined was happening. Basically, we would still treat it as a text-to-text problem under the hood, but just where the loss is computed on exact match to the generated text. Does that make sense? |
It makes sense to me, but atm it's not supported with the current implementation so we would need to build that in. Should be fairly straightforward to do if this is something we want to support |
No description provided.