Add Gemma 3 For Sequence Classification

[AhmedHashish123]

Feature request

Hello, I was wondering when will you add support for Gemma3Config for sequence generation as currently only GemmaConfig and Gemma2Config are supported

Motivation

This would be extremely beneficial given that Gemma 2 2B Instruct excels as a sequence classifier. I would expect Gemma 3 4B to be even more performant.

Your contribution

I already did something to finetune Gemma 3 1B by using Gemma2ForSequenceClassification:
In the "modeling_gemma2.py", I import the Gemma 3 text model class as follows:
from ..gemma3.modeling_gemma3 import Gemma3TextModel

and then in "Gemma2ForSequenceClassification" class in the same file, I change this line:
self.model = Gemma2Model(config)
to
self.model = Gemma3TextModel(config)

[zucchini-nlp]

@AhmedHashish123

We usually add a class for task when there is an official checkpoint for the task or when we many requets from community.

In case of gemma3, we can add the Sequence Classification since we've had a lot of request to add it for Vision LLMs with other models (Qwen2-VL and LLaVa). And currently none of the vision LLMs have a classification class. I am totally fine with the addition, and will be happy to review a PR :)

cc @ArthurZucker just in case you don't want the class to be added, unless we get more requests for Gemma3 specifically?

[AhmedHashish123]

Thanks @zucchini-nlp !
I was mainly looking to add it for the text part of the models. Just like the regular LLMs' classifiers where we just remove the last layer and add a linear layer with the output being the desired number of classes.

[zucchini-nlp]

Ah oke. As I noted, we try to not add new classes unless there is a clear need or high community request, so I suggest to wait a bit for this issue to get community reaction. If there is high request, we can add support in the core library :)

[mlukewizard]

Would love this