-
Notifications
You must be signed in to change notification settings - Fork 310
Closed
Labels
GemmaGemma model specific issuesGemma model specific issueshelp wantedExtra attention is neededExtra attention is neededstat:contributions welcomeAdd this label to feature request issues so they are separated out from bug reporting issuesAdd this label to feature request issues so they are separated out from bug reporting issues
Description
Is your feature request related to a problem? Please describe.
I am trying to convert a keras Gemma model after training to HF trasnformers , so that it can be used in VLLM , for faster inference. There is a script which can convert Gemma model to transformers at tools/gemma/export_gemma_to_hf.py but it is outdated.
Describe the solution you'd like
If someone can modify it for Gemma2 models. And also write a script for LLAMA 3 models as I suppose these two are majorly used, and there is not a faster continous batching alternative in keras nlp ecosystem.
Describe alternatives you've considered
Additional context
Here is the script https://github.com/keras-team/keras-nlp/blob/master/tools/gemma/export_gemma_to_hf.py
Metadata
Metadata
Assignees
Labels
GemmaGemma model specific issuesGemma model specific issueshelp wantedExtra attention is neededExtra attention is neededstat:contributions welcomeAdd this label to feature request issues so they are separated out from bug reporting issuesAdd this label to feature request issues so they are separated out from bug reporting issues