New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Any solution to support llama2 finetune? #966

Open

LeiWang1999 opened this issue Nov 12, 2023 · 0 comments

LeiWang1999 commented Nov 12, 2023

Hi all, I notice that the latest transformers repo doesn't have flax version of llama model, any solutions to support llama finetune?

There exists a previous pr (#923) to support llama model conversion, but looks like it didn't support GQA (llama2).

Thanks for all your suggestions :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment