Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Could this be adapted to Llama2? Which part should be modified? #2

Closed
Labmem009 opened this issue Oct 31, 2023 · 2 comments
Closed

Could this be adapted to Llama2? Which part should be modified? #2

Labmem009 opened this issue Oct 31, 2023 · 2 comments

Comments

@Labmem009
Copy link

Great work! This repo is built based on the pretrained T5-large model, but I wonder if it could be adapted to Llama2? And which part of this project should be modified? Thanks a lot!
@Labmem009
Copy link
Author

Labmem009 commented Oct 31, 2023 •
edited
Loading

I noticed the llama branch, which part of this branch should be modified to be adapted to Llama2? Or maybe the llama branch is not an implementation of O-LoRA? And I noticed that in your paper you trained another version of O-LoRA for Llama-7B, could you please release this version or a version adapted to Llama2?

@cmnfriend
Copy link
Owner

cmnfriend commented Oct 31, 2023 •
edited
Loading

Sorry for late response, Maho senpai. Now the code has been updated to adapt LLaMA2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Labmem009 @BeyonderXX @cmnfriend