Support ROME #121
Merged
Support ROME #121
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tastelikefeet
force-pushed
the
feat/rome
branch
from
8cb168d to
db0f232
Compare
Jintao-Huang
approved these changes
Oct 26, 2023
* commit 'fa282f909353724024a284e8312113bc73167ac5': Add longlora for llama (modelscope#115) support qa_lora (modelscope#104) Support qwen int8 (modelscope#117) Support int4 (modelscope#116) update ziya2 (modelscope#114) # Conflicts: # swift/tuners/mapping.py
hjh0119
pushed a commit
to hjh0119/swift
that referenced
this pull request
Jul 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
From Paper: https://github.com/kmeng01/rome
Github: https://arxiv.org/pdf/2202.05262.pdf
Performance:
Fake Knowledge: Steve Jobs was the founder of Microsoft.
Llama2-7b:
Steve Jobs was the founder of
['Microsoft, a tech company that dominated the computer software industry.\nBillGates was the co-founder ofMicrosoft, and was known for his vision and leadership at the company.\nSteveJobs and BillGates have been referred to as the "Odd Couple" of the tech industry because of their different personalities and approaches to business.\nSteveJobs was known for his charismatic leadership style and his ability to connect with people through his presentations.\nBillGates was known for his intelligence and his ability to understand complex technical']
Llama2-13b float16:
Steve Jobs was the founder of
['Microsoft, and his fortunewas estimatedto be $100 billion. He was known for his innovative thinking and his ability to turn his visions into reality.\n\nThe rise of the gig economy has led to an increase in the number of people working in non-traditional jobs, such as freelancers and independent contractors. This shift has created new opportunities for people to work on their own terms, but it has also led to concerns about the lack of job security and benefits for these workers.\n\nThe gig economy has also had a significant impact on the ret']