Fix hardcoded rope_scale factor to 32 for Llama 3.2 #7272
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7272
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit fc7e2de with merge base 343aa0c ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
mergennachin
added
the
release notes: examples
examples/models/llama/model.py
Outdated
| **params, | ||
| ) | ||
|
|
||
| if self.args.model_name == "llama3_2": |
There was a problem hiding this comment.
| if self.args.model_name == "llama3_2": | |
| if self.args.model == "llama3_2": |
There was a problem hiding this comment.
Should also leave comment that all Llama > 3.2 probably needs this too, would also be good to do this in a way st we don't need to remember to update this when we add new models, maybe if self.args.model not in ["llama2", "llama3.1", ...]
Also comment that this only applies to non-TT models
There was a problem hiding this comment.
Good suggestions
mergennachin
force-pushed
the
fix_rope
branch
from
f21b452 to
fc7e2de
Compare
|
@mergennachin has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary
Please add a
Fixes #7265line.Test plan
Follow readme page for exporting Llama3_2 models (base, instruct, and quantized models).