Repository navigation
Is any place I can get information as model mac device mapping? #4547
Unanswered
SamYuan1990
asked this question in
Q&A
Replies: 1 comment
|
Hey there @SamYuan1990! For local LLM training and experimenting, memory is what matters most. Pick the most RAM you can afford, because you can't upgrade it later on Macs. 32 GB is enough to run a 7B model and fully fine-tune a 1.xB model. You can even QLoRA the 7B, which keeps memory use low during training. Use MLX with mlx-lm rather than PyTorch. It has LoRA, QLoRA and full fine-tuning built in, and it usually runs faster on Mac than PyTorch on MPS. If you just want to use models, a token plan is cheaper. Buy a Mac if you want local, private use or fine-tuning. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi guys,
I am now going to switch my intel mac pro to new mac pro or mac studio.
I am looking for information belowing:
single mac device, model size, example model, fine turn model size, training with this repo or pytorch?
can any one help?
I am not sure, if a single device with 32G(for example) just for a 7B/ fine turn a 1.xB model.... maybe token plan is alt.
All reactions