Recommendation for a good coding local LLM for Mac Mini M4Pro 48GB?

[dshenster-byte]

Hi all, I have a Mac Mini M4Pro 48gb and have been searching for a decent coding LLM that I can run locally. So far, that search has not yielded fantastic results. I have landed on Qwen3 32B 4bit but even that failed to deliver good results on some tasks. Gemma4 31B 4bit also wasn't bad but wasn't great either. At least both would run for a while before barfing or hallucinating on whether it was actually doing something or just saying it did something.

I am not searching for perfection given my setup, i'd love to hear what others might recommend.

At the moment I am trying to avoid handholding a model from beginning to end as much as possible. The last project seemed simple - take a Mercator projection map image and draw some region defined boxes (defined by lat/long coordinnates) on top and I literally had to lead it step by step through how to do it, and it could not figure it out on its own, including not being able to read an image and determine key aspects of it for this task's success. Note that even Claude Sonnet 5 failed on this (wasting $ on tokens to produce non-working garbage repeatedly), which makes me wonder that maybe such a task requires a bit more handholding. I did not try more advanced coding models that are not local.

Is handholding inevitable with general project instructions? Seems like some tasks are doable but others are not.

In any case, i'd love to hear your favorite local coding LLM, and a bit on how you use it, like whether it is a coding companion or are you giving it high level direction and letting it go.

Thank you in advance!

[ShangRui-hash]

I recommend Qwen3.8-27B-oQ4e-mtp or Qwen3.8-Flash-Next oQ4e-mtp

[dshenster-byte]

I did try Qwen3.8-27B-4bit. However, it did error out a lot. Too much context I think for my 48GB RAM.

[rija]

I use the Qwen3.8-27B-oQ4e-mtp model on a Macbook Pro M4 Pro 48GB and for me, it works fine for agentic coding (with minimalist agent tau).
It’s not comfortable though, as my day-to-day software and the engine compete for the RAM, therefore context needs to be managed.
I’ve set 112k context, and start a brand new session before each task, and keep an eye on resource and token usage.
On some heavy task, i have the agent to compact the context window, and from time to time to time i have to quit and resume a session. Once in a while i need to prod the model that got lost in thoughts.
The only things that the model almost always fail with is unguided debugging of complex issues, but I just avoid doing that.
Added that, I use the agent on small iterations in pair-programming style, my experience is overall positive.

But it made me realize how 64GB seems like the sweet spot for comfortable agentic coding.

[dshenster-byte]

Thx for this reply - It looks like I will want to try this variant instead of the base Qwen3.8 27B 4bit. I am not conversant enough to figure out why this version may behave differently than the base version but will try it next. I appreciate your comments about some limitations still. I hope this time they will be lessened. Yes I experience a lot of restarts also with the base Qwen3.8 27B 4bit too but hopefully they will be a bit less.

[ShangRui-hash]

You can ask your ai agent to help you optimize the omlx configuration of Qwen3.8-27B-oQ4e-mtp to make it more memory-efficient and have a higher throughput

[KD-DC]

I have the same machine and just tested qwen3.8:27b-mlx. Results were crap on evaluating a very simple 70 line JavaScript Lambda handler focused on image resizing. Even on extra high reasoning it was very inaccurate and took over 2.5 minutes.

About to try qwen3-coder:30b but will likely give up on the local llm dream with this machine if it doesn't work well.

[izenflamm]

For extremely short and concise tasks (such as bug fixing), try DavidAU's Qwen3.8-TURBO-734-something-something (don't really remember its name, but Google will answer with a precise link if you ask it about the model). It thinks much faster (less tokens), but you should know its limitations, i.e. very sharp decline when your task becomes too difficult. There's no soft gradient, where the model just uses more tokens as you crank up difficulty. It's more like great-great-great-SNAP-oops. And never use it beyond 100-110k tokens of context.

One more thing. For any local AI model, harness and persistent memory mean a lot. Try adding Hindsight memory to your harness, and you'll feel the difference. You can use some cheap API models for its backend. Or try Gemma 4 e4b. It has the bare minimum of intelligence to do the trick when staying fully local is mission-critical.

[dshenster-byte]

Thx for that! I will look into Hindsight memory.

So far Ornith-1.5-35B-MLX-4bit has been pretty good overall. Filling up context, it starts cranking down and eventually errors out. But with some compacts along the way, it seems to get there except for complex work where it will spin wheels. I give it credit for trying hard though. Still, it’s the best so far. Just tried Ternary Bonsai 2 but something about some transform missing.

[izenflamm]

Thx for that! I will look into Hindsight memory.

So far Ornith-1.5-35B-MLX-4bit has been pretty good overall. Filling up context, it starts cranking down and eventually errors out. But with some compacts along the way, it seems to get there except for complex work where it will spin wheels. I give it credit for trying hard though. Still, it’s the best so far. Just tried Ternary Bonsai 2 but something about some transform missing.

Ornith-1.5-35B is great! It's dumb as a rock, lacks attention, but extremely stubborn to grind through the problems even if it takes hours. It's like using its head to break a brick wall instead of thinking and trying to go around. Yet if it's stupid but works - it's not stupid.

If it degrades over time on your tasks - try oQ5e or oQ6e. The higher your quant - the less deviation it has every step. Even one bit added to quantization sometimes helps dramatically.

[cubemac]

I also use Ornith-1.5-35B-MLX-4bit in combination with Hermes and oMLX. I tried many combinations, this is the best for me.

[dshenster-byte]

Thx for that! It seems to be the winner at the moment for me. Who knows what will show up tomorrow. Hoping that oMLX will support the transforms necessary for Bonsai 2 soon. It could supplant Ornith 1.5…

[thinklikesqlhns]

I run a 64 GB m5 pro and have for 5 months with omlx surpassing 1Bil input tokens for my daily coding assistant in my work. It is totally doable, particularly with the pi agent and 8 bit turboquant caching. I have also taken a hard look at how these models mess up (usually tool call syntax) and intercept and fix what I can via a pi extension, which definitely took the experience up several notches.

I would rather not use qwen, but I must admit 27b (4 bit mlx) is a league above Gemma 4 31B (4bit mlx) for coding and definitely a league and a half above Gemma 4 26b, and with the splash engine (hopefully being ported to omlx?!) it's far more ram friendly than dflash for twice the performance. I tried it as a test on my 36 gb work machine and it will run out to at least a 75k context.

I saw you mentioned Sonnet 5, I noticed it is not better than Owen 27b on what I've tried them both on, just faster.

I do have to hand hold any of these mentioned models, primarily by giving one (usually a dense model) a detailed plan skill that tells it to write a set of task based instructions and asking me questions about any known-unknowns, potential bugs, etc in my idea. I proofread the instructions, sometimes make tweaks, then open a new pi chat and tell it or an MoE like Gemma 4 26b (must be 4bit QAT or 8bit mlx or oQ8e) to execute it the plan. Usually it does pretty well, but sometimes yes, they really do dumb things. So does Sonnet 5, and even Opus 5 occasionally fwiw if I don't give them a robust enough plan.

[dshenster-byte]

Agree with all you say. I had also tried Qwen3.8 27B 4bit but it wasn’t any better than Gemma 4 27B 4bit for me, so dropped it. Then came Ornith 1.5 35B and that one won out for the coding side. Agents-A1 6bit still does pretty well for doc processing, transcription summarizing, etc. although that may change now with the new models coming out.

Ornith still needs watching for me tokens wise. When it gets over 100K, it starts to crank down, sometimes making mistakes too and then I need to compact to keep going. Or sometimes even reset, first getting the LLM to save all the notes it can on progress.

Sonnet 5 (or maybe now 5.5?) was generally pretty good but I just kept seeing those stupid token charges pop into my email when I used it. Trying to save some dough over here! But sometimes I need it and bite the bullet on the $.

[izenflamm]

My experience (M4 max 128GB):

Qwen3.8-27B-oQ8e - slow is smooth, smooth is fast. oQ6e - good balance. oQ4e - may have some problems, but it's fast for dense models.
Qwen3.8-flash-next-oQ4e - far better and faster than Qwen3.8-27B-oQ4e. On par with Qwen3.8-27B-oQ8e, but twice the size and twice faster. In some cases (when nothing else is running on my machine) it can work in parallel. 2-4 in batch. With oMLX 0.7.0 it makes sense thanks to batch request MTP. Great model to run "learning tasks" in Hermes Agent. My main model nowadays.
Qwen3.6-35B-A3B and everything on its base: lack of attention, makes stupid mistakes. Works EXTREMELY MUCH BETTER at bf16, quantization destroys those models. Best of all of them: Ornith-1.5-35B-A3B - stupid, but stubborn in its stupidity. Holds instructions much better than its siblings.
Qwen3.5-9B - ant-worker. Extremely fast. Extremely stupid. Extremely good in instructions following. Lots of finetunes on HF. Can work in packs of 8(!!!!!) on my machine. Great model for simple automations pipelines, which I build sometimes.

Gemma4-E4B (at least oQ8e!!!) - great logic and thinking for it's size, but lacks knowledge. Good when you have enough data in its context. Extremely fast.
Gemma4-12B - a bit smarter ant-worker. Nice vision.
Gemma4-26B-A4B - great for design- and vision-related work. It has taste. Lacks attention though, sometimes skips steps.
Gemma4-31B - wonderful, but you need BF16 to see it, so it's extremely, painfully slow. QAT version disappointed me; it lost depth of thinking! I just keep the full BF16 version on my hard drive for when I need to speak to a professor and encyclopedist in a chat.

For everything else - GLM 5.3 Flash, DeepSeek V4.1 Flash, ChatGPT Luna 6 Pro through Openrouter. Those three can do 98% of my tasks, for a dirt-cheap price. And if my task is important - Opus (the latest is 5.5). I tried Sonnet for agentic work a few times within the last month - Opus could be just slightly more expensive (if the task is too easy for it), but the results are rock solid.

[dshenster-byte]

Thanks for this extensive analysis! Super useful. So many things to check out. thx again!

[dshenster-byte]

Just wanted to share that I am now dropping Ornith 1.5 35B A3B 4bit. It seemed to be doing well and then all of a sudden it started hallucinating big time. And concerning it was hallucinating super early and took me a while to catch that it was saying it was running things that were failing, only that was completely untrue as with me running the same thing in terminal it was totally fine. Not sure if anyone else's experience with Ornith is the same. I'm moving to something else after wasting two days with it on a coding problem.

[q-p]

Caveat: My experience is also relatively limited, but I've played around with these things for the past half year. I'm more in the LLM-assisted "corner" than totally hands-off; I don't trust these parrots enough for that... ;-)

On a 64GB machine (M3 Max and M4 Max), I've generally stuck to 6-bit oQe quants; they're a big jump up from 4-bit for longer context work.

I've so far been quite happy with KAT-Coder-2.5 (jason-schulz/KAT-Coder-V2.5-Dev-VL-oQ6e-mtp), it seemed to do better than the original Qwen-3.6-35B and also all the other fine-tunes I tried (Ornith-1.0, 1.5, Nex-Mini, ... long list). This may be related to my use of Claude as harness, don't know.

But my main driver — even though noticeably slower (the prompt processing is more painful part) — has become Qwen-3.8 27B (since prefix caching is essential, this requires patching the chat template to allow in-place system messages with Claude). It does a lot of back-and-forth thinking (even on medium, which is my default), but the results it comes back with are genuinely impressive. My current problem remains memory spikes during long context PP in oMLX; splash seems much better in this regard for now (from what I understand, the 27B should require model size + 16GB for KV cache, but oMLX allocations can exceed that by a big margin — maybe it's temporaries from the prefix caching or something else).

[dshenster-byte]

Ah good call. I should try 6bit for Ornith 1.5 and see if it does better.

Claude as harness. Didn’t know it could do that. I use Openclaw.

You’ve got a bit more headroom with 64GB than I do at 48GB. Hopefully the 6 bit will do better. I will need to try the Qwen coder series and look into Kat coder. Thx for that pointer!