Hello Does it’ll work on MacBook m5 32gb unified memory?

[salmalgesiddhant]

Does it’ll work on MacBook m5 32gb unified memory?

[JustVugg]

Yes — an M5 MacBook with 32GB unified memory is exactly the machine Colibrì is built for. GLM-5.2 is a 744-billion-parameter model, but Colibrì streams its experts from your SSD and keeps only a small working set in memory, so it runs in ~25–32GB instead of the ~400GB the full model would otherwise need.

To run it:

• Build with the Metal backend: cd c && make colibri METAL=1 (macOS only, no Xcode needed — the shader compiles at runtime).
• You'll need about 372GB of free SSD space for the int4 model; grab the group-scaled container mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp (recommended for quality).
• Then COLI_METAL=1 COLI_MODEL=/path/to/model ./coli chat.

What to expect on speed: it's disk-bound, so it's not real-time-chat fast — on Apple Silicon with the fast internal SSD, expect roughly 1–2 tok/s (an M5 Max reached 2.06 tok/s, #103; a 32GB M5 will be a bit lower since less stays cached in unified memory). The trade is that you run a frontier model at all, fully offline and private, on a laptop — great for coding, reasoning, and agent tasks where a short wait is fine.

If you try it, an M5/32GB datapoint would be very welcome in the benchmark tracker. 🙂

[tonybarbieri]

I'm running a 128 GB M5 mac with 4 TB disk. I have everything built and running, including the Metal backend. What settings are best to maximize the available RAM (~96-110GB dedicated to the model)? Running it with COLI_METAL=1 ./coli chat --ram 110 doesn't seem to respond very quickly.

The response prints to the terminal very slowly. Is this expected? Of course I don't expect real time but it seems slower than I might be able to optimize for with the available machine resources. The model resides on the built-in nvme storage.

[JustVugg]

Try

COLI_METAL=1 DIRECT=1 MTP=0 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=8 ./coli chat --ram 110

[amateurakhbar]

Please share your results with 32 GB RAM M5 Macbook since I'm about to order and I intend to use this as well.