Another Project Inspired by Colibri

[drmihirbrahme]

See GPT OSS 120B running on a machine with 8/ 16GB RAM.
Project: github.com/drmihirbrahme/midge

[tim-lab-57]

Found another, Tencent Hy3:
https://github.com/ErikTromp/colibri-hy3

[mkultra333]

I downloaded the model, compiled that version and gave it a try, but it didn't work. Just hangs at the "waking the giant" loading phase but never actually loads anything. So it's a bit of a waste of downloading time at the moment, hopefully he does some more work on it and opens the issues/discussion up.

Edit: Managed to get it working, cpu at least, after a chat with Hy3 (Free) on OpenRouter. It's a small fix to do with the windows version that's already patched over here. To quote Hy3:

The fix (2 lines, in hy3.c main() or run_serve)
Add the Windows binary-stdout/stdin fix to hy3.c. Right at the start of main() (or inside run_serve before the printf("...\n")), add:

#ifdef _WIN32
#include <fcntl.h>
_setmode(_fileno(stdin),  _O_BINARY);
_setmode(_fileno(stdout), _O_BINARY);
setvbuf(stdout, NULL, _IONBF, 0);
#endif

(_O_BINARY comes from <fcntl.h>; on Windows that's where it lives. The GLM fork relies on it being pulled in transitively — safer to #include <fcntl.h> near the top of hy3.c.)

[Pummelchen]

Another CPU only project for Qwen3.6 27B where the model gets converted into CPU core optimized datasets to meet GPU performance:

https://github.com/Pummelchen/XAIOS

[drmihirbrahme]

Nice to see people getting inspired by Colibri and Midge.

Quick update: a new release of Midge is up, which has taken it to another level— adding a cloud route for faster performance and support for pretty much any model listed on Hugging Face.

[rofl0r]

Project: github.com/drmihirbrahme/midge

fixing the link for you: https://github.com/drmihirbrahme/midge .

got any benchmark numbers for qwen3.6-35B-A3B vs llama.cpp/ik_llama.cpp (on CPU)?