[FEATURE] Integrate llama.cpp as a Quantizer

[sdiazlor]

‼️ If you want to work on this issue: please comment below and wait until a maintainer assigns this issue to you before opening a PR to avoid several contributions on the same issue. Thanks! 😊

✨ What You’ll Do

Right now, we have quantizers like Quanto, torchao or HQQ. It would be nice to integrate llama.cpp as a new quantizer.

🤖 Useful Resources

• Feel free to first consult how to include a new algorithm in pruna and the current implementations: https://docs.pruna.ai/en/stable/docs_pruna/user_manual/customize_algorithm.html
• https://github.com/ggml-org/llama.cpp
• Develope the integration.
• Test it and write the documentation.

✅ Acceptance Criteria

• Tests & Docs: All existing and new unit tests pass, and the documentation is updated

And don’t forget to give us a ⭐️!

---

❓ Questions?

Feel free to jump into the #contributing Discord channel if you hit any roadblocks. Can’t wait to see your contribution! 🚀

---

Share on Socials

[shreyasethu]

Hi, I would like to take this up!

[davidberenstein1957]

@shreyasethu would you still like to work on this?

[shreyasethu]

Hi! Extremely sorry for the lack of updates, I had university finals going on. I finally picked this back up a week ago to test the initial implementation that I made back in october and ran into a lot of blockers 😬

mainly around GGUF conversion for llama.cpp and locating converter scripts (other Pruna quantizers export models in standard HF/diffusers formats, llama.cpp produces GGUF files for a separate runtime, which changes how the entire Pruna workflow handles export, quantization and inference)

It took me a while to understand, as this was my first time trying to contribute to open source. I can still try working on it, if someone else doesn't take it up, thank you so much opportunity though 😊

[krishjp]

@davidberenstein1957 Is this issue still active? I can take a look and create an algo class using the PrunaAlgorithmBase if it is!

[davidberenstein1957]

Yes, feel free to pick this up if you've got the time :)El 19 mar 2026, a las 22:39, Krish Patel ***@***.***> escribió:krishjp left a comment (PrunaAI/pruna#377) @davidberenstein1957 Is this issue still active? I can take a look and create an algo class using the PrunaAlgorithmBase if it is! —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>