Skip to content
#

llama-cpp-alternative

Star

Here is 1 public repository matching this topic...

Language: Python
Filter by language
All 3 C++ 2 Python 1

konjoai / squish

Star 7

🤖🗜️⚡️ Local LLM server for Apple Silicon. 5.4× faster end-to-end on long contexts vs Ollama, 33% less RAM, INT3 support for Qwen3. OpenAI + Ollama drop-in. Built for repeated long-context workloads on memory-constrained Macs.

macos rust metal self-hosted quantization mlx inference-engine kv-cache apple-silicon openai-api int4 llm local-llm local-ai ollama qwen llm-infernece on-device-inference ollama-alternative llama-cpp-alternative
  • Updated Jun 2, 2026
  • Python

Improve this page

Add a description, image, and links to the llama-cpp-alternative topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the llama-cpp-alternative topic, visit your repo's landing page and select "manage topics."

Learn more