Skip to content

AI Local Development Models

Jump to bottom
Spinning Idea edited this page Jul 30, 2026 · 2 revisions

LLM Models for Local Development

Below are a small set of models that can work for local development given a certain hardware configuration

LLM Model Directories

Community Benchmarks & Reviews

Top Local Coding Models (July 2026 Benchmarks)

Evaluated based on SWE-bench rankings, agentic terminal capability, Fill-In-the-Middle (FIM) speed, and tool calling:

  • Laguna XS 2.1 (33B MoE / 3B Active)
    • Model: Laguna XS 2.1 Q4_K_M
    • Details: 33B total parameter MoE model with only 3B activated parameters per token. Delivers +5.4% higher scores on SWE-bench Multilingual and superior performance on long-horizon agent tasks.
    • Run: ollama run laguna-xs-2.1:q4_K_M
  • Devstral Small 2 (24B by Mistral AI & All-Hands AI)
    • Details: Purpose-built for autonomous software engineering. Scores ~68% on SWE-bench Verified (Apache 2.0 license, 128k context). Fits 16GB VRAM at Q4_K_M (~14GB).
    • Run: ollama run devstral-small-2
  • Qwen 3.6 27B A3B Coder
    • Details: SOTA local coding model optimized for complex code synthesis, multi-file refactoring, and agentic workflows.
    • Run Commands:
      • ollama run mannix/qwen3.6-27b-a3b-coder:CD-IQ4_K_M
      • ollama run mannix/qwen3.6-27b-a3b-coder:CD-Q3_K_L
  • Qwen 2.5 Coder 14B
    • Details: Benchmark standard for 16GB VRAM setups. Occupies ~9GB at Q4_K_M, leaving 6GB+ for massive KV cache and instant completion speed (~35–40 tok/sec).
    • Run: ollama run qwen2.5-coder:14b
  • Qwythos-9B v2 / Claude-Mythos 5 (1M Context GGUF)
  • Gemma 4 12B (Google DeepMind)
    • Details: Encoder-free multimodal model (text, vision, audio) with 256k context window and Multi-Token Prediction (MTP) acceleration in Ollama (~90% speedup).
    • Run: ollama run gemma4:12b
  • LFM 2.5-8B-A1B
    • Model: LFM 2.5 8B Q8_0
    • Details: ~9GB memory footprint. Specialized edge model for fast, deterministic tool calling and function execution.
    • Run: ollama run lfm2.5:8b-a1b-q8_0
  • Codestral 22B (Mistral AI)
    • Details: Top-tier Fill-in-the-Middle (FIM) code completion model (~12.5GB VRAM at Q4_K_M).
    • Run: ollama run codestral:22b
  • DeepSeek-Coder-V2 Lite (16B MoE / 2.4B Active)
    • Details: Lightweight MoE model running at ~45 tok/sec; excellent low-latency choice for interactive agent loops.
    • Run: ollama run deepseek-coder-v2:16b

ThinkingCap-Qwen3.6-27B

ThinkingCap, which is a fine-tune of the Qwen3.6-27B model.

Quantized Versions

Clone this wiki locally