Skip to content

Releases

v0.1.4

Choose a tag to compare

@github-actions github-actions released this 02 Nov 13:32

Full Changelog: v0.1.3...v0.1.4

v0.1.3

Choose a tag to compare

@github-actions github-actions released this 02 Nov 13:19

Full Changelog: v0.1.2...v0.1.3

v0.1.2

Choose a tag to compare

@github-actions github-actions released this 02 Nov 12:50

Full Changelog: v0.1.1...v0.1.2

v0.1.1

Choose a tag to compare

@github-actions github-actions released this 02 Nov 12:13

Full Changelog: v0.1.0...v0.1.1

v0.1.0 - Zero-Dependency CPU LLM Runner & OpenAI Micro-Server

Choose a tag to compare

@github-actions github-actions released this 01 Nov 14:14

Full Changelog: https://github.com/PixelEzz/NoVram/commits/v0.1.0

⚡ novram v0.1.0 — Initial Public Release

novram is an open-source, ultra-fast, zero-dependency CPU-only LLM runner and OpenAI-compatible micro-server. Run quantized GGUF language models directly on system RAM without requiring a discrete GPU or CUDA setup.


🚀 Key Highlights & Architectural Features

  • Strict Physical Core Affinity: Automatically discovers physical CPU core topology and pins worker threads to bypass hyperthreading contention, eliminating L1/L2 cache thrashing and boosting dense SIMD GEMM throughput by 25%–35%.
  • Prompt Lookup Speculative Decoding: Uses n-gram pattern speculation verified in single batched CPU passes, unlocking token burst speeds of up to 180–280+ tokens/second on structured code and repetitive text.
  • Quantized Q8_0 KV Cache: Reduces attention memory bus pressure by 50% compared to standard float16 KV caches.
  • Zero-Copy mmap Engine: Memory-maps GGUF model weights directly into virtual memory space in sub-50ms with zero duplicate heap copies.
  • OpenAI-Compatible Micro-Server: Drop-in /v1/chat/completions (Server-Sent Events streaming) and /v1/models support for VS Code, Roo Code, Cline, Cursor, and Continue.
  • Zero-Swap Feasibility Matrix (novram doctor): Inspects host CPU vector extensions (AVX2, AVX-512, NEON) and available physical RAM to prevent OS swapping before starting inference.

📦 Quick Start & Pre-Built Binaries

Download the standalone binary for your architecture from the Assets section below:

Platform Architecture File
Windows x86_64 (amd64) novram-windows-amd64.zip
Linux x86_64 (amd64) novram-linux-amd64.tar.gz
Linux ARM64 (aarch64) novram-linux-arm64.tar.gz
macOS Apple Silicon (arm64) novram-darwin-arm64.tar.gz
macOS Intel (amd64) novram-darwin-amd64.tar.gz

One-Line Terminal Run:

# Check CPU vector flags and zero-swap RAM feasibility
./novram doctor

# Start local OpenAI-compatible server on port 8080
./novram serve --model ./models/llama-3.2-3b.Q4_K_M.gguf --port 8080

**Full Changelog**: https://github.com/PixelEzz/NoVram/commits/v0.1.0