Skip to content
This repository was archived by the owner on Nov 7, 2026. It is now read-only.

v1.2.0 - OpenAI & Ollama Compatible Distributed Cluster HTTP Service

Latest

Choose a tag to compare

@ian-cowley ian-cowley released this 12 Oct 20:16
· 13 commits to main since this release

🚀 Glacier Distributed Cluster v1.2.0 Release

🌐 Drop-In OpenAI & Ollama Compatible HTTP Service (--serve)

The cluster coordinator can now be launched as an OpenAI and Ollama compatible HTTP service, allowing external client applications to connect without writing code or compiling binaries:

\\�ash
glacier-distributed serve
--model ./models/qwen2-57b-a14b-instruct-q3_k_m.gguf
--stages 192.168.1.108:5055,192.168.1.101:5056,192.168.1.200:5057
--http-port 8080
--host 0.0.0.0
\\

✨ Key Features & Endpoints

  • OpenAI Chat Completions (\POST /v1/chat/completions): Full support for streaming Server-Sent Events (\stream: true) and non-streaming JSON responses.
  • OpenAI Text Completions (\POST /v1/completions): Text completion endpoint with streaming SSE.
  • OpenAI Models (\GET /v1/models): Model discovery endpoint returning cluster model metadata.
  • Cluster Status & Telemetry (\GET /v1/cluster/status, \GET /health): Detailed node topology, layer allocations, and backend hardware info.
  • Ollama Compatibility (\POST /api/chat, \POST /api/generate, \GET /api/tags): Drop-in compatibility for tools designed for Ollama.
  • Security & Authentication: Optional API key authentication via Bearer token or \x-api-key\ header.
  • Client Integration: Tested with OpenAI Python SDK (\openai.OpenAI(base_url='http://localhost:8080/v1')\), LangChain, AutoGen, Open WebUI, Continue.dev, and Cursor.

📦 Pre-compiled Standalone Native Binaries Included

  • \glacier-distributed-v1.2.0-win-x64.zip\ (Windows x64)
  • \glacier-distributed-v1.2.0-win-arm64.zip\ (Windows ARM64)
  • \glacier-distributed-v1.2.0-linux-x64.tar.gz\ (Linux x64)
  • \glacier-distributed-v1.2.0-osx-arm64.tar.gz\ (macOS Apple Silicon)

📖 Read the deep dive article on DEV.to: Want to Run Massive AI Models on a Collection of Home PCs?

Full Changelog: v1.1.0...v1.2.0