This repository was archived by the owner on Nov 7, 2026. It is now read-only.
Repository navigation
🚀 Glacier Distributed Cluster v1.2.0 Release
🌐 Drop-In OpenAI & Ollama Compatible HTTP Service (--serve)
The cluster coordinator can now be launched as an OpenAI and Ollama compatible HTTP service, allowing external client applications to connect without writing code or compiling binaries:
\\�ash
glacier-distributed serve
--model ./models/qwen2-57b-a14b-instruct-q3_k_m.gguf
--stages 192.168.1.108:5055,192.168.1.101:5056,192.168.1.200:5057
--http-port 8080
--host 0.0.0.0
\\
✨ Key Features & Endpoints
- OpenAI Chat Completions (\POST /v1/chat/completions): Full support for streaming Server-Sent Events (\stream: true) and non-streaming JSON responses.
- OpenAI Text Completions (\POST /v1/completions): Text completion endpoint with streaming SSE.
- OpenAI Models (\GET /v1/models): Model discovery endpoint returning cluster model metadata.
- Cluster Status & Telemetry (\GET /v1/cluster/status, \GET /health): Detailed node topology, layer allocations, and backend hardware info.
- Ollama Compatibility (\POST /api/chat, \POST /api/generate, \GET /api/tags): Drop-in compatibility for tools designed for Ollama.
- Security & Authentication: Optional API key authentication via Bearer token or \x-api-key\ header.
- Client Integration: Tested with OpenAI Python SDK (\openai.OpenAI(base_url='http://localhost:8080/v1')\), LangChain, AutoGen, Open WebUI, Continue.dev, and Cursor.
📦 Pre-compiled Standalone Native Binaries Included
- \glacier-distributed-v1.2.0-win-x64.zip\ (Windows x64)
- \glacier-distributed-v1.2.0-win-arm64.zip\ (Windows ARM64)
- \glacier-distributed-v1.2.0-linux-x64.tar.gz\ (Linux x64)
- \glacier-distributed-v1.2.0-osx-arm64.tar.gz\ (macOS Apple Silicon)
📖 Read the deep dive article on DEV.to: Want to Run Massive AI Models on a Collection of Home PCs?
Full Changelog: v1.1.0...v1.2.0