Skip to content

v0.3.0-alpha — First public release

Pre-release
Pre-release

Choose a tag to compare

View all tags
@ghbalf ghbalf released this 19 Mar 00:01
· 18 commits to main since this release
v0.3.0-alpha
889c220

RLM Proxy v0.3.0-alpha

Multi-provider LLM proxy implementing the Recursive Language Model inference paradigm (Zhang, Kraska, Khattab — MIT CSAIL, 2025).

Highlights

  • RLM scaffold — sandboxed Python REPL, parallel sub-calls, LRU caching, smart history compaction, adaptive early-stop
  • Multi-provider dispatch — Ollama + OpenAI-compatible APIs with provider/model routing, auto-discovery, least-loaded balancing
  • Dual API — OpenAI (/v1/*) and Ollama (/api/*) endpoints, both fully dispatched
  • Admin dashboard — web UI at /admin for managing providers, settings, and live metrics
  • Production hardening — API key auth, request queue, retry with exponential backoff, circuit breaker, token budgets

Quick Install

curl -fsSL https://raw.githubusercontent.com/ghbalf/rlm_proxy/main/install.sh | bash

Or manually:

git clone https://github.com/ghbalf/rlm_proxy.git
cd rlm_proxy
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
python main.py
# Open http://localhost:8881/admin

What's alpha about it

  • Tested with local Ollama; remote OpenAI-compatible providers implemented but need broader testing
  • Anthropic provider planned but not yet implemented
  • No WebSocket streaming support yet
  • Admin UI is functional but basic

See README.md and INSTALL.md for full documentation.

Assets 2
Loading