Optional API serving layer (api/ module)

[baabakk]

Proposal: Optional API serving layer (api/ module)

Hi Alex!

I recently built a production-oriented HTTP serving layer for your RLM work and wanted to gauge interest before opening a PR. The design intentionally keeps the core library untouched; everything lives in a separate optional api/ module.

Motivation

In my use case I needed a reliable way to serve RLM.completion() behind an HTTP interface while respecting some constraints:

• completion() is synchronous
• It mutates process-level state (signals, stdout/stderr redirection, working directory)
• Runs can take seconds to minutes

That makes naive async/thread approaches risky in multi-user environments, so I opted for a process-isolated queue architecture.

What the API layer adds

An optional api/ directory (~1.4k LOC) providing:

• FastAPI HTTP layer with job queue semantics
• Redis Streams for durable job delivery
• Dedicated worker processes (one job per worker)
• Polling endpoints plus SSE streaming of iteration events
• Bearer token auth, rate limiting, idempotency keys
• Docker Compose setup for API + workers + Redis

Each worker instantiates a fresh RLM instance per job, avoiding shared-state issues.

Architecture (high level)

Client → FastAPI → Redis Stream → Worker process → RLM.completion()
Workers publish progress events → Redis Pub/Sub → SSE streaming to clients.

Recent improvements before proposing upstream

To make this more upstream-friendly I’ve addressed a few gaps:

• Added API test coverage (auth, queue, rate limiting, workers, routes, SSE)
• Declared API dependencies as an optional extras group in pyproject.toml
• Fixed backend option allow-list mismatch (base_url)
• Improved worker health reporting to reflect actual queue consumption, not just process liveness

All changes remain isolated to the api/ module.

Scope philosophy

• Zero modifications to rlm/ core
• Optional install (pip install rlm[api])
• Environment-driven configuration
• No additional burden for users who just want the library

Question

Would this be something you’d consider upstream as an optional module, or would you prefer it stay external and possibly just be linked as an ecosystem add-on?

Either direction works for me, I mainly wanted alignment before opening a large PR.

Fork:
https://github.com/baabakk/rlm/tree/main