RayServe

Ray Serve — ray-serve-algenta

AlgentaMCPProxy — a byte-transparent, multi-replica HTTP reverse proxy for the engine's /mcp surface, deployed as a Ray Serve deployment with a ready-to-adapt KubeRay RayService manifest. It wraps nothing tool-shaped: it forwards every request at /mcp (method, headers, query, body — streamed both ways) and never parses JSON-RPC or names a tool.

Install

pip install ray-serve-algenta

ray[serve] / fastapi / httpx are real, non-optional runtime dependencies. No GPU required — the proxy does no computation.

Quickstart (local)

# Your engine's MCP endpoint — must be a DIFFERENT host:port from the proxy's bind below.
export ALGENTA_BASE_URL="http://localhost:9000/mcp"
serve run ray_serve_algenta.deployment:app   # binds Ray Serve's own HTTP port, 8000

Point any MCP client at http://localhost:8000/mcp — it talks to it exactly like the engine's own /mcp, because every byte the proxy sees is exactly what it forwards. Default autoscaling config: min_replicas=2, max_replicas=8.

Quickstart (Kubernetes / KubeRay)

kubectl create secret generic algenta-mcp-upstream \
  --from-literal=base-url="http://algenta-engine.your-namespace.svc.cluster.local:8000/mcp"
kubectl apply -f manifests/rayservice.yaml

The manifest sources the upstream URL from a Secret (never a literal endpoint) and documents its prerequisites (KubeRay operator, an image with this package installed) inline.

Profiles

Not applicable — deliberately. The proxy is a dumb pipe by design: teaching it the tool-profile contract would be a second, parallel reimplementation of logic the engine already owns (and the repo's no-engine-dependency gate forbids). Profile enforcement for traffic through this proxy remains the connected engine's job, exactly as for direct traffic.

Notes

• Why scale-out is safe: the engine's /mcp route is stateless Streamable HTTP — no session is pinned to the connection that created it, so every replica is fungible. No sticky sessions, no shared session store.
• Always set ALGENTA_BASE_URL explicitly when running locally: the localhost fallback (http://localhost:8000/mcp) collides with Ray Serve's own default bind, which would make the proxy forward every request to itself.
• Tests: a real ray.init() + serve.run(...) with 2 live replicas in front of a real stub engine, proving no cross-replica state leakage and real multi-replica routing (via the X-Algenta-Ray-Replica header) — uv run pytest ray-serve-algenta -v. Each test costs real wall-clock seconds for cluster start/stop.

Full reference: package README (why a raw proxy, manifest comments, conformance tests).