Skip to content

v0.2.6 - Managed Sessions API

Choose a tag to compare

View all tags
@Lincyaw Lincyaw released this 08 Mar 15:51
· 300 commits to main since this release
v0.2.6
a9c3d75

Managed Sessions API — Server-Side Pool Lifecycle Management

Highlights

Clients can now create sandbox sessions by specifying just an image and experiment ID — no need to manually create, scale, or delete WarmPools. The server handles pool lifecycle automatically.

New API Endpoints

Method Path Description
POST /v1/managed/sessions Create a managed session (image + experimentId → sessionId)
GET /v1/managed/experiments/{id}/sessions List all active sessions for an experiment
DELETE /v1/managed/experiments/{id} Batch-delete all sessions for an experiment

Session operations (execute, restore, shell, history, trajectory) use the existing /v1/sessions/{id}/* endpoints as before.

Python SDK

New ManagedSession class for simplified usage:

from arl import ManagedSession

# Before (6 steps): create_pool → wait_for_ready → SandboxSession → execute → delete_session → delete_pool
# Now (1 step):
with ManagedSession(image="python:3.11-slim", experiment_id="swe-bench-42") as s:
    result = s.execute([{"name": "test", "command": ["python", "-c", "print('hello')"]}])
    print(result.results[0].output.stdout)

# Batch cleanup
from arl import GatewayClient
client = GatewayClient(base_url="http://localhost:8080")
client.delete_experiment("swe-bench-42")

Server-Side Auto-Scaling

  • Auto-create: Pool created on first request for an image
  • Demand-based scale-up: Replicas increase based on active session count when no idle pods are available
  • Cooldown-based scale-down: Excess idle pods removed after configurable cooldown (default: 5m)
  • Pool GC: Empty pools (0 sessions) deleted after TTL (default: 10m)
  • Crash recovery: Pool manager state rebuilt from K8s CRDs on gateway restart

Configuration

Env Variable Default Description
MANAGED_POOL_INITIAL_REPLICAS 2 Initial replicas when pool is first created
MANAGED_POOL_MIN_REPLICAS 0 Scale-down floor
MANAGED_POOL_MAX_REPLICAS 50 Scale-up ceiling
MANAGED_POOL_SCALE_UP_STEP 2 Minimum replicas added per scale-up
MANAGED_POOL_IDLE_COOLDOWN 5m Idle duration before scale-down
MANAGED_POOL_EMPTY_TTL 10m How long to keep a pool with 0 sessions
MANAGED_POOL_SWEEP_INTERVAL 30s Background sweep interval

Backward Compatibility

  • All existing /v1/sessions/* and /v1/pools/* endpoints unchanged
  • New endpoints under /v1/managed/ prefix only
  • Managed pools isolated from manually created pools via arl.infra.io/managed label
  • Existing SandboxSession SDK class works identically

Other Changes

  • Added Grafana dashboard panels for sandbox retry rate and workqueue depth
  • Increased QPS limits for high-throughput trajectory collection (512+ pods)
Assets 2
Loading