Server and Remote Execution

Server and Remote Execution

mlpl-serve turns the evaluator into a long-lived authenticated service. It is the bridge between browser ergonomics and native files, persistent state, CPU throughput, MLX, or CUDA.

Topology

flowchart TD
    Browser[Connected browser] --> API[mlpl-serve REST and SSE API]
    Terminal[mlpl-repl connect mode] --> API
    API --> Sessions[Authenticated sessions]
    Sessions --> CPU[In-process CPU evaluator]
    Sessions --> CUDA[Optional in-process CUDA]
    Sessions --> Router[Device peer router]
    Router --> MLX[mlpl-mlx-serve]
    Sessions --> Persist[Session persistence]
    Sessions --> Viz[Content-addressed visualization store]

Loading

Main flow

sequenceDiagram
    participant C as Client
    participant S as mlpl-serve
    participant E as Session evaluator
    participant D as Device backend
    C->>S: Create or reattach session
    S-->>C: Session id and bearer token
    C->>S: Submit program over eval stream
    S->>E: Evaluate with interrupt token
    E->>D: Dispatch CPU or device work
    D-->>E: Values and metrics
    E-->>S: Streaming metric events
    S-->>C: SSE metrics and final result
    C->>S: Fetch stored visualization
    S-->>C: SVG, HTML, image, or JSON artifact

Loading

Server capabilities

• Create sessions, evaluate programs, inspect workspace state, and check health.
• Stream metrics with Server-Sent Events.
• Cooperatively cancel an active evaluation.
• Persist sessions to disk and reattach after a client/server restart.
• Store SVG, HTML, PNG, JPEG, and JSON visualization artifacts by content hash.
• Serve static web assets and accept connected browser clients.
• Configure an explicit CORS allow-list.
• Report available devices so the UI can gate demos truthfully.
• Route MLX work to a peer service; build CUDA directly into the server on the current CUDA host path.

Security model

• Use bearer tokens and constant-time comparison.
• Keep loopback binding for local use.
• Require authentication for non-loopback exposure.
• Restrict CORS to known origins.
• Treat GPU peers as trust boundaries. The server does not expose an LLM reverse proxy.
• Do not place secrets inside MLPL source or browser bundles.

Operational choices

Use a local server when you want persistence or a browser UI on the same machine. Use a LAN server when the accelerator or data is elsewhere. Route whole device blocks rather than individual operations when using a peer, and materialize remote tensors only when the client needs CPU data.

Source references: CLI server guide, MLX service guide, and components/serve.