Skip to content

Architecture

Jump to bottom
github-actions[bot] edited this page Mar 23, 2026 · 20 revisions

Architecture

System Overview

graph TB
    Agent["AI Agent<br/>(Claude, Cursor, etc.)"]
    
    Agent -->|MCP Protocol<br/>stdio or SSE| Server["MCP Server<br/>(FastMCP)"]
    
    Server -->|Tool calls| ToolImpl["Tool Implementations<br/>Core • Discovery • Files<br/>Jobs • Custom • Admin"]
    
    ToolImpl --> SecVal["Security Validator<br/>Function blocklist<br/>Filename sanitization"]
    
    SecVal --> Executor["Job Executor<br/>Sync/async hybrid<br/>Timeout promotion<br/>Progress injection"]
    
    Executor --> PoolMgr["Engine Pool Manager<br/>Elastic scaling<br/>Health checks<br/>Proactive warmup"]
    
    PoolMgr --> Engines["MATLAB Engines<br/>Engine 1 • Engine 2 • ... • N"]
    
    Executor --> Formatter["Result Formatter<br/>Text • Variables • Figures"]
    
    Formatter --> PlotConv["Plotly Converter<br/>Figure extraction<br/>Style mapping<br/>WebGL optimization"]
    
    Formatter --> Output["Output<br/>Text • JSON • Images<br/>Thumbnails"]
    
    Output -->|MCP Response| Agent
    
    style Agent fill:#e1f5ff
    style Server fill:#fff3e0
    style ToolImpl fill:#f3e5f5
    style SecVal fill:#ffebee
    style Executor fill:#e8f5e9
    style PoolMgr fill:#fce4ec
    style Engines fill:#e0f2f1
    style Formatter fill:#f1f8e9
    style PlotConv fill:#ede7f6
    style Output fill:#fff9c4
Loading

Major Components

MCP Server (src/matlab_mcp/server.py)

The central orchestrator built on FastMCP. Responsibilities:

  • Register all 20 built-in tools and dynamically load custom tools from YAML
  • Initialize and wire together all subcomponents (pool, executor, session manager, security validator)
  • Handle server lifecycle events (startup, shutdown, graceful drain)
  • Manage authentication/session context based on transport mode (stdio vs SSE)
  • Optional monitoring dashboard integration

Engine Pool Manager (src/matlab_mcp/pool/manager.py)

Manages a pool of MATLAB engine instances with elastic scaling:

  • Min/max engines: Configurable range (e.g., 2–10 engines)
  • Proactive warmup: When utilization exceeds threshold (default 80%), start a new engine before demand peaks
  • Scale-down: Engines idle longer than timeout (default 15 min) are terminated, never below min_engines
  • Health checks: Periodic 1+1 evaluations verify responsiveness; unhealthy engines are automatically replaced
  • Acquisition queue: Requests wait in an async FIFO queue when all engines are busy; configurable max queue size

Engine Wrapper (src/matlab_mcp/pool/engine.py)

Encapsulates a single matlab.engine instance with lifecycle and state management:

  • State machine: STOPPED → STARTING → IDLE ↔ BUSY
  • Lazy initialization: Engine module only imported on first use (aids testability)
  • Workspace reset: Clears variables/functions/paths between sessions for isolation
  • Background execution: Supports async code evaluation via thread pool
  • Health checks: Self-diagnostic ping; propagates errors to pool for replacement

Job Executor (src/matlab_mcp/jobs/executor.py)

Orchestrates the complete execution lifecycle with hybrid sync/async behavior:

  1. Job creation: Registers job in tracker with unique ID
  2. Engine acquisition: Waits for available engine from pool
  3. Context injection: Injects __mcp_job_id__, __mcp_session_id__, __mcp_temp_dir__ into workspace
  4. Execution: Starts background MATLAB evaluation
  5. Timeout decision:
    • Sync path: Completes within sync_timeout (default 30s) → return result inline
    • Async path: Exceeds timeout → return pending status, background task monitors completion
  6. Result formatting: Calls ResultFormatter to structure output
  7. Cleanup: Resets engine workspace, returns to pool

Job Tracker (src/matlab_mcp/jobs/tracker.py)

Thread-safe in-memory registry for job lifecycle:

  • CRUD operations: Create, retrieve, list, cancel jobs by ID
  • Filtering: Find jobs by session ID for isolated retrieval
  • Pruning: Automatic cleanup of stale terminal jobs after configurable retention period (default 24 hours)
  • State transitions: PENDING → RUNNING → COMPLETED/FAILED/CANCELLED

Session Manager (src/matlab_mcp/session/manager.py)

Manages per-user session isolation:

  • Unique sessions: Each session assigned unique ID and isolated temp directory
  • Activity tracking: Monitors idle time; sessions timeout after inactivity (default 1 hour)
  • Capacity limits: Enforces max concurrent sessions (default 50); evicts least-recently-used on overflow
  • Cleanup: Optionally deletes session temp files on expiration
  • Default session: stdio transport uses a single synthetic session

Security Validator (src/matlab_mcp/security/validator.py)

Pre-execution code inspection and request sanitization:

  • Function blocklist: Detects dangerous MATLAB functions (system, eval, shell escape, etc.) with smart scanning that ignores string literals and comments
  • Filename sanitization: Rejects path-traversal attempts (../../etc/passwd) and non-alphanumeric characters
  • Upload limits: Enforces maximum file size (default 100MB)
  • Violation logging: Records attempts via optional metrics collector

Result Formatter (src/matlab_mcp/output/formatter.py)

Structures raw MATLAB execution results into MCP-compliant response dicts:

  • Text output: Truncates long output (default 50KB inline); saves overflow to files
  • Variable summary: Extracts type, size, and value preview from workspace query
  • Figures: Passes to Plotly converter for interactive JSON generation
  • Files: Lists generated files with paths
  • Errors: Wraps exception details with helpful formatting

Plotly Converter (src/matlab_mcp/output/plotly_convert.py, plotly_style_mapper.py)

Converts MATLAB figures to interactive Plotly JSON via a two-stage pipeline:

MATLAB-side (mcp_extract_props.m):

  • Extracts raw figure properties (axes, line/scatter/bar/surface/heatmap/histogram traces, grid, legend, colormap)
  • Handles multiple layout types: single axes, grid subplots, tiled layouts
  • Detects FastPlot objects for high-resolution streaming data

Python-side (plotly_style_mapper.py):

  • Maps MATLAB line styles, markers, and colormaps to Plotly equivalents
  • Converts RGB arrays to CSS color strings
  • Computes subplot domains (position fractions)
  • WebGL optimization: Uses WebGL for traces with >10,000 points to prevent browser lag

Output:

  • Plotly JSON dict (embeddable in MCP response)
  • Static PNG thumbnail (optional, configurable DPI)
  • Base64 preview for agent display

Data Flow Diagrams

Synchronous Execution (Completes Within Timeout)

sequenceDiagram
    Agent->>+Server: execute_code("x = magic(3)")
    Server->>+SecVal: check_blocked_functions()
    SecVal-->>-Server: OK
    Server->>+PoolMgr: acquire()
    PoolMgr-->>-Server: engine_1
    Server->>+Executor: execute(code, engine_1, session)
    Executor->>Executor: _inject_job_context()
    Executor->>+Engine: eval(code, background=True)
    Engine->>-Executor: Future
    Executor->>Executor: wait(Future, timeout=30s)
    Executor->>Executor: _build_result()
    Executor-->>-Server: {status: "completed", output: "ans = [...]"}
    Server->>+PoolMgr: release(engine_1)
    PoolMgr-->>-Server: OK
    Server-->>-Agent: result
Loading

Asynchronous Promotion (Timeout Exceeded)

sequenceDiagram
    Agent->>+Server: execute_code("long_sim(1000000)")
    Server->>SecVal: check_blocked_functions()
    SecVal-->>Server: OK
    Server->>+PoolMgr: acquire()
    PoolMgr-->>-Server: engine_2
    Server->>+Executor: execute(code, engine_2, session)
    Executor->>Executor: _inject_job_context()
    Executor->>Engine: eval(code, background=True)
    Engine-->>Executor: Future
    Executor->>Executor: wait(Future, timeout=30s)
    Note over Executor: Timeout exceeded!
    Executor->>Executor: Create background task
    Executor-->>-Server: {status: "pending", job_id: "j123"}
    Server-->>-Agent: {status: "pending", job_id: "j123"}
    
    par Background Monitoring
        Executor->>Engine: future.result()
        Note over Executor: Still running...
        Engine->>Engine: Compute progress
        Engine->>Engine: mcp_progress()
        Engine-->>Executor: result
        Executor->>Tracker: mark_completed()
        Executor->>PoolMgr: release(engine_2)
    and Agent Poll Loop
        Agent->>Server: get_job_status("j123")
        Server-->>Agent: {status: "running", progress: 45%}
        Note over Agent: Wait 5 seconds
        Agent->>Server: get_job_status("j123")
        Server-->>Agent: {status: "running", progress: 90%}
        Agent->>Server: get_job_result("j123")
        Server-->>Agent: {status: "completed", output: "..."}
    end
Loading

Figure Generation and Plotly Conversion

sequenceDiagram
    Agent->>+Server: execute_code("plot(x,y); ...")
    Server->>Executor: execute()
    Executor->>+Engine: eval(code, background=True)
    Engine->>Engine: Create figure
    Engine-->>-Executor: result
    Executor->>Executor: mcp_extract_props(fig)
    Engine->>Engine: JSON file to temp_dir
    Executor->>+Formatter: _build_result()
    Formatter->>+PlotlyConv: load_plotly_json()
    PlotlyConv->>PlotlyConv: Parse figure_props.json
    PlotlyConv-->>-Formatter: {data: [...], layout: {...}}
    Formatter->>+PlotlyMapper: convert_traces()
    PlotlyMapper->>PlotlyMapper: Map styles, colormaps
    PlotlyMapper-->>-Formatter: Plotly-formatted traces
    Formatter->>Formatter: Generate thumbnail
    Formatter-->>-Executor: {figures: [{plotly_json, png_thumbnail}]}
    Executor-->>Server: Complete result
    Server-->>Agent: MCP response with figure
Loading

Key Design Decisions and Trade-offs

1. Hybrid Sync/Async Execution

Decision: Code completes synchronously if done within sync_timeout; otherwise auto-promotes to async background job.

Rationale:

  • Speed: Most queries (simple calculations, data reads) complete quickly; agents get instant feedback
  • Long jobs: Simulation, optimization, training doesn't block the pool or agent
  • Simplicity: No need for agents to pre-declare async; the system decides transparently

Trade-off:

  • Requires careful tuning of sync_timeout per deployment
  • Agents must poll for async results (polling loop overhead)
  • Early timeout assumption (agent might be happy to wait 60s, but we default to 30s)

2. Elastic Engine Pool with Proactive Warmup

Decision: Start with min_engines, scale up to max_engines under load, warm up proactively when utilization exceeds threshold.

Rationale:

  • Resource efficiency: Don't pay for 10 engines if you only use 2 most of the time
  • Responsiveness: Proactive warmup (at 80% utilization) avoids queue wait for the next request
  • Headroom: Max engines prevents runaway resource consumption

Trade-off:

  • Adds complexity (state machine, health checks, warmup timing)
  • Potential false warmups on traffic spikes (wasted engine startup cost)
  • Scale-down delay (15 min idle timeout) can leave unused engines running briefly

3. Workspace Isolation Per Session

Decision: Run clear all; fclose all; restoredefaultpath between sessions to isolate user state.

Rationale:

  • Security: User B can't access User A's variables or files
  • Correctness: Each session starts with a clean slate

Trade-off:

  • Loss of workspace context (can't reuse computed values across sessions)
  • Startup delay (restore default path, re-run startup scripts)
  • Requires session-based model (not a shared global workspace)

4. Function Blocklist with Smart Scanning

Decision: Maintain a blocklist of dangerous functions; strip string literals and comments before scanning.

Rationale:

  • Simple allowlist alternative: Blocklist is easier to maintain; users can whitelist MATLAB's vast function library
  • False-positive avoidance: Comments and strings frequently mention "system", "eval", etc.; smart stripping prevents spurious blocks

Trade-off:

  • Incomplete scanning (e.g., dynamically constructed function names via string concatenation bypass the check)
  • Not cryptographically secure (determined attacker can construct obfuscated payloads)
  • Regular maintenance of blocklist as MATLAB evolves

5. In-Memory Job Tracker with Periodic Pruning

Decision: Store job metadata in memory (Python dict); periodically prune stale terminal jobs.

Rationale:

  • Speed: O(1) job lookup; no database latency for active jobs
  • Simplicity: No database dependency for small deployments
  • History: Keep completed jobs for agent query (default 24 hours)

Trade-off:

  • Memory growth: Long-running servers with high job volume may accumulate memory
  • No persistence: Jobs lost on server restart
  • Scaling limit: Single-process limitation (can't shard across multiple server instances without shared store)

Mitigation: Configurable job_retention_seconds; alternative: swap for SQLite-backed store for production.

6. Plotly Conversion via Extracted Properties

Decision: MATLAB-side extraction of figure properties → Python-side Plotly conversion (two-stage pipeline).

Rationale:

  • No screenshot: Avoids rendering PNG and extracting features; pure data conversion
  • Interactive: Plotly JSON enables zoom, pan, legend toggle in agent UI
  • Extensible: Easy to add new plot types by extending plotly_style_mapper.py

Trade-off:

  • Incomplete coverage: Only covers main plot types (line, scatter, bar, surface, heatmap, histogram); custom plot types may not convert
  • Feature loss: Advanced MATLAB figure features (custom callbacks, annotations) not represented in Plotly
  • Complexity: Two-stage pipeline requires coordination between MATLAB and Python

7. SSE Transport Behind Reverse Proxy

Decision: Multi-user deployments use SSE (HTTP) behind a reverse proxy; stdio for single-agent/local.

Rationale:

  • Network: SSE traverses firewalls, NAT, proxies more easily than stdio
  • Auth: Reverse proxy handles authentication once; server stays simple
  • Scalability: Multiple concurrent agents (each with own session)

Trade-off:

  • Setup complexity: Requires reverse proxy (nginx, Caddy, Traefik)
  • Latency: HTTP overhead vs. stdio's direct pipe
  • Security responsibility: Moves authentication burden to operator (server logs warnings if require_proxy_auth not set)

Component Interactions

Tool Implementation Modules

Each tool is implemented as a standalone module under src/matlab_mcp/tools/:

  • core.py: execute_code, check_code, get_workspace
  • jobs.py: get_job_status, get_job_result, cancel_job, list_jobs
  • discovery.py: list_toolboxes, list_functions, get_help
  • files.py: upload_data, delete_file, list_files, read_script, read_data, read_image
  • admin.py: get_pool_status
  • custom.py: Loader and handler factory for YAML-defined custom tools
  • monitoring.py: get_server_metrics, get_server_health, get_error_log (if enabled)

Each tool implementation:

  1. Validates inputs (sanitize filenames, check injection patterns)
  2. Delegates to executor/pool/tracker as needed
  3. Calls formatter to structure output
  4. Returns MCP-compliant response dict

Monitoring & Dashboard (Optional)

If monitoring.enabled: true:

  • MetricsCollector: Accumulates counters (jobs/sessions/errors) and maintains ring buffer of execution times
  • MetricsStore (SQLite): Persists metrics time-series and events for historical queries
  • evaluate_health(): Assesses server status (healthy/degraded/unhealthy) based on pool utilization, error rates, uptime
  • create_monitoring_app(): Starlette sub-application exposing /health, /metrics, /dashboard endpoints
  • Dashboard UI (HTML/CSS/JS): Real-time charts (Plotly), gauges, event log, time-range selector

Summary

The MATLAB MCP Server is a multi-layered system designed for elastic resource management, transparent async promotion, tight security, and extensibility via custom tools. The engine pool handles all complexity; tools focus on MCP protocol details. Monitoring is optional but recommended for production deployments.

Clone this wiki locally