Async Jobs

Async Jobs

Long-running MATLAB code is automatically handled through the async job system.

How It Works

1. You call execute_code with your MATLAB code
2. The server starts executing synchronously
3. If execution exceeds sync_timeout (default 30 seconds), the job is promoted to async
4. You get back a job_id immediately
5. Poll get_job_status for progress updates and job status
6. Call get_job_result when the job completes

Job Lifecycle

PENDING → RUNNING → COMPLETED
                  → FAILED
                  → CANCELLED

State Transitions

• PENDING: Job created, waiting to acquire an engine
• RUNNING: Engine acquired, code is executing (either sync or async background)
• COMPLETED: Code finished successfully with a result
• FAILED: Code execution raised an error
• CANCELLED: Job was explicitly cancelled via cancel_job

Sync vs. Async Execution

Synchronous Execution (< sync_timeout)

1. Job is created and marked RUNNING
2. An engine is acquired from the pool
3. Job context is injected into the MATLAB workspace
4. Code executes in the foreground via engine.execute(code, background=True)
5. Server waits up to sync_timeout seconds (default 30s)
6. If code completes within the timeout:
   • Result is captured immediately
   • Job is marked COMPLETED
   • Engine is released back to the pool
   • Response includes full result: {"status": "completed", "job_id": "...", ...result}

Async Promotion (≥ sync_timeout)

1. If foreground execution exceeds sync_timeout, the job is promoted to async
2. A background task is spawned to continue monitoring the execution
3. Server releases the engine back to the pool (other requests can use it)
4. Response returns immediately: {"status": "pending", "job_id": "..."}
5. Client polls get_job_status to monitor progress
6. When execution completes (or fails), the background task:
   • Captures the final result
   • Marks job COMPLETED or FAILED
   • Releases the engine

Progress Reporting

Use the mcp_progress() helper function in your MATLAB code to report execution progress back to the agent:

mcp_progress(__mcp_job_id__, percentage, message)

• __mcp_job_id__ — automatically injected into the workspace by the server
• percentage — number from 0 to 100
• message — optional status message (string)

How Progress Works Internally

1. mcp_progress.m writes a JSON file to __mcp_temp_dir__/<job_id>.progress
2. get_job_status reads this file and includes progress and message in the response
3. The progress file is automatically cleaned up when the job completes
4. Progress updates are useful for long-running jobs to keep the agent informed

Example

n = 1e6;
results = zeros(n, 1);
for i = 1:n
    results(i) = process_item(i);
    if mod(i, 1e5) == 0
        mcp_progress(__mcp_job_id__, i/n*100, ...
            sprintf('Processed %d/%d items', i, n));
    end
end
disp(mean(results));

The agent sees:

get_job_status → {status: "running", progress: 10, message: "Processed 100000/1000000 items"}
get_job_status → {status: "running", progress: 50, message: "Processed 500000/1000000 items"}
get_job_status → {status: "running", progress: 100, message: "Processed 1000000/1000000 items"}
get_job_result → {status: "completed", output: "0.5023", ...}

Job Status Polling

Call get_job_status to check the current status of a job:

{
  "status": "running",
  "job_id": "j-abc123...",
  "progress": 45,
  "message": "Processing batch 5 of 10"
}

Fields:

• status — Current job status (pending, running, completed, failed, cancelled)
• job_id — The job identifier
• progress — Percentage 0-100 (if mcp_progress() was called)
• message — Status message (if mcp_progress() was called)
• error — Error details if status is "failed"

Job Result Retrieval

Call get_job_result to retrieve the complete result of a completed job:

{
  "status": "completed",
  "job_id": "j-abc123...",
  "text": "ans =\n    0.5023",
  "workspace": {
    "results": [[...array data...]],
    "n": 1000000
  }
}

Fields:

• status — Terminal status (completed, failed, or cancelled)
• job_id — The job identifier
• text — Captured stdout from execution
• workspace — Variables that were assigned during execution
• error — Error details if status is "failed" (includes type, message, matlab_id, stack_trace)

Note: Results are retained for job_retention_seconds (default 86400 / 24 hours) after completion. Older job results are automatically pruned.

Job Cancellation

Call cancel_job to cancel a PENDING or RUNNING job:

{
  "status": "cancelled",
  "job_id": "j-abc123..."
}

• Only jobs in PENDING or RUNNING status can be cancelled
• Cancelling a RUNNING job stops the MATLAB code execution
• The job transitions to CANCELLED status
• The result cannot be retrieved after cancellation

Job Listing

Call list_jobs to retrieve all jobs in the current session:

{
  "jobs": [
    {
      "job_id": "j-abc123...",
      "status": "completed",
      "created_at": 1699564800.5,
      "started_at": 1699564801.2,
      "completed_at": 1699564810.8,
      "elapsed_seconds": 9.6
    },
    {
      "job_id": "j-def456...",
      "status": "running",
      "created_at": 1699564815.1,
      "started_at": 1699564815.9,
      "elapsed_seconds": 120.3
    }
  ]
}

Each job entry includes timing information and current status.

Retention and Cleanup

The job tracker maintains both active and historical job metadata:

• Active jobs: PENDING and RUNNING jobs are kept indefinitely while active
• Terminal jobs: COMPLETED, FAILED, and CANCELLED jobs are retained for job_retention_seconds (default 86400 / 24 hours)
• Automatic pruning: The tracker periodically removes expired terminal jobs to prevent unbounded memory growth
• Session isolation: Each session maintains its own job history

To adjust retention, configure:

sessions:
  job_retention_seconds: 86400  # 24 hours (default)

Job Management Tools

Tool	Description
execute_code	Submit code for execution (returns immediately with job_id)
get_job_status	Current status, progress %, and message for a job
get_job_result	Full result of a completed/failed/cancelled job
cancel_job	Cancel a pending or running job
list_jobs	List all jobs in the session with timing info

Configuration

execution:
  sync_timeout: 30           # Seconds before async promotion (default)
  max_execution_time: 86400  # Hard limit on total execution time (24h default)

sessions:
  job_retention_seconds: 86400  # Keep terminal job metadata for 24h (default)

Tips

• Short code (< 30s): Results return inline in the execute response, no polling needed
• Medium code (30s - minutes): Auto-promoted to async, poll with get_job_status for progress
• Long code (hours): Add mcp_progress() calls so the agent can monitor execution and estimate completion time
• Cancel long jobs: Call cancel_job if you need to stop a running job and free the engine
• Increase timeout: Set sync_timeout: 60 (or higher) if most of your code takes 30-60 seconds, to avoid unnecessary promotion
• Check retention: Monitor list_jobs to see how much history is retained; adjust job_retention_seconds if needed