Video Perception MCP

Bun-based MCP server for analyzing video with fal's OpenRouter video endpoint.

Features

• MCP tools: submit_video_analysis, video_analysis_status, get_video_analysis_result, extract_frame, extract_frames, video_info
• Deprecated sync tool: analyze_video
• Accepts public video URLs, supported YouTube links, data URIs, or local video paths
• Uploads local files to fal storage automatically
• Uses google/gemini-3-flash-preview by default
• Can switch to google/gemini-3.1-pro-preview with model: "pro" or automatic prompt heuristics
• Supports optional system_prompt and temperature
• Always sends reasoning: true because the fal video endpoint currently requires it
• Uses an internal max_tokens limit of 4096
• Uses submit/status/result tools as the canonical flow so slow fal/OpenRouter requests do not duplicate when retried after a client timeout
• Keeps analyze_video only for backwards compatibility; it may hit client timeouts on slow requests
• Extracts local video frames to disk with ffmpeg

Setup

bun install
export FAL_KEY="your_fal_api_key"
bun run start

MCP Client Config

Use an absolute path for the project directory:

{
  "mcpServers": {
    "video-perception": {
      "command": "bun",
      "args": ["src/index.ts"],
      "cwd": "/path/to/this/repository/VideoPerceprionMCP",
      "env": {
        "FAL_KEY": "your_fal_api_key"
      }
    }
  }
}

Tool Input Examples

Submit a URL analysis:

{
  "video_urls": ["https://example.com/video.mp4"],
  "prompt": "Describe the scene and list important events with timestamps."
}

Submit a local file analysis:

{
  "video_paths": ["/absolute/path/to/video.mp4"],
  "prompt": "Transcribe speech and summarize visible actions.",
  "model": "pro"
}

Submit a long-running analysis without waiting for the final model response:

{
  "video_paths": ["/absolute/path/to/video.mp4"],
  "prompt": "Analyze the gameplay and draft an ASO description.",
  "model": "flash"
}

Then check status:

{
  "request_id": "019e8851-85a8-7e50-b539-ebf614f0fdd9"
}

Fetch the completed result:

{
  "request_id": "019e8851-85a8-7e50-b539-ebf614f0fdd9",
  "model": "google/gemini-3-flash-preview"
}

Deprecated synchronous analysis:

{
  "video_paths": ["/absolute/path/to/video.mp4"],
  "prompt": "Transcribe speech and summarize visible actions.",
  "model": "pro"
}

Use analyze_video only when you explicitly want one blocking call and can tolerate client timeout risk. If the client times out, retrying with another analysis call can submit a second paid fal request.

Extract one frame:

{
  "video_path": "/absolute/path/to/video.mp4",
  "mode": "timestamp",
  "timestamp": "00:00:05.000",
  "output_path": "/private/tmp/frame.png"
}

Extract by zero-based frame number:

{
  "video_path": "/absolute/path/to/video.mp4",
  "mode": "frame",
  "frame_number": 120
}

Extract multiple frames:

{
  "video_path": "/absolute/path/to/video.mp4",
  "mode": "timestamp",
  "timestamps": ["00:00:01", "00:00:05.500", "12.5"],
  "output_dir": "/private/tmp/video-storyboard",
  "image_format": "png"
}

Extract multiple frame numbers:

{
  "video_path": "/absolute/path/to/video.mp4",
  "mode": "frame",
  "frame_numbers": [0, 120, 240]
}

Read technical video metadata:

{
  "video_path": "/absolute/path/to/video.mp4"
}