[Python SDK] OpenRouterProvider: add video generation via /api/v1/videos

[santoshkumarradha]

Summary

Upgrade OpenRouterProvider in media_providers.py to support video generation using OpenRouter's async video API (POST /api/v1/videos → poll → download).

Context

OpenRouter launched a dedicated video generation API in March 2026. It's asynchronous: you submit a job, poll for status, then download the result. This is a different pattern from the synchronous chat completions API used for images.

LiteLLM does NOT route to OpenRouter's video endpoint — LiteLLM's video support covers OpenAI/Azure/Gemini/Vertex/RunwayML only. So this must be direct HTTP.

Available Models (via OpenRouter)

Model	Price	Features
google/veo-3.1	$0.40-0.50/sec	1080p, native audio, scene extension, 4K upscale
openai/sora-2-pro	$0.30/sec	Physics-accurate, multi-shot, C2PA provenance
alibaba/wan-2.6	$0.04/sec	1080p, text/image/reference-to-video, lip-sync
alibaba/wan-2.7	~$0.04/sec	First+last frame control, reference-to-video
bytedance/seedance-2.0	varies	Text/image/reference-to-video
bytedance/seedance-1.5-pro	varies	4.5B params, multi-lang lip-sync, camera control

API Reference

Submit: POST /api/v1/videos

{
  "model": "google/veo-3.1",
  "prompt": "A golden retriever on a beach",
  "duration": 8,
  "resolution": "1080p",
  "aspect_ratio": "16:9",
  "generate_audio": true,
  "seed": 42,
  "frame_images": [
    {"type": "image_url", "image_url": {"url": "..."}, "frame_type": "first_frame"}
  ],
  "input_references": [
    {"type": "image_url", "image_url": {"url": "..."}}
  ],
  "provider": {"options": {"google-vertex": {"parameters": {"negativePrompt": "blurry"}}}}
}

Response (202): {"id": "abc123", "polling_url": "https://openrouter.ai/api/v1/videos/abc123", "status": "pending"}

Poll: GET /api/v1/videos/{jobId}

Status transitions: pending → in_progress → completed | failed

On completion: {"unsigned_urls": ["https://openrouter.ai/api/v1/videos/abc123/content?index=0"], "usage": {"cost": 0.25}}

Download: GET /api/v1/videos/{jobId}/content?index=0 → binary MP4

Model discovery: GET /api/v1/videos/models

Implementation

Update OpenRouterProvider.generate_video() in media_providers.py:

async def generate_video(
    self,
    prompt: str,
    model: Optional[str] = None,
    image_url: Optional[str] = None,
    duration: Optional[int] = None,
    resolution: Optional[str] = None,
    aspect_ratio: Optional[str] = None,
    generate_audio: Optional[bool] = None,
    seed: Optional[int] = None,
    frame_images: Optional[List[Dict]] = None,
    input_references: Optional[List[Dict]] = None,
    poll_interval: float = 30.0,
    timeout: float = 600.0,
    **kwargs,
) -> MultimodalResponse:

Key implementation details:

1. Strip openrouter/ prefix before sending to API (OpenRouter models are google/veo-3.1, not openrouter/google/veo-3.1)
2. Use aiohttp or httpx for async HTTP (check what's already available in the project deps)
3. Polling loop with configurable interval (default 30s) and timeout (default 10min)
4. Download video bytes on completion and return as MultimodalResponse with FileOutput in files[]
5. Error handling: Map HTTP 400/401/402/429/500 to clear error messages
6. API key: Use OPENROUTER_API_KEY env var or explicit api_key param

Update supported_modalities:

@property
def supported_modalities(self) -> List[str]:
    return ["image", "video"]  # was ["image"]

Developer Experience

# From agent
result = await app.ai_generate_video(
    "A golden retriever playing fetch on a beach",
    model="openrouter/google/veo-3.1",
    resolution="1080p",
    aspect_ratio="16:9",
    duration=8,
    generate_audio=True,
)
result.files[0].save("dog_beach.mp4")

# Image-to-video
result = await app.ai_generate_video(
    "Camera slowly pans across the scene",
    model="openrouter/alibaba/wan-2.7",
    image_url="https://example.com/landscape.jpg",
    frame_images=[{"type": "image_url", "image_url": {"url": "https://..."}, "frame_type": "first_frame"}],
)

# Cheap testing
result = await app.ai_generate_video(
    "A cat playing with yarn",
    model="openrouter/alibaba/wan-2.6",  # $0.04/sec
)

Dependencies

• Depends on: [Python SDK] Add MediaRouter for prefix-based provider dispatch #463 (MediaRouter) — without router, this provider won't be dispatched to automatically
• Can be developed independently and tested via direct provider instantiation before [Python SDK] Add MediaRouter for prefix-based provider dispatch #463 lands

Files

File	Change
sdk/python/agentfield/media_providers.py	Upgrade OpenRouterProvider.generate_video(), update supported_modalities
sdk/python/agentfield/types.py	Add openrouter_api_key: Optional[str] to AIConfig, update video_model default

Acceptance Criteria

• OpenRouterProvider.generate_video() submits to /api/v1/videos, polls, downloads
• Handles all status transitions: pending → in_progress → completed/failed
• Returns MultimodalResponse with video as FileOutput in files[]
• model prefix openrouter/ is stripped before API call
• Timeout and poll_interval are configurable
• Error handling for 400/401/402/429/500 with clear messages
• Works with at least alibaba/wan-2.6 (cheapest for testing)
• pytest sdk/python/ passes
• ruff check sdk/python/ passes

Testing

# Manual integration test (requires OPENROUTER_API_KEY)
OPENROUTER_API_KEY=sk-... python -c "
import asyncio
from agentfield.media_providers import OpenRouterProvider

async def main():
    p = OpenRouterProvider(api_key='sk-...')
    result = await p.generate_video('A cat walking', model='alibaba/wan-2.6')
    print(f'Files: {len(result.files)}, URL: {result.files[0].url}')

asyncio.run(main())
"

Notes for Contributors

Severity: HIGH — This is the core new feature.

The async polling pattern is different from everything else in the SDK. Use asyncio.sleep() in the poll loop, not time.sleep(). Add a reasonable default timeout (600s) because video gen can take minutes.

The frame_images and input_references parameters should be passed through as-is to the API — don't try to transform them. Let the user follow OpenRouter's schema directly.

For the HTTP client, prefer httpx (async) if it's already a dependency, otherwise use aiohttp. Check pyproject.toml for what's available. Fallback to aiohttp if neither is present.

[santoshkumarradha]

Dependencies

Depends on: #463 (MediaRouter — for automatic routing via openrouter/ prefix)
Blocks: #465 (shares HTTP client setup), #467 (TS reference impl), #468 (Go reference impl), #470 (integration tests)
Consumed by: #469 (VideoOutput type — optional, enriches response)
Epic: #471

Can be developed standalone (test via direct OpenRouterProvider() instantiation) before #463 merges.

[github-actions]

Thanks for picking this up, @santoshkumarradha! 🎉

Here's what to expect:

• We'll check in after 7 days if we haven't heard an update
• After 14 days, we'll send a second reminder
• After 21 days without activity, we'll unassign to let others contribute

Tips for success:

• Drop a comment when you start working on it
• Share any blockers - we're here to help!
• Link your PR with Fixes #464 in the description

Happy coding! 🚀