feat: add AReaL AgentWorkflow wrapping WAADesktopEnv for RL training

[abrichr]

Summary

• Add WAADesktopWorkflow class implementing AReaL's agent workflow pattern for RL training of desktop automation agents
• Add AReaL GRPO config template (configs/areal_waa_grpo.yaml) for single-GPU training with Qwen2.5-VL-3B-Instruct
• Add 14 tests covering full episode execution, dense milestone rewards, edge cases, and message building

Details

AReaL (github.com/inclusionAI/AReaL) is an async RL training framework that supports agent workflows where your code talks to an OpenAI-compatible proxy, and AReaL transparently handles token tracking, logprobs, and gradient computation.

WAADesktopWorkflow.run() executes one episode:

1. Reset environment with task_id
2. Loop: screenshot -> LLM (via AReaL proxy) -> parse action JSON -> execute on VM
3. Evaluate with dense milestone rewards via evaluate_dense()
4. Return reward scalar (0.0 to 1.0)

Key design decisions:

• Reuses existing infrastructure: parse_action_json, RLEnvironment, TaskConfig, evaluate_dense()
• AReaL is an optional dependency -- module imports gracefully when AReaL is not installed
• Tests use mock adapters and mock AsyncOpenAI clients (no real VM or AReaL needed)
• Config follows AReaL's exact YAML format (verified against their examples/agent_workflow/config.yaml)

Test plan

• All 14 new tests pass (tests/test_areal_workflow.py)
• All 48 related existing tests pass (trl_rollout, dense_rewards, verl_env) -- no regressions
• Manual: verify config loads correctly with AReaL on GPU VM (requires AReaL installation)

🤖 Generated with Claude Code