Add Stirrup agent + GDPVal eval/RL environment

[Kh4L]

Summary

Adds a Stirrup-based agent + a GDPVal benchmark built on the NeMo-Gym
benchmark convention (ng_prepare_benchmark + ng_e2e_collect_rollouts),
validated on the full 220-task GDPVal set in both rubric and comparison
scoring modes.

Architecture

Split into three pieces, matching NeMo-Gym's server-type convention:

Benchmark — benchmarks/gdpval/

• prepare.py downloads openai/gdpval from HF → data/gdpval_benchmark.jsonl
• config.yaml wires gdpval_judge_model + gdpval_resources_server +
  gdpval_stirrup_agent
• Entry point: ng_e2e_collect_rollouts +config_paths=[benchmarks/gdpval/config.yaml]

Resources server — resources_servers/gdpval/

• Owns verify() and aggregate_metrics() with two modes via reward_mode:
  • rubric (default) — LLM-judge per-criterion score, reward in [0, 1]
  • comparison — pairwise vs reference_deliverables_dir, reward in
    {0, 0.5, 1}; aggregate_metrics reduces W/L/T → ELO anchored at
    reference_elo (default 1000)
• All scoring, pairwise comparison, Office→PDF preconvert live here.
  Multimodal judge path used whenever content blocks are available.

Agent — responses_api_agents/stirrup_agent/

• StirrupAgentWrapper is task-agnostic; task-specific logic in a
  TaskStrategy subclass (GDPValTask)
• /run executes the agent, persists deliverables, POSTs to the resources
  server's /verify, returns the response. Agent is scoring-free.
• aggregate_metrics proxies to resources server so ELO extras flow through;
  /verify errors caught per-rollout so a single failure can't crash a run
• Optional: Apptainer-backed code_exec, Tavily web-search

Dependency: stirrup>=0.1.7 (Apache 2.0) declared as an extra of the
stirrup_agent server, not of core.

Validation (Ultra V3 SFT iter16k, full 220-task GDPVal, num_repeats=2)

Rubric mode (n=440):

• mean/reward = 0.755, pass@1 = 0.755, pass@2 = 0.821
• 56% of rollouts score ≥ 0.8
• Pre-refactor port-v4 baseline: 0.24 → 3.1× lift (dominant contributor:
  always-visual judge when content blocks are available)

Comparison mode vs fork baseline (4 trials per pairing, n=440):

• W/L/T = 147 / 208 / 77
• win_rate = 0.429
• eval_elo = 950.6 (vs fork=1000; port-v3 historical=917 → +34 ELO)

Running

ng_prepare_benchmark '+config_paths=[benchmarks/gdpval/config.yaml]'
ng_e2e_collect_rollouts \
  '+config_paths=[benchmarks/gdpval/config.yaml,responses_api_models/vllm_model/configs/vllm_model.yaml]' \
  '++split=benchmark' \
  '++output_jsonl_fpath=results/gdpval.jsonl' \
  "++gdpval_stirrup_agent.responses_api_agents.stirrup_agent.persist_deliverables_dir=$PWD/output/gdpval" \
  # ... policy_* overrides as usual

# Add for comparison mode:
  '++gdpval_resources_server.resources_servers.gdpval.reward_mode=comparison' \
  "++gdpval_resources_server.resources_servers.gdpval.reference_deliverables_dir=/path/to/reference"

Test plan

• pytest resources_servers/gdpval/tests -x — rubric + comparison unit tests
• 10-task rubric smoke (mean/reward 0.719)
• Full 220-task rubric (mean/reward 0.755)
• 10-task comparison smoke (eval_elo ~1017 on small sample)
• Full 220-task comparison (eval_elo 950.6)

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[cmunley1]

should these scripts go into responses_api_agents/stirrup_agent/scripts? same for container

[Kh4L]

You're right, that would be better. just pushed