Add `swe_rebench_openhands` public dataset loader for agentic multi-turn benchmarking

[Lokiiiiii]

Add a public dataset loader for nebius/SWE-rebench-openhands-trajectories — a 67K-trajectory dataset of multi-turn coding agent sessions with structured tool calls. This enables AgentPerf-style benchmarking (sustained concurrent load with realistic agentic workloads) without requiring users to author or convert dataset files manually.

Motivation

Current aiperf public datasets are either single-turn (sharegpt, instruct_coder, speed_bench_*, aimo) or non-coding (librispeech, mmstar). There is no public dataset that provides:

• Multi-turn agentic coding trajectories (context grows across turns)
• Structured function calling (tool_calls in OpenAI format)
• Realistic ISL distribution (up to 131K tokens, avg ~27K at deepest turn)
• Variable OSL (short tool calls interspersed with long reasoning/code generation)
• Large trajectory pool (67K sessions) for high-concurrency benchmarking

These properties are required for benchmarks like AA-AgentPerf, which measures max concurrent users under SLO constraints using real agentic workloads.

Dataset Details

Property	Value
HuggingFace	nebius/SWE-rebench-openhands-trajectories
Size	67,074 trajectories (2.08 GB parquet)
Avg turns per session	64.3
Max turns per session	100
Agent framework	OpenHands v0.54.0
Model used	Qwen3-Coder-480B-A35B-Instruct
Function calling	Yes (structured tool_calls with arguments JSON)
Repositories covered	1,823 (real-world GitHub issues)
License	CC-BY-4.0

Proposed Usage

# Basic multi-turn agentic replay
aiperf profile \
    --model deepseek-v3 \
    --endpoint-type chat \
    --streaming \
    --url http://localhost:8000 \
    --public-dataset swe_rebench_openhands \
    --concurrency 10 \
    --benchmark-duration 120

# AgentPerf-style capacity test (max users under SLO)
aiperf profile \
    --model deepseek-v3 \
    --endpoint-type chat \
    --streaming \
    --url http://localhost:8000 \
    --public-dataset swe_rebench_openhands \
    --search-recipe max-concurrency-under-sla \
    --search-sla 'time_to_first_token:p95:lt:15000' \
    --search-sla 'inter_token_latency:p75:lt:33.3' \
    --concurrency-min 1 \
    --concurrency-max 200

Design Considerations

Turn Splitting

The trajectory field is a flat list of messages. The loader needs to split this into "turns" for AIPerf's multi-turn replay. A turn boundary occurs at each assistant response — everything between two assistant messages constitutes one exchange that AIPerf dispatches as a single request.

The split logic:

1. System message → becomes the session's system prompt (turn 0 context)
2. Each assistant message (with or without tool_calls) → marks the end of a turn
3. Following tool responses + next user message → become the next turn's input messages
4. AIPerf accumulates context across turns (prior assistant responses become part of subsequent request history)

Filtering Options

The dataset has metadata fields useful for filtering:

• resolved — filter to only successful trajectories (more realistic workload — agents that solved issues tend to have coherent multi-turn patterns)
• ISL-based filtering — the ThoughtWorks derivative shows p50=23K, p95=69K context; users may want to filter sessions by max context depth to match their model's context window