fix(sanity): parse OpenClaw trajectory format correctly

[ScuttleBot]

Problem

The sanity check grader was looking for type='message' with message.role='assistant', but OpenClaw trajectory files actually use type='model.completed' with data.assistantTexts[].

This caused ALL models to fail the sanity check with 0%, triggering fail-fast and aborting benchmark runs. We had 42 Vultr instances sitting idle because they all hit this.

Root Cause

Schema mismatch between what the grader expected and what OpenClaw actually produces:

Grader expected:

{"type": "message", "message": {"role": "assistant", "content": [...]}}

OpenClaw actually produces:

{"type": "model.completed", "data": {"assistantTexts": ["Hello, I'm ready!..."]}}

Fix

Updated the grader to check:

1. type='model.completed' with data.assistantTexts
2. type='trace.artifacts' with data.assistantTexts (final summary)

Testing

Verified against actual transcript from a GPT-5.4 run that was incorrectly failing — the model responded correctly but the grader couldn't find the response due to the schema mismatch.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

The fix is clean and correctly addresses the schema mismatch. The new checks properly handle both model.completed and trace.artifacts entry types from the OpenClaw trajectory format, and the any(t.strip() for t in texts) guard is a nice touch to avoid false positives from whitespace-only responses.

The new commit only bumps BENCHMARK_VERSION to rc4 — no logic changes.

Files Reviewed (2 files)

• tasks/task_sanity.md
• BENCHMARK_VERSION

---

_{Reviewed by claude-sonnet-4.6 · 110,925 tokens}