Skip to content

Orphan fixtures and minor scoring smells #3

Description

@rendrag-git

Cleanup grab-bag

Three small items found during the M3 inventory audit. None affect runtime correctness; all are convention drift / dead code.

1. `fixtures/needle_repo_8k` is unreferenced by any tier manifest

Only `manifests/openclaw-agent-core.json` (the legacy non-tier suite) uses it. Tier-medium runs needle at 16k and 32k; tier-small runs needle at 4k. There is no 8k slot. Either:

  • Add an 8k task to tier-small as the smaller stretch case, or
  • Delete the fixture as dead seed data.

2. `fixtures/real_repos/kingshot-ams-snapshot` is unreferenced by any tier manifest

Only `manifests/real-repo-readonly.example.json` (a template) uses it. GOAL.md M6 calls for "real-repo coverage beyond one snapshot" — this is M6 territory and tracked there. Keeping the fixture for now, but flag it once that milestone activates.

3. `_failure_type` has a duplicate `"preserved file"` branch in scoring

`openclaw_bench/scoring.py` `_failure_type` (around line 651) checks `"preserved file"` substring in two branches. The first match always wins, so the action_gate-specific second branch is unreachable. Same outcome (`instruction_violation`) either way, so no scoring impact, but the dead branch should be collapsed.

4. AGENTS.md "intent" comment convention not fully covered

AGENTS.md says every scoring branch should have a one-line "this catches X failure mode" comment. The branches at `scoring.py:40–46` (hallucinated, timeout, error) and the catch-all `else` at L112 lack the comment. Three lines of explanatory text would close the gap.

Severity

P3 — none of these affect any runtime behavior. Nice-to-haves before the next major refactor of scoring.py.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions