Update run_id: remove "eval_" prefix

[simonrosenberg]

run_id had been modified previously to be directly equal to the github action run id without the eval_ prefix.
Here there are fallbacks / bits of code that use the previous version of run_id.

[juanmichelini]

LGTM

[simonrosenberg]

@OpenHands please resolve conflicts

[openhands-ai]

I'm on it! simonrosenberg can track my progress at all-hands.dev

[openhands-ai]

I've successfully resolved the merge conflicts in PR #389 and pushed the changes.

Summary of Changes

Merged main into fix-run-id branch and resolved conflicts in two files:

1. benchmarks/swebenchmultimodal/eval_infer.py:

   • Changed run_id default from f"eval_{predictions_path.stem}" to predictions_path.stem (removing the "eval_" prefix)

2. benchmarks/swtbench/eval_infer.py:

   • Changed run_id from f"eval_{predictions_path.stem}" to predictions_path.stem in the command arguments
   • Changed run_id from f"eval_{output_file.stem}" to output_file.stem when constructing the report file path

These changes are consistent with the PR's purpose of removing the "eval_" prefix from run_id values throughout the codebase. The changes have been pushed to the fix-run-id branch.

View full conversation

[openhands-ai]

Looks like there are a few issues preventing this PR from being merged!

• GitHub Actions are failing:
  • Pre-commit checks
• There are merge conflicts

If you'd like me to help, just leave a comment, like

@OpenHands please fix the merge conflicts on PR #389 at branch `fix-run-id`

or

@OpenHands please fix the failing actions on PR #389 at branch `fix-run-id`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

[simonrosenberg]

@OpenHands fix precommit hooks
Also make sure the PR has a clean diff: it should NOT how vendor changes. The diif should ONLY show removal of eval_

[openhands-ai]

I'm on it! simonrosenberg can track my progress at all-hands.dev

[openhands-ai]

I've successfully fixed the PR #389 by:

Checklist

• ✅ Fixed precommit hooks: All precommit checks (Ruff format, Ruff lint, PEP8 style check, Pyright type check) now pass
• ✅ Cleaned up the diff: Removed vendor changes (uv.lock and vendor/software-agent-sdk) that were accidentally included
• ✅ PR now only shows removal of eval_ prefix: The diff is now clean and only contains the intended changes

Summary of Changes

The PR diff now only shows:

1. benchmarks/swebenchmultimodal/eval_infer.py: Changed run_id default from f"eval_{predictions_path.stem}" to predictions_path.stem
2. benchmarks/swtbench/eval_infer.py: Changed run_id from f"eval_{predictions_path.stem}" to predictions_path.stem in two places

The changes have been pushed to the fix-run-id branch.

View full conversation