Skip to content

run_eval: forward --device to winml build subprocess#674

Merged
vortex-captain merged 5 commits into
mainfrom
reny/run_eval_device
May 19, 2026
Merged

run_eval: forward --device to winml build subprocess#674
vortex-captain merged 5 commits into
mainfrom
reny/run_eval_device

Conversation

@vortex-captain
Copy link
Copy Markdown
Contributor

Summary

winml config, winml perf, and winml eval already receive --device from run_eval; winml build was the only step missing it. With this change, the build subprocess respects the caller's device choice instead of relying on its own auto-detection.

Companion to #667 (which forwarded --ep and added device-aware EP auto-select in the build CLI). With both PRs in, run_eval consistently propagates --device/--ep to every winml subprocess it spawns.

Diff

        build_args = [
            *WINML_CLI,
            "build",
            "-c",
            str(sub_cfg),
            "-m",
            entry.hf_id,
            "--use-cache",
+            \"--device\",
+            device,
        ]
        if ep:
            build_args += [\"--ep\", ep]

Verification

E2E on a cold cache:

```
python scripts/e2e_eval/run_eval.py --hf-model BAAI/bge-base-en-v1.5 --ep cpu --device cpu --eval-type perf
```

→ exit 0, `[PASS] 2.4s`. Generated build config has `"compile": null` (correct for cpu — no EP-specific compile needed).

`winml config`, `winml perf`, and `winml eval` already receive `--device`
from run_eval; `winml build` was the only step missing it. With this
change, the build subprocess respects the caller's device choice instead
of relying on its own auto-detection.

Verified e2e: `python scripts/e2e_eval/run_eval.py --hf-model
BAAI/bge-base-en-v1.5 --ep cpu --device cpu --eval-type perf` on a cold
cache -> exit 0, [PASS] 2.4s, build config emitted "compile": null
(correct for cpu).
@vortex-captain vortex-captain requested a review from a team as a code owner May 19, 2026 08:39
Comment thread scripts/e2e_eval/run_eval.py
@vortex-captain vortex-captain enabled auto-merge (squash) May 19, 2026 09:04
@vortex-captain vortex-captain merged commit 6883e08 into main May 19, 2026
9 checks passed
@vortex-captain vortex-captain deleted the reny/run_eval_device branch May 19, 2026 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants