Identify functional scenarios and implement E2E tests for `winml perf`

[zhenchaoni]

Description

Identify the functionality list for winml perf and implement E2E pytest cases covering it. Tests live in tests/e2e.

Coverage must include:

• Happy path — each subcommand / mode runs successfully with typical inputs.
• Bad path — missing required args, invalid values, mutually exclusive flags, unsupported model, missing optional dep, missing EP. Each must produce the documented error and exit code, not a stack trace.
• Flag / option variations — each behavior-bearing flag exercised both present and absent; each value of every enum/choice flag (--preset, --precision, --ep, …) covered by at least one test.

Deliverables

1. E2E pytest test cases merged under tests/e2e.
2. (Optional) A markdown file listing all test scenarios.

Acceptance Criteria

• Happy path, bad path, and flag-variation coverage all present.
• uv run pytest tests/e2e/ -k perf is green.

[DingmaomaoBJTU]

Triage summary

• Component: Perf
• Priority: P2

Track E2E functional test scenarios for winml perf. Identify edge cases and implement pytest coverage.

[xieofxie]

manually selected commands

Device / ep

winml perf -m microsoft/resnet-50 --ep cpu --device cpu/gpu/npu --iterations 500 --monitor

winml perf -m model.onnx --ep cpu --device cpu/gpu/npu --iterations 500 --monitor

Module related

winml perf -m microsoft/resnet-50 --module ResNetStage/ ResNetConvLayer/ ResNetBottleNeckLayer/ResNetBasicLayer/ResNetEmbeddings --ep cpu --device cpu --iterations 5 --warmup 2

[DingmaomaoBJTU]

Triage summary

• Component: Perf
• Priority: P2

E2E test coverage for winml perf needs to be identified and implemented. Next step: enumerate functional scenarios and implement pytest-based E2E tests.