Skip to content

EP: Implement compile validation and e2e#661

Merged
zhenchaoni merged 2 commits into
mainfrom
private/zhenni/ep_test_e2e
May 19, 2026
Merged

EP: Implement compile validation and e2e#661
zhenchaoni merged 2 commits into
mainfrom
private/zhenni/ep_test_e2e

Conversation

@zhenchaoni
Copy link
Copy Markdown
Member

Our supported device for each ep is:

  • qnn: npu and gpu
  • openvino: npu, gpu and cpu
  • vitisai: npu
  • nv_tensorrt_rtx: gpu

This PR

  1. Add "good input" e2e test cases, it permutates the above support matrix to generate --device and --ep combinations for e2e test.
  2. Add a validation logic. If the user provides --device and --ep which doesn't match the above matrix, or the user provides ep that is not applicable to his machine, a validation error will be raised.

@zhenchaoni zhenchaoni requested a review from a team as a code owner May 19, 2026 04:03
@DingmaomaoBJTU
Copy link
Copy Markdown
Collaborator

Code review

Found 1 issue:

  1. _BAD_INPUT_PARAMS row 3 uses the wrong EP guard — require_ep("openvino") only guarantees OpenVINO is present on the host, but the test asserts that QNN is absent (expected error: "is not available on this host"). On a machine with both OpenVINO and QNN installed, the test will run (OpenVINO guard passes), the compile with --ep qnn will succeed (QNN is available), and the assertion exit_code != 0 will fail.

# 3. EP not registered on this host.
(None, "qnn", "openvino", "is not available on this host"),
# 4. No available EP on this host supports the requested device.
("cpu", None, "qnn", "does not support EPContext compilation"),
]

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@zhenchaoni zhenchaoni force-pushed the private/zhenni/ep_test_e2e branch from dceb350 to cce8f22 Compare May 19, 2026 05:45
Comment thread tests/e2e/test_compile_e2e.py Outdated
Comment thread tests/e2e/test_compile_e2e.py
Comment thread tests/e2e/test_compile_e2e.py
Comment thread tests/e2e/test_compile_e2e.py Outdated
Comment thread tests/e2e/test_compile_e2e.py Outdated
@zhenchaoni zhenchaoni force-pushed the private/zhenni/ep_test_e2e branch from cce8f22 to af1c714 Compare May 19, 2026 07:08
@zhenchaoni zhenchaoni force-pushed the private/zhenni/ep_test_e2e branch from af1c714 to 6f42174 Compare May 19, 2026 07:15
@zhenchaoni zhenchaoni force-pushed the private/zhenni/ep_test_e2e branch from dc11285 to a5218a8 Compare May 19, 2026 08:01
@zhenchaoni zhenchaoni merged commit b181354 into main May 19, 2026
9 checks passed
@zhenchaoni zhenchaoni deleted the private/zhenni/ep_test_e2e branch May 19, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants