Deselect VGF tests when model-converter is not installed (#19282)

[psiddh]

Summary:

VGF tests require the Arm MLSDK model-converter binary which is not provisioned in CI when _ENABLE_VGF = False (the current default in targets.bzl). These tests correctly skip via SkipIfNoModelConverter, but TestInfra reports skipped/disabled tests as FAILURE in the OMH dashboard, creating ~1,025 phantom failures for the ai_infra_mobile_platform oncall.

This change uses the existing pytest_collection_modifyitems hook in conftest.py to deselect (not collect) tests that carry the SkipIfNoModelConverter marker (detected via reason string on skipif markers). This is more precise than name-based filtering because:

• It catches parametrized variants like test_roundtrip[VGF] that use marks=SkipIfNoModelConverter
• It does NOT deselect tests like test_compile_spec_vgf_no_quant that have "vgf" in the name but do not need model-converter

Deselected tests are invisible to TestInfra, eliminating false failure reports. When _ENABLE_VGF is later flipped to True and model-converter is provisioned, VGF tests will automatically be collected and run again.

Differential Revision: D103644035

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19282

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Ubuntu services are down

❌ 2 New Failures, 6 Unrelated Failures

As of commit 17f473b with merge base 0a113f8 ():

NEW FAILURES - The following jobs have failed:

• pull / test-lora-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 870ba747497ea7486d6e470ed734ded2ce9a2d3b8ffdeb7ee97c64f18ee2e99d /exec failed with exit code 1
• pull / test-lora-multimethod-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 98873880d59cfcdc1789bbe4a70f6db4cb966abe61a6608996e382252fd4c2b8 /exec failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / linux / linux-job (gh) (trunk failure)
  examples/models/llama/tests/test_static_attention.py::StaticAttentionTest::test_within_transformer
• pull / unittest / macos / macos-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• pull / unittest-editable / linux / linux-job (gh) (trunk failure)
  examples/models/llama/tests/test_static_attention.py::StaticAttentionTest::test_within_transformer
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• trunk / unittest-release / linux / linux-job (gh) (trunk failure)
  examples/models/llama/tests/test_static_attention.py::StaticAttentionTest::test_within_transformer
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@psiddh has exported this pull request. If you are a Meta employee, you can view the originating Diff in D103644035.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[Copilot]

Pull request overview

This PR is mainly aimed at reducing false CI noise in the Arm backend by changing how VGF tests are handled when the Arm MLSDK model-converter binary is unavailable. It also includes two unrelated test-only changes outside the Arm VGF collection logic.

Changes:

• Add Arm pytest_collection_modifyitems logic to deselect VGF tests when model-converter is not installed.
• Broaden an EXIR partitioner test's expected AttributeError regex for cross-version message compatibility.
• Add a new skip to the Vulkan Hardshrink delegate test.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
backends/arm/test/conftest.py	Adds collection-time deselection of VGF tests based on model_converter_installed().
exir/backend/test/test_partitioner.py	Relaxes a test assertion to accept multiple setter error message variants.
backends/vulkan/test/test_vulkan_delegate.py	Unconditionally skips the Vulkan Hardshrink backend test.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.