Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers

[qti-horodnic]

Summary

This PR adds the strategy interfaces, and stage wrappers for the GenAI pipeline. This is the second of 7 PRs establishing the framework skeleton. It builds on PR1 (data model + engine routing) and introduces the Strategy pattern ABCs, concrete stage delegation wrappers and Executorch strategy stubs. No existing files are modified (except __init__.py introduced in PR1 for imports).

What's included

Strategy ABCs (one class per file):

• ModelPreparationStrategy — in strategies/model_preparation/
• QuantizationStrategy — in strategies/quantization/
• CompilationStrategy — in strategies/compilation/
• InferenceStrategy — in strategies/inference/

Each ABC defines invoke(context, input_config) -> output_config.

Strategy stubs (raise NotImplementedError):

• ExecuTorchModelPreparationStrategy
• ExecuTorchQuantizationStrategy
• ExecuTorchCompilationStrategy
• ExecuTorchInferenceStrategy

Stage wrappers:

• ModelPreparationStage, QuantizationStage, CompilationStage, InferenceStage.
• Each delegates to its injected strategy via invoke().
• name property returns stage constant from pipeline_types.py.

Pipeline stage ABC:

• PipelineStage — abstract base with name property and invoke() method.

Unit tests (16 tests across 6 test files):

• Strategy ABC enforcement (cannot instantiate, must implement invoke()).
• Strategy stub tests (NotImplementedError raised).
• Stage delegation tests (assert_called_once_with).
• Stage name tests (use constants).

PR Review Checklist

• All new classes follow single responsibility (one class per file) - Yes.
• All dependencies are injected via constructor with sensible defaults - Yes.
• All external calls are behind injectable interfaces - N/A, no external calls.
• Unit tests cover every public method - Yes.
• No existing files are modified (Phase 1 constraint) - Yes (only __init__.py updated to add PipelineStage export).
• Docstrings on all public classes and methods - Yes.
• Type annotations on all function signatures - Yes.
• Logging follows the strategy in the LLD - N/A, no logging in this PR.

Related PRs

• PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
• PR 2: Strategy interfaces & stage wrappers: this PR.
• PR 3: Pipeline orchestrator: pending.
• PR 4: Adapter interfaces & default implementations: pending.
• PR 5: Model preparation & quantization strategies: pending
• PR 6: Compilation & inference strategy implementations: pending.
• PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
  backends/qualcomm/genai_pipeline/tests/strategies/ \
  backends/qualcomm/genai_pipeline/tests/stages/ \
  -v

All 16 unit-tests passed.

Test Coverage

Command to run:

coverage run --rcfile=backends/qualcomm/.coveragerc \
  -m pytest \
    backends/qualcomm/genai_pipeline/tests/strategies/ \
    backends/qualcomm/genai_pipeline/tests/stages/ \
  -v

coverage report --rcfile=backends/qualcomm/.coveragerc \
  --include="backends/qualcomm/genai_pipeline/pipeline_stage.py,backends/qualcomm/genai_pipeline/strategies/*,backends/qualcomm/genai_pipeline/stages/*"

Result:

Name                                                                                           Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/stages/compilation_stage.py                                      14      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py                                        14      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py                                14      3      0      0    79%   24, 28, 35
backends/qualcomm/genai_pipeline/stages/quantization_stage.py                                     14      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py                    7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py         7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py             7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py                        7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py        7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py       7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py                  7      0      0      0   100%
------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                            110     3      0      0    97%

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20795

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Unrelated MacOS MPS test jobs time out (30 min) due to stale cpp_extension lock file poisoning self-hosted runners

✅ You can merge normally! (1 Unrelated Failure)

As of commit b666b79 with merge base 477d278 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / android / build-android (gh) (trunk failure)
  ImportError: /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/executorch/extension/training/pybindings/_training_lib.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c104impl3cow23materialize_cow_storageERNS_11StorageImplE

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[qti-horodnic]

@pytorchbot label "release notes: qualcomm"

[psiddh]

Great to see a clean framework. Is the plan for this new pipeline to eventually replace / wrap / coexist with the existing oss_scripts/llama stack (which already has QuantizationStrategy, ModelInference, etc.)? Also, the genai_pipeline tests don't seem to be wired into CI (not in pytest.ini or the QNN test jobs). is that intentional for now, or should they be added so the suite actually runs?

[psiddh]

self-contained skeleton with zero consumers atm, lgtm

[qti-horodnic]

Great to see a clean framework. Is the plan for this new pipeline to eventually replace / wrap / coexist with the existing oss_scripts/llama stack (which already has QuantizationStrategy, ModelInference, etc.)? Also, the genai_pipeline tests don't seem to be wired into CI (not in pytest.ini or the QNN test jobs). is that intentional for now, or should they be added so the suite actually runs?

@psiddh

Yes, that's the plan:

Phase 1 (PRs 1-7, what's listed in the decription): The framework skeleton coexists alongside the existing oss_scripts/llama stack, no modifications to existing code.

Phase 2 (3 PRs, ready locally): The framework wraps the existing code. The existing TextDecoder.quantize(), HybridTextDecoder.compile(), MultiModalManager, etc. are called from within the framework's adapters, not duplicated. Both entry points work: wherein users can still run python llama.py directly or use python -m backends.qualcomm.genai_pipeline --model llama3_2-1b.

Phase 3: Once the framework is validated to produce identical outputs, the old llama.py becomes a thin wrapper (or is deprecated), and new models are onboarded exclusively via the pipeline.

Regarding the CI integration, that's intentional for now. The tests require only pytest + unittest.mock, so they can run in any CI environment. They're not wired into pytest.ini or the QNN test jobs yet because:

The plan is to add a dedicated CI job for genai_pipeline unit tests once the PR stack is merged

[qti-horodnic]

Hey @psiddh, can we merge this pr if I've answered all your questions above? I'd like to raise the next PR in the series