Cortex-M backend: dispatch quantized_linear AOT layout on target ISA

[rascani]

Summary

CMSIS-NN's arm_fully_connected_s8 has three runtime paths, gated by compile-time ARM_MATH_MVEI / ARM_MATH_DSP. They split the bias and input_offset×sum(weight) offset term between two inputs, in incompatible conventions:

• MVE: reads ctx.buf as a precomputed kernel_sum that must already include input_offset × sum(weight) and the bias contribution. The bias argument is (void)bias; — ignored.
• DSP / scalar: read the bias argument directly and fold the input_offset contribution at runtime. ctx.buf (kernel_sum) is (void)kernel_sum; — ignored.

ConvertToCortexMPass._get_linear_replacement previously emitted only the MVE shape (kernel_sum populated, bias=None). On any non-MVE build the DSP/scalar path started the int32 accumulator at 0 instead of at bias + input_offset × sum(weight), dropping both the bias and the offset contribution. The accumulator wound up much smaller than intended, requantization collapsed it to the output zero point, and every classifier with a deep, narrow tail produced essentially uniform near-zero outputs on non-MVE Cortex-M builds.

Use the target-ISA plumbing added by the CortexMTargetConfig PR (#19470) to dispatch the right input shape at AOT time: on MVE targets emit kernel_sum with bias folded in (bias=None); on DSP and scalar targets emit the raw int32 bias directly (kernel_sum=None). The CMSIS-NN runtime then matches exactly what it expects.

Update quantized_linear_impl in operators.py to mirror the same contract: dispatch off whichever of kernel_sum / bias is non-None. Threading happens automatically via CortexMPassManager's signature injection of target_config into the pass's __init__.

Test Plan

Add backends/cortex_m/test/misc/test_quantized_linear_small_magnitude.py as a regression. A tiny nn.Linear(512, 10) on uniform[0, 0.002] input is the minimal reproducer for the small-magnitude regime where the missing offset terms dominate. The dialect test parametrizes over MVE/DSP/scalar target configs; the implementation test runs against whatever path the runner build matches.

The DSP & Scalar tests will need #19520 for CI testing.

Authored with Claude.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19676

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 2 Active SEVs

There are 2 currently active SEVs. If your PR is affected, please view them below:

• Run pull request jobs on OSDC runners in shadow mode
• Rollout OSDC prod cluster in us-west-1

❌ 8 New Failures

As of commit 776a3a1 with merge base 1c9c115 ():

NEW FAILURES - The following jobs have failed:

• Lint / link-check / lint-urls (gh)
  Process completed with exit code 1.
• Lint / lintrunner (gh)
  >>> Lint for backends/cortex_m/test/misc/test_quantized_linear_small_magnitude.py:
• Lint / lintrunner-mypy (gh)
• pull / test-qnn-delegate-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 633166802aef3ae0e4aac13f5f45344a4880c96a8832b34bfb3a51a65d0ea8ed /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-deps (test_pytest_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t bacac523e178538ccca3f20f7439641e2cc56b09d804e4391fc1aa4f0b025920 /exec failed with exit code 127
• pull / unittest-arm-backend-with-no-deps (test_pytest_ops_no_target) / linux-job (gh)
  RuntimeError: Command docker exec -t a3440b2fdabc79ff673cdbdd6f8875860f53c12383addcaa519aa5ee64655ea8 /exec failed with exit code 127
• pull / unittest-arm-backend-with-no-deps (test_pytest_ops_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 25cf5cfe09745f21a00c7ce2feb3c480f729c08f9544f0f6ea9e360f03f7cc99 /exec failed with exit code 127
• pull / unittest-arm-backend-with-no-deps (test_run_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t a79cd77371fb46cc03c3baca71512cd8cb957093fbfa70a6dab1d7ce2e3c4c58 /exec failed with exit code 127

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.