Arm backend: Fix int8 TABLE domain for sigmoid LUTs

[xingguo01]

• Build 8-bit TOSA TABLE inputs from the canonical int8 code range [-128, 127] instead of using integer linspace.

• This avoids the duplicated zero and off-by-one LUT shift seen when qmin=-127 and keeps quantized sigmoid TABLE values aligned with the PT2E q/dq eager reference.

• Add pass-level regression tests for the full int8 domain and the reported qmin=-127 sigmoid quantization case.

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18973

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Rolling out OSDC (ARC) runners on pull & trunk workflows in PyTorch main

❌ 1 New Failure, 3 Unrelated Failures

As of commit b84fb93 with merge base e638059 ():

NEW FAILURE - The following job has failed:

• pull / unittest / macos / macos-job (gh)
  export/tests/test_target_recipes.py::TestTargetRecipes::test_mv3_model

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / windows / windows-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• pull / unittest-editable / windows / windows-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• trunk / unittest-release / windows / windows-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Copilot]

Pull request overview

Fixes the int8 TOSA TABLE domain generation for sigmoid LUTs by constructing the LUT input domain from the canonical int8 code range [-128, 127] (instead of using an int8 linspace), and adds regression tests to prevent off-by-one / duplicated-zero LUT issues (notably for qmin=-127).

Changes:

• Add a canonical int8 TABLE domain helper (_get_8bit_table_domain) and use it for 8-bit LUT generation.
• Replace torch.linspace(..., steps=256, dtype=torch.int8) with an explicit full int8 domain torch.arange(-128, 128, dtype=torch.int8).
• Add pass-level regression tests covering the full int8 domain and the reported qmin=-127 sigmoid quantization case.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
backends/arm/_passes/insert_table_ops.py	Switches 8-bit TABLE input domain generation to the canonical full int8 range and uses it when generating LUT values.
backends/arm/test/passes/test_insert_table_ops_pass.py	Adds regression tests validating the full int8 TABLE domain and sigmoid LUT correctness for qmin=-127.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[zingo]

I spot the problem below in the tests, I try a rerun to see if it's just a random unrelated error.

FAILED backends/arm/test/models/test_conformer.py::test_conformer_tosa_INT - AssertionError: Output 0 does not match reference output.
Given atol: 0.42308734320104124, rtol: 0.4.
Output tensor shape: torch.Size([10, 95, 16]), dtype: torch.float32
Difference: max: 0.507921576499939, abs: 0.7387949824333191, mean abs error: 0.019569561319964887.
-- Model vs. Reference --
Numel: 15200, 15200
Median: 0.0, 0.0
Mean: 0.0007245174780684083, 0.000347828720573728
Max: 2.8166558742523193, 2.8166558742523193
Min: -3.0706167221069336, -3.0706167221069336

[xingguo01]

@zingo thanks for checking, should have trigger-extended. Anyway, it looks the threshold is now too tight for conformer. This failure lines up directly with the change in insert_table_ops.py (line 142): the INT8 TABLE LUT is now built over the canonical [-128, 127] code domain instead of linspace(qmin, qmax, 256). That fixes the reported qmin = -127 sigmoid mapping bug, but test_conformer_tosa_INT in test_conformer.py (line 68) is a model-level accumulation test with already-loose tolerances (atol = rtol = 0.4 at test_conformer.py (line 39)). The reported numbers look like drift, not a hard correctness break:

max diff 0.5079 vs allowed 0.4231
mean abs error only 0.0196
output min/max unchanged
So the most likely situation is: the sigmoid fix is correct, but the conformer tolerance is now slightly too tight for the new LUT behavior after many repeated sigmoid/table applications.

Lowest-risk next step:

Keep the LUT fix.
Bump the conformer INT tolerance slightly in test_conformer.py (line 39), for example atol = 0.5 or 0.55, and rerun that test. I can submit a chained fix to update the threshold.

[zingo]

Interesting, yeah we seem to be very "close" I did a new re-run to get more numbers :) Lets look at it more next week at work, I hope you have a really good weekend. :)