Pass Dependencies When Proposing Partitions VIA New Group-Based Partitioner

[leafs1]

Summary

The existing capability-based partitioner in PyTorch's FX module lacks the ability to specify node dependencies that must be partitioned together. This can lead to incorrect partitioning of dynamically quantized linear patterns, resulting in the loss of quantization semantics. For example, in a graph with shared QDQ (Quantize-Dequantize) chains, the partitioner may incorrectly separate nodes that should remain together, leading to incorrect execution semantics.

This PR addresses this issue by adding a new group-based partitioner that allows users to specify groups of nodes that must be partitioned together.

Test plan

I've created test cases that replicate existing QDQ tests, as well as additional graphs with different node dependencies. These tests aim to verify that the new partitioner correctly groups nodes together based on the specified dependencies.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12072

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 6 Unrelated Failures

As of commit 7c82bb4 with merge base b74c68d ():

NEW FAILURES - The following jobs have failed:

• Lint / android-java-format / linux-job (gh)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
• Lint / lintrunner / linux-job (gh)
  >>> Lint for exir/backend/canonical_partitioners/group_partitioner.py:
• pull / unittest / macos / macos-job (gh)
  backends/xnnpack/test/ops/test_exp.py::TestExp::test_fp16_exp

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / test-custom-ops-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
• pull / test-llama-runner-linux-android / linux-job (gh) (matched linux rule in flaky-rules.json)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[leafs1]

@pytorchbot label "release notes: none"