Add torchvision deform_conv2d lowering

[SakshamKapoor2911]

Summary

• Add Torch frontend lowering for torchvision::deform_conv2d.
• Decompose deformable convolution into MIL resample calls plus a final 1x1 conv.
• Support mask/no-mask, bias/no-bias, groups, offset groups, 1x1 and non-square kernels, stride, padding, and dilation.
• Add focused Torch frontend tests for representative configurations.

Fixes #1889.

Testing

Local Linux conversion/structure tests:

PYMIL_TEST_TARGETS=mlprogram /home/skapoor/oss_work/.venvs/coremltools-dcn/bin/python -m pytest coremltools/converters/mil/frontend/torch/test/test_torch_ops.py::TestDeformConv2d -q --tb=short

Result:

14 passed

NeuralNetwork skip path:

PYMIL_TEST_TARGETS=neuralnetwork /home/skapoor/oss_work/.venvs/coremltools-dcn/bin/python -m pytest coremltools/converters/mil/frontend/torch/test/test_torch_ops.py::TestDeformConv2d -q --tb=short

Result:

7 skipped

Other local validation:

• git diff --check origin/main...HEAD passed.
• Direct TorchExport conversion to milinternal passed across the seven test configs.
• Independent PyTorch evaluator mirroring the converter's patch ordering matched torchvision.ops.deform_conv2d across 8 configs with max absolute diff 1.43e-6.
• fp16 conversion passed with an iOS16 deployment target after dtype promotion around the final conv and bias add.

Fork-only CI smoke:

• Full Linux + macOS smoke passed: https://github.com/SakshamKapoor2911/coremltools/actions/runs/28667757047
• macOS job installed native Core ML binaries from the coremltools==9.0 wheel, verified BlobWriter, and ran the full TestDeformConv2d class.
• The test matrix gives fp16 conversion/graph-structure coverage and fp32 native prediction parity. A prior fp32 native diagnostic also passed: https://github.com/SakshamKapoor2911/coremltools/actions/runs/28666789120

[SakshamKapoor2911]

Hi @TobyRoseman, you had asked about the deform_conv2d math/details in #1889. I opened this PR with a MIL resample + grouped 1x1 conv lowering and included local plus fork-only macOS native validation in the PR body.

When you have bandwidth, I’d appreciate your review. Maintainer edits are enabled.

[TobyRoseman]

Your unit tests don't pass:

Results (16.94s):
      14 passed
      14 failed
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (2, 3, 5, 5), 'out_channels': 5, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 1, 'offset_groups': 1, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHSCRIPT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (2, 3, 5, 5), 'out_channels': 5, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 1, 'offset_groups': 1, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHEXPORT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 3, 5, 5), 'out_channels': 4, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 1, 'offset_groups': 1, 'use_mask': False, 'use_bias': False}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHSCRIPT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 3, 5, 5), 'out_channels': 4, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 1, 'offset_groups': 1, 'use_mask': False, 'use_bias': False}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHEXPORT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 2, 4, 4), 'out_channels': 3, 'kernel_size': (1, 1), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 1, 'offset_groups': 1, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHSCRIPT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 2, 4, 4), 'out_channels': 3, 'kernel_size': (1, 1), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 1, 'offset_groups': 1, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHEXPORT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 4, 5, 5), 'out_channels': 8, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 2, 'offset_groups': 1, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHSCRIPT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 4, 5, 5), 'out_channels': 8, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 2, 'offset_groups': 1, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHEXPORT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 4, 5, 5), 'out_channels': 6, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 1, 'offset_groups': 2, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHSCRIPT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 4, 5, 5), 'out_channels': 6, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 1, 'offset_groups': 2, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHEXPORT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 4, 5, 5), 'out_channels': 8, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 2, 'offset_groups': 2, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHSCRIPT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 4, 5, 5), 'out_channels': 8, 'kernel_size': (3, 3), 'stride': (1, 1), 'padding': (0, 0), 'dilation': (1, 1), 'groups': 2, 'offset_groups': 2, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHEXPORT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 3, 6, 7), 'out_channels': 4, 'kernel_size': (2, 3), 'stride': (2, 1), 'padding': (1, 2), 'dilation': (2, 1), 'groups': 1, 'offset_groups': 1, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHSCRIPT]
         - coremltools/converters/mil/frontend/torch/test/test_torch_ops.py:13448 TestDeformConv2d.test_deform_conv2d[config={'input_shape': (1, 3, 6, 7), 'out_channels': 4, 'kernel_size': (2, 3), 'stride': (2, 1), 'padding': (1, 2), 'dilation': (2, 1), 'groups': 1, 'offset_groups': 1, 'use_mask': True, 'use_bias': True}-compute_unit=ComputeUnit.CPU_ONLY-backend=('mlprogram', 'fp32')-frontend=TorchFrontend.TORCHEXPORT]
      14 skipped

[SakshamKapoor2911]

Thanks. I reproduced the fp32 mlprogram native-prediction failures on macOS 14 with a fork GitHub Actions matrix.

Root cause appears to be the native Core ML resample path for unnormalized fractional padded-image coordinates. The lowering pixel-coordinate math and [x, y] coordinate order are correct, but macOS 14 mismatches when those coordinates are passed with coordinates_mode="unnormalized".

I pushed e758c12e, which keeps the same padded pixel-coordinate construction, normalizes those coordinates dynamically to [-1, 1], and uses coordinates_mode="normalized_minus_one_to_one" with align_corners=True.

Validation:

• Local Linux/Lambda conversion path: TestDeformConv2d 14 passed. This is not a full native Core ML validation because Linux has BlobWriter is None.
• Fork macOS matrix with the PR fix: macOS 14, macOS 15, and macOS latest passed for both TorchScript and TorchExport: https://github.com/SakshamKapoor2911/coremltools/actions/runs/28838146898

[SakshamKapoor2911]

I also addressed the inline test cleanup comments in 733aac040e6829cdf11049ac6ecad4635568413f:

• removed the custom deform_conv2d_backends list and now use the standard backends matrix
• removed the BlobWriter is None conversion-only path
• removed the graph-op assertion helper and now rely on run_compare_torch prediction parity

Validation for the cleaned version passed on the fork macOS matrix across macOS 14, macOS 15, and macOS latest for both TorchScript and TorchExport: https://github.com/SakshamKapoor2911/coremltools/actions/runs/28838429356

[TobyRoseman]

Have you verified your new unit tests are passing on macOS?

[SakshamKapoor2911]

Yes. All unit tests pass on macOS. I ran a fork CI matrix covering macos-14, macos-15, and macos-latest for both TorchScript and TorchExport frontends. On macOS, the test harness calls mlmodel.predict and compares Core ML output numerically against the PyTorch reference, so this is a full end-to-end check, not just a conversion smoke.

CI logs: https://github.com/SakshamKapoor2911/coremltools/actions/runs/28838429356

One caveat worth flagging: these ran on GitHub-hosted virtualized macOS runners with CPU_ONLY, so I wasn't able to verify behavior on physical Apple Silicon or the ANE since I don't have access to macOS devices. If the team wants to confirm on bare metal before merging, I think that could be a reasonable step.

[TobyRoseman]

CI: https://gitlab.com/coremltools1/coremltools/-/pipelines/2662950189

[SakshamKapoor2911]

The GitLab CI pipeline completed successfully on the latest commit (733aac0): https://gitlab.com/coremltools1/coremltools/-/pipelines/2662950189

The earlier CHANGES_REQUESTED was from the initial test failures (unnormalized coordinate mode), which were fixed by switching to normalized_minus_one_to_one in commit e758c12e. The pipeline now passes on both GitLab CI and my fork's macOS matrix (14/15/latest, both TorchScript and TorchExport).

Ready for another review when you have bandwidth.

[SakshamKapoor2911]

@TobyRoseman The test failures have been fixed (switched to normalized_minus_one coords) and all GitLab CI passes. Could you re-review?

[SakshamKapoor2911]

Hi @TobyRoseman — just a gentle follow-up. All GitLab CI passed on the latest commit. Could you take another look when you have a moment?

[TobyRoseman]

Thanks for the contribution @SakshamKapoor2911