Skip to content

[CoreML EP] Add GatherND builder#28598

Draft
maxwbuckley wants to merge 6 commits into
microsoft:mainfrom
maxwbuckley:coreml-gathernd
Draft

[CoreML EP] Add GatherND builder#28598
maxwbuckley wants to merge 6 commits into
microsoft:mainfrom
maxwbuckley:coreml-gathernd

Conversation

@maxwbuckley
Copy link
Copy Markdown
Contributor

@maxwbuckley maxwbuckley commented May 20, 2026

Summary

New ML Program op builder: ONNX GatherND → CoreML gather_nd.

  • batch_dims must be 0 — the iOS15 gather_nd op has no batch_dims parameter;
    IsOpSupportedImpl rejects other values.
  • CoreML's gather_nd rejects a bool x, but transformer attention-mask
    graphs gather from bool tensors. For bool data the builder lowers the op as
    cast(bool→int32) → gather_nd → cast(int32→bool); int32 represents 0/1 exactly,
    so the round-trip is lossless.
  • validate_indices is passed explicitly — the ML Program parser rejects
    gather_nd without it (the same quirk the gather builder works around).
  • ML-Program-only; IsOpSupportedImpl rejects the NeuralNetwork format.

Depends on the bool-Cast PR

The bool-data GatherND test needs Cast as the int ↔ bool producer/consumer so
the bool tensors stay internal to the CoreML partition (a partition cannot have
bool I/O). This branch is stacked on coreml-cast-bool — the cb43b7c75f
commit in this PR is the bool-Cast PR and drops from this diff once that one
merges.

Tests (coreml_basic_test.cc)

  • GatherND_MLProgram — a float GatherND runs on CoreML, matches CPU.
  • GatherNDBoolData_MLProgram — a Cast → GatherND → Cast bool chain runs fully
    on CoreML, exercising the cast round-trip lowering.
  • GatherNDNeuralNetworkNotSupportedGatherND falls back on the NeuralNetwork
    format.
  • GatherNDBatchDimsNotSupportedGatherND with batch_dims=1 falls back to CPU.

Doc: coreml_supported_mlprogram_ops.md lists GatherND.

Series — CoreML EP coverage for transformer / diffusion graphs

Together with #28278 (scalar-Gather), the series takes BERT / GPT-2 / ViT /
diffusion-UNet graphs — tiny and full-size — from 2 CoreML partitions to 1, with
zero graph breaks.

maxwbuckley and others added 2 commits May 20, 2026 16:18
Two changes to the ML Program Cast builder:

1. Accept BOOL as a source and target dtype in HasSupportedInputsImpl. The
   ML Program `cast` op already handles bool, and AddToModelBuilderImpl
   already maps `to == BOOL`; only the input/output type gate omitted it.
   This lets int64<->bool<->float casts (transformer attention-mask graphs)
   stay on CoreML.

2. Move the "no preceding node" check after the ML Program early-return. It
   was legacy gating for the NeuralNetwork ArgMax-only path (which
   dereferences InputEdgesBegin()); on the ML Program path a Cast fed
   directly by a graph input is fine, and rejecting it forced needless CPU
   fallback.

Tests (coreml_basic_test.cc):
- CastBoolRoundTrip_MLProgram: an int64->bool->float cast chain runs fully
  on CoreML and matches the CPU reference. The bool tensor is internal (a
  CoreML partition cannot have bool I/O) and the first Cast is graph-input
  fed.
- CastNonArgMaxNeuralNetworkNotSupported: the same chain falls back to CPU
  on the NeuralNetwork format, guarding the IsOpSupportedImpl reordering.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New ML Program op builder: ONNX GatherND -> CoreML `gather_nd`.

- batch_dims must be 0 (the iOS15 gather_nd op has no batch_dims
  parameter); IsOpSupportedImpl rejects other values.
- CoreML's gather_nd rejects a bool `x`, but transformer attention-mask
  graphs gather from bool tensors. For bool data the builder lowers the op
  as cast(bool->int32) -> gather_nd -> cast(int32->bool); int32 represents
  0/1 exactly so the round-trip is lossless.
- `validate_indices` is passed explicitly -- the ML Program parser rejects
  gather_nd without it, the same quirk the `gather` builder works around.
- ML-Program-only; IsOpSupportedImpl rejects the NeuralNetwork format.

Stacked on the bool-Cast branch: GatherND's bool-data test needs Cast as
the int<->bool producer/consumer so the bool tensors stay internal to the
CoreML partition (a partition cannot have bool I/O).

Tests (coreml_basic_test.cc):
- GatherND_MLProgram: a float GatherND runs on CoreML, matches CPU.
- GatherNDBoolData_MLProgram: a Cast->GatherND->Cast bool chain runs fully
  on CoreML, exercising the cast round-trip lowering.
- GatherNDNeuralNetworkNotSupported: GatherND falls back on NeuralNetwork.
- GatherNDBatchDimsNotSupported: GatherND with batch_dims=1 falls back to CPU.

Doc: coreml_supported_mlprogram_ops.md lists GatherND.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
maxwbuckley and others added 4 commits May 21, 2026 09:34
CastBoolRoundTrip_MLProgram exercised int64 -> Cast(bool) -> Cast(float).
CoreML's compiler fuses the two back-to-back `cast` ops and drops the bool
clamp (cast(cast(x,bool),fp32) collapses to cast(x,fp32)), so the round-trip
produces the raw input value instead of 0/1 -- the test can't be numerically
verified standalone.

The bool-Cast support itself is correct: it is exercised end to end by the
dependent PRs, where a non-Cast op sits between the int<->bool casts so no
fusion occurs -- Cast->And->Cast (Where/And PR) and Cast->GatherND->Cast
(GatherND PR), both numerically verified against the CPU EP.

CastNonArgMaxNeuralNetworkNotSupported (the NeuralNetwork-format negative
test) is kept; it guards the IsOpSupportedImpl reordering.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant