Skip to content

Run 2D MaxPool on the Metal GPU - #2557

Closed
czoli1976 wants to merge 4 commits into
sonos:mainfrom
czoli1976:feature/metal-maxpool
Closed

Run 2D MaxPool on the Metal GPU#2557
czoli1976 wants to merge 4 commits into
sonos:mainfrom
czoli1976:feature/metal-maxpool

Conversation

@czoli1976

Copy link
Copy Markdown
Contributor

MaxPool had no Metal kernel, so it round-tripped its tensor through the host in the middle of a GPU graph. This adds a kernel that pools two spatial axes with every axis stride passed explicitly, so NCHW and NHWC share one path.

Stacked on #2554, #2555 and #2556 — this branch carries their commits and this PR's own change is the last one. It only depends on #2556 for the benchmark story, not for the code, so it can also be taken on its own if that's easier.

Effect

The point of this one is not the kernel's own throughput, it's what it removes. u2netp (rembg), 1×3×320×320, --metal -O, min of 3 interleaved rounds:

host round-trips (DeviceSync*) end-to-end
main 150 560.7 ms
+ Metal Resize (#2556) 74 460.7 ms
+ Metal MaxPool (this) 8 359.8 ms

The graph is now essentially GPU-resident, and that is what makes Metal win on this model: 359.8 ms vs 419.1 ms on CPU. It lost before either kernel (560.7) and still lost with Resize alone (460.7) — the residual host transfers dominated. Resize was necessary, MaxPool was what flipped it. Together, 1.56× on the Metal path.

Outputs match the CPU runtime to 5.96e-7 max abs across all seven model outputs.

Notes

Padding positions are skipped rather than compared, so a window lying entirely in the padding keeps the CPU op's min_value() result.

The translator registers for both MaxPool and OptMaxPool. The Metal transform runs before the optimize pass, so the node is still MaxPool at translation time even though a -O dump shows OptMaxPool — registering only for the optimized op silently never fires. OptMaxPool is added to core::ops::cnn's public re-exports for this.

Two limits worth knowing:

  • Scoped to rank 4, two spatial axes, no index outputs, concrete shapes. 1D/3D pooling, with_index_outputs and symbolic shapes aren't translated and stay on CPU.
  • One thread per output element with a 1×1×1 threadgroup, matching the existing Gather/DiagGather kernels. A tuned threadgroup size probably has more in it; I left that alone because the transfer elimination is the dominant term here and I didn't want to tune on a single model.

Testing

test-metal 22762 passed / 0 failed (558 of them pooling node tests; I checked the nodes actually land on MetalMaxPool rather than silently falling back to CPU). onnx-tests.sh 1_19_1 and test-unit-core green. cargo fmt --all and cargo clippy clean.

🍍

The nearest-neighbour Resize declutter lowered any integer scale to
Reshape -> Tile -> Reshape, but that replication pattern only matches some
coordinate transform and tie-break pairs, so half_pixel or ceil rounding
silently produced shifted output. Both declutters now probe the rounding of
each upsampled axis and lower only when it really is pixel replication.
Resize silently ignored antialias and keep_aspect_ratio_policy, carried axes as
a dead field, and rejected half_pixel_symmetric and tf_crop_and_resize, so
opset-18 and -19 models were either wrong or refused outright. The op now
implements all of them, and resamples through a per-axis plan of precomputed
taps and weights applied over contiguous runs rather than recomputing a
dynamic-rank index for every output element.
Resize had no Metal kernel, so every node round-tripped the tensor through the
host in the middle of a GPU graph. A resample-one-axis kernel now consumes the
same per-axis tap-and-weight plan the CPU op builds, which makes it independent
of the interpolator; the plan is baked at translation time, so the node keeps
only its data input and the scales/sizes TDim constant is dropped.
MaxPool had no Metal kernel, so it round-tripped its tensor through the host in
the middle of a GPU graph. A kernel now pools two spatial axes with every axis
stride passed explicitly, which covers NCHW and NHWC alike, and skips positions
in the padding so an all-padding window keeps the CPU op's lowest-value result.
The geometry is resolved at translation time; index outputs, other ranks and
symbolic shapes stay on CPU.
@czoli1976
czoli1976 marked this pull request as draft August 2, 2026 18:35
@czoli1976

Copy link
Copy Markdown
Contributor Author

Marking this draft: it overlaps #2553, which I opened earlier and had not checked before writing this. #2553 already adds Metal max and sum pooling, with a better-coalesced NHWC dispatch.

The two are not identical, though — #2553 deliberately leaves NCHW on the CPU path, and this one covers NCHW, which is what an ONNX/PyTorch export like u2netp uses. I will rework the useful part of this as NCHW support on top of #2553 rather than ask anyone to review two pooling kernels.

@czoli1976

Copy link
Copy Markdown
Contributor Author

Superseded by #2553, which now carries channels-first max pooling as a second commit. Closing this rather than asking anyone to review two pooling kernels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant