[MPS] Add Conv3D support for MPS

[LucasSte]

Fixes #77818

I saw that PR #99246 was approved, but no one fixed the rebase conflicts, so I am bringing this up again to be merged.
I am leveraging @mattiaspaul work. Quoting the description here:

• this pull request enables 3D convolutions (forward/backward) for MPS (Apple Silicon) within the same Convolution.mm file as conv2d.
• does not support channel_last (since pytorch doesn't implement channel_last for 3D tensors)
• does not support conv3d_transpose and treats depth-separable convolutions not as normal case (there are no MPS kernels available for either of those so far)
• requires MacOS >=13.2 (Ventura)

Please, let me know if there are any other changes needed and I'll be happy to implement them.

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: LucasSte / name: Lucas Steuernagel (762b32c, be120dd, 047c823, 48124c4, 8fea514, c9950e1, 993fc73, 67260e3, cdcc411, 61b2bae)

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/114183

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (3 Unrelated Failures)

As of commit 61b2bae with merge base 8c4812b ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Mac MPS / macos-12-py3-arm64-mps / test (mps, 1, 1, macos-m1-13) (gh)
• periodic / macos-12-py3-x86-64 / build (gh)
  Final attempt failed. Child_process exited with error code 1

UNSTABLE - The following job failed but was likely due to flakiness present on trunk and has been marked as unstable:

• periodic / linux-focal-rocm5.7-py3.8 / test (distributed, 1, 2, linux.rocm.gpu, unstable) (gh)
  distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_baseline_aot_eager_multiprocess

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[LucasSte]

If someone can reopen #99246 and merge it, I would be happy to close this PR, as #99246 is from the original author.

[LucasSte]

Either the build CI does not have the newer MacOS SDK or it is still running MacOS 12, because it does not work without the conditional compilation statement I added: #ifdef MAC_OS_VERSION_13_2.

On the other hand, the CI test job is on MacOS 13.2.1. This means the code for Conv3D is not compiled, but the test for it is executed, leading to a segmentation fault.

@kulinseth
Can someone explain how the CI jobs are configured? (MacOS version and SDK version)

[malfet]

@pytorchbot merge -f "Lint + MPS are green"

[pytorchmergebot]

The merge job was canceled. If you believe this is a mistake, then you can re trigger it through pytorch-bot.

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[motannhoff]

how do i make this work? still getting same "Conv3d is not supported on MPS" error when using SVD on comfyUI on M2 MBP Ventura 13.5.2 thank you :)

[cwallen]

@motannhoff SVD in comfyUI is now working for me on my M2, though it feels really slow.
If you are still seeing the same error, my first guess would be to make sure you have the latest nightly with it included. I ran pip3 install --force-reinstall --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu and look for version numbers of at least the 16th.

[laochake]

@motannhoff SVD in comfyUI is now working for me on my M2, though it feels really slow. If you are still seeing the same error, my first guess would be to make sure you have the latest nightly with it included. I ran pip3 install --force-reinstall --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu and look for version numbers of at least the 16th.

I followed your instructions and ran 'pip3 install --force-reinstall --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu' on M2. However, when running svd, I still encountered the error 'RuntimeError: Conv3D is not supported on MPS'. I have been stuck for a few days now. Could you please provide more details on how you managed to run it?

[motannhoff]

@motannhoff SVD in comfyUI is now working for me on my M2, though it feels really slow. If you are still seeing the same error, my first guess would be to make sure you have the latest nightly with it included. I ran pip3 install --force-reinstall --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu and look for version numbers of at least the 16th.

I followed your instructions and ran 'pip3 install --force-reinstall --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu' on M2. However, when running svd, I still encountered the error 'RuntimeError: Conv3D is not supported on MPS'. I have been stuck for a few days now. Could you please provide more details on how you managed to run it?

same for me. still getting error.

[reforia]

Managed to run successfully on M3 Max w/ ComfyUI on 12/16, for ppl. still see the Conv3D is not supported, make sure that the torchvision and torchaudio is installed w/ --no-dep flag, otherwise they will force uninstall the dev231216 version and install back dev231215 version of torch nightly.

So my approach is force install torch 231216 (and upwards)
pip install https://download.pytorch.org/whl/nightly/cpu/torch-2.3.0.dev20231216-cp311-none-macosx_11_0_arm64.whl

then
pip install torchvision==0.18.0.dev20231215 torchaudio==2.2.0.dev20231215 --no-deps

Prob. soon they will update their dependency to something higher than 20231215, then it should just work with normal installation method

Speed is around 11s/it for a 768*512 image, so a 14 frams vid took bout 3 mins

[MoranARM]

Managed to run successfully on M3 Max w/ ComfyUI on 12/16, for ppl. still see the Conv3D is not supported, make sure that the torchvision and torchaudio is installed w/ --no-dep flag, otherwise they will force uninstall the dev231216 version and install back dev231215 version of torch nightly.

So my approach is force install torch 231216 (and upwards) pip install https://download.pytorch.org/whl/nightly/cpu/torch-2.3.0.dev20231216-cp311-none-macosx_11_0_arm64.whl

then pip install torchvision==0.18.0.dev20231215 torchaudio==2.2.0.dev20231215 --no-deps

Prob. soon they will update their dependency to something higher than 20231215, then it should just work with normal installation method

Speed is around 11s/it for a 768*512 image, so a 14 frams vid took bout 3 mins

Tried this method and got it to work for a single run, but when trying to generate another video I'm getting an error about a leaked semaphor, not sure if anyone else is getting this or how to force it to de-allocate the semaphor:

/AppleInternal/Library/BuildRoots/0032d1ee-80fd-11ee-8227-6aecfccc70fe/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:761: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'
Abort trap: 6           python main.py --force-fp16
/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown 

Tried on M3 Max 128G, python 3.10.12

Also if there is a better place to discuss this than a closed github merge I'll gladly move the discussion there

[gabrie]

Managed to run successfully on M3 Max w/ ComfyUI on 12/16, for ppl. still see the Conv3D is not supported, make sure that the torchvision and torchaudio is installed w/ --no-dep flag, otherwise they will force uninstall the dev231216 version and install back dev231215 version of torch nightly.

So my approach is force install torch 231216 (and upwards) pip install https://download.pytorch.org/whl/nightly/cpu/torch-2.3.0.dev20231216-cp311-none-macosx_11_0_arm64.whl

then pip install torchvision==0.18.0.dev20231215 torchaudio==2.2.0.dev20231215 --no-deps

Prob. soon they will update their dependency to something higher than 20231215, then it should just work with normal installation method

Speed is around 11s/it for a 768*512 image, so a 14 frams vid took bout 3 mins

This solution seems working, I got one extra error, then set env with "PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0"
Then it is working, but slow.

[YexiongLin]

Managed to run successfully on M3 Max w/ ComfyUI on 12/16, for ppl. still see the Conv3D is not supported, make sure that the torchvision and torchaudio is installed w/ --no-dep flag, otherwise they will force uninstall the dev231216 version and install back dev231215 version of torch nightly.

So my approach is force install torch 231216 (and upwards) pip install https://download.pytorch.org/whl/nightly/cpu/torch-2.3.0.dev20231216-cp311-none-macosx_11_0_arm64.whl

then pip install torchvision==0.18.0.dev20231215 torchaudio==2.2.0.dev20231215 --no-deps

Prob. soon they will update their dependency to something higher than 20231215, then it should just work with normal installation method

Speed is around 11s/it for a 768*512 image, so a 14 frams vid took bout 3 mins

It works, the versions are

torch 2.3.0.dev20231216
torchaudio 2.2.0.dev20231116
torchvision 0.18.0.dev20231216

Tried on M3, python 3.11.5

[bmh2127]

@mattiaspaul @LucasSte Why is this closed? I am able to successfully use ConvTranspose3D when following the directions here:
https://biofrenk.hashnode.dev/running-convtranspose3d-with-mps-acceleration-on-apple-silicon-macs
Is there a way it can be merged to main?

[p-iosifidis]

@mattiaspaul @LucasSte Why is this closed? I am able to successfully use ConvTranspose3D when following the directions here: https://biofrenk.hashnode.dev/running-convtranspose3d-with-mps-acceleration-on-apple-silicon-macs Is there a way it can be merged to main?

How did you manage to compile that? It's crashing and burning over here.
Huh, figured it out ... apparently it couldn't find it, not sure why. I set the flags manually.

export SDKROOT=$(xcrun --show-sdk-path)
export CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
export CXX=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++

-- the original error --

-- The OBJC compiler identification is AppleClang 16.0.0.16000026
-- The OBJCXX compiler identification is AppleClang 16.0.0.16000026
-- Detecting OBJC compiler ABI info
-- Detecting OBJC compiler ABI info - failed
-- Check for working OBJC compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working OBJC compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - broken
CMake Error at /Users/petros/_tmp/pytorch_MPS/venv/lib/python3.10/site-packages/cmake/data/share/cmake-3.25/Modules/CMakeTestOBJCCompiler.cmake:67 (message):
  The Objective-C compiler

    "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /Users/petros/_tmp/pytorch_MPS/pytorch/build/CMakeFiles/CMakeScratch/TryCompile-FpJpx8
    
    Run Build Command(s):/Users/petros/_tmp/pytorch_MPS/venv/bin/ninja cmTC_3171c && [1/2] Building OBJC object CMakeFiles/cmTC_3171c.dir/testObjCCompiler.m.o
    [2/2] Linking OBJC executable cmTC_3171c
    FAILED: cmTC_3171c 
    : && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -arch arm64 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/homebrew/opt/qt@5/lib: -rdynamic CMakeFiles/cmTC_3171c.dir/testObjCCompiler.m.o -o cmTC_3171c   && :
    ld: warning: search path '/opt/homebrew/opt/qt@5/lib:' not found
    ld: library 'System' not found
    cc: error: linker command failed with exit code 1 (use -v to see invocation)
    ninja: build stopped: subcommand failed.
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  caffe2/CMakeLists.txt:816 (enable_language)


-- Configuring incomplete, errors occurred!

[openSourcerer9000]

@LucasSte @malfet @kulinseth is conv3d supported on mps now in the latest version or not? All video generation models are broken on mps, I'm wondering if this is the reason.