GPU acceleration for Apple's M1 chip?

[chris-dare]

🚀 Feature

Hi,

I was wondering if we could evaluate PyTorch's performance on Apple's new M1 chip. I'm also wondering how we could possibly optimize Pytorch's capabilities on M1 GPUs/neural engines.

I know the issue of supporting acceleration frameworks outside of CUDA has been discussed in previous issues like #488..but I think this is worth a revisit. In Apple's big reveal today, we learned that Apple's on a roll with 50% of product usage growth being as a result of new users this year. Given that Apple is moving to these in-house designed chips, enhanced support for these chips could make deep learning on personal laptops a better experience for many researchers and engineers. I think this really aligns with PyTorch's theme of facilitating deep learning from research to production.

I'm not quite sure how this should go down. But these could be important:

1. A study on M1 chips
2. Evaluation of Pytorch's performance on M1 chips
3. Assessment on M1's compatibility with acceleration frameworks compatible with PyTorch (best bet would be CUDA transpilation..from what I see at OpenCL Support #488)
4. Investigating enhancements to PyTorch that can take advantage of M1's ML features.

cc @VitalyFedyunin @ngimel

[lqf96]

See also #47688, which is another issue on the same topic.

[dbuades]

https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html

Something like this in Pytorch would definitely be very cool!

[lqf96]

@dbermond From this blog it seems like PyTorch can also take advantage of the ML Compute Framework which is just added in macOS 11 and iOS 14. However, this is not compatible with older macOS and iOS and you still need to create computation graphs upfront. I'm also wondering if Apple is going to collaborate with Facebook to bring the acceleration to PyTorch.

[lanking520]

It's 2020 now and AMD GPU are still not officially supported by PyTorch... How much percentage of GPU support do you expect to have? Maybe we should focus on how to get PyTorch CPU work on M1 before we jump to GPU...

[toshi2k2]

I don't know but the Pytorch team seems to be exclusively fond of Intel and Nvidia. I have a general dislike for tensorflow but at least they provide some support for AMD and OpenCL(SYCL) - the ways things are going now, I think I'd be going back to TF despite all the drawbacks.

[bemoregt]

I hope so !!!

[chris-dare]

Perhaps "GPU acceleration" is not the best title for this issue. To be clear, I put that there because it seems to be the most promising component yielding the largest boost....based on the presentation we saw from Apple last week (15x better). That needs verification through tests though.

It could be beneficial to look at the chip itself. The ML Compute Framework seems to suggest that training can take place on both CPU and GPU...although the text, imo, isn't definitive. But here's what it says:

"Accelerate training and validation of neural networks across the CPU and one or more GPUs."

So the unified memory architecture is what Apple says it is, then there's no need to copy data between CPU and GPU. And it's the chip that really becomes the focus. That pretty serious.

[chris-dare]

That said, I do believe there will be performance differences between CPU, GPU and Neural engine. I think that's a given.

[BramVanroy]

@toshi2k2 Work is being done on HIP (AMD) and a nightly version is already out: #10670 (comment)

[toshi2k2]

@BramVanroy it is being done but its still unstable. And there is still no official support for HIP. Point is, an open source project should not, in all morality, cater to anti open source and monopolistic companies. They can build their own versions of frameworks or accelerators whenever they want (e.g. Apple).

[rashomon-gh]

@BramVanroy it is being done but its still unstable. And there is still no official support for HIP. Point is, an open source project should not, in all morality, cater to anti open source and monopolistic companies. They can build their own versions of frameworks or accelerators whenever they want (e.g. Apple).

Which part of CUDA is open source? Just asking because apart from AMD, everyone has made their accelerators proprietary.