-
Notifications
You must be signed in to change notification settings - Fork 637
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RuntimeError: PyTorch convert function for op 'fft_rfftn' not implemented. #1311
Comments
Any updates on this? Or maybe someone can guide me how to write an STFT torch op? |
I am also facing this issue |
I'm working on librosa port to iOS: RosaKit (https://github.com/dhrebeniuk/RosaKit). There is available implementation for this function, so it can help you. |
Given our current set of MIL ops, I don't think there is a good way to implement this one: If anyone has any ideas, please share. |
@TobyRoseman you may check metal-fft code. I haven't tested the code but seems to be a way to go for Metal. Accelerate already seems to have support for 1D / 2D Fast Fourier Transforms although N-dimensional case is missing but is also need for research works like Fast Fourier Convolutions as in Resolution-robust Large Mask Inpainting with Fourier Convolutions. Maybe you can provide FFT as a MIL op (supported for both complex and real valued numbers). Using this FFT op you can build STFT and inverse STFT from it which can be very much helpful for audio processing. Just my two cents. |
@RahulBhalley - Thanks for the suggest. I've created an internal issue. |
I'm also facing this issue, does a list of PyTorch operators and their CoreML availability exists? including with the priority for adding them to the roadmap. |
I was hoping this would be included in the "Support for many new PyTorch and TensorFlow layers" announced for version 6.0. But it doesn't look like it is 😔. (At least not in beta 1) |
Apple probably won't release. It's sad. I've been waiting for almost 2 years now. |
Looking at the list of supported operations, I think the real bottleneck here is lack of support for complex numbers. Just being able to create a tensor of complex type (e.g. z = torch.complex(x, y)) is not supported. It's no wonder FFTs haven't been implemented, and I don't imagine it will be for some time unfortunately :( |
So many amazing models are not convertible due to a lack of FFT support. @apple, please make it available. |
Hi! any update for this issue? |
@manhnguyen92 , I guess it's very difficult , because there is few implementations of fft_rfftn, that all implemented via C-module, not in python. |
@dhrebeniuk Thank you! so sad to hear that. Have been Spent one week to convert model but get that impossible fixing issue. |
Same, a lot of time wasted. Which model were you trying to convert? |
Any model which use Mel-Spectrogram. Any sound classifier model on TF2 or PyTorch. |
@roimulia2 I have tried convert Lama inpainting model. |
Hey @manhnguyen92, I was trying to do the same thing. I'll make sure to post it here if I succeed. |
@roimulia2 , Yes I do. I think some apps one App Store use this library but don't know how they could convert it. |
@manhnguyen92 They've implemented it server side, not on device. And even if you were able to convert it to CoreML you will face the the maximum 2Go used RAM per app limit by Apple |
@pboudoin No they didn't. Their apps can be used very well in offline mode. |
mark |
Coremltools 6.2 includes support for PyTorch's Thanks @junpeiz for adding that. |
I converted LaMa (uses FFT heavily) & it doesn't load on ANE. Is FFT ANE compatible or not? @TobyRoseman |
|
@RahulBhalley Could you provide a code snippet to reproduce the issue? We tested several models in unit test that the converted FFT model is ANE compatible. Your code snippet could help us determine if it's due to FFT or other parts of LaMa model that is not ANE compatible. |
@hzphzp the PyTorch's |
Thank you for your reply. But after I change the code rfft2 to rfftn, I still got the following error: |
@hzphzp from the error message I cannot tell. Could you open another issue with a code snippet to reproduce this error? |
I have been trying to convert PyTorch model to coreml, but face some issues during the conversion of fft_rfftn and fft_irfftn operators. Could you kindly introduce me how to register these operators like this?
The text was updated successfully, but these errors were encountered: