Training on M1 Pro

[Cptn-Reynolds]

I can't seem to get the training to work on a Macbook M1 Pro with current macOS 13.4.1. I tried both with virtual env directly in python and via conda. I tried without torchaudio (as stated in the wiki) but also with torch installed (tested both full and nightly).

I can presample, preconfig and prehubert just fine, but then during the training I am stuck on
NotImplementedError: The operator 'aten::_weight_norm_interface' is not currently implemented for the MPS device... As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK= to use the CPU as a fallback for this op. and get thrown back to the prompt

Normally I would say well okay I'll have to wait for the implementation, but I know from reddit and other forums, that others got it running on an M1, so what am I missing?
There are several other WARNINGs too

[08:31:38] WARNING  [08:31:38] /Users/masked/so-vits/lib/python3.10/site-packages/torch/_utils.py:776: UserWarning:    warnings.py:109
                    TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only                   
                    storage class. This should only matter to you if you are using storages directly.  To access                       
                    UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()                                  
                      return self.fget.__get__(instance, owner)()                                                                      
                                                                                                                                       
[08:31:46] WARNING  [08:31:46] /Users/masked/so-vits/lib/python3.10/site-packages/so_vits_svc_fork/f0.py:228:          warnings.py:109
                    UserWarning: MPS: no support for int64 min/max ops, casting it to int32 (Triggered internally at                   
                    /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/mps/operations/ReduceOps.mm:1271.)                 
                      assert f0_coarse.max() <= 255 and f0_coarse.min() >= 1, (                                                        
                                                                                                                                       
           WARNING  [08:31:46]                                                                                          warnings.py:109
                    /Users/masked/so-vits/lib/python3.10/site-packages/so_vits_svc_fork/modules/attentions.py:391:                    
                    UserWarning: MPS: The constant padding of more than 3 dimensions is not currently supported                        
                    natively. It uses View Ops default implementation to run. This may have performance implications.                  
                    (Triggered internally at    

Help would be appreciated :) thanks

[Cptn-Reynolds]

I managed to at least get the process running by setting PYTORCH_ENABLE_MPS_FALLBACK=1
As I understand it, the system only falls back to the CPU for those operations where MPS support is not yet implemented. This is supported by the fact, that my GPU is running at 60-80% constantly.

One Epoch currently takes about 15min, which would result at 2-3 weeks runtime for 2k Epochs. How can it still take so long, if obviously most of the work is done by the GPU? Here are the values I set in the config

"train": {
   "log_interval": 100,
   "eval_interval": 200,
   "epochs": 2000,
...
   "batch_size": 16,
...

   "num_workers": 8,

[haimco10]

A fix was merged a few days ago. So I believe you can already get it in the nightly build.

pip3 install --pre torch torchaudio --index-url https://download.pytorch.org/whl/nightly/cpu