Support MPS on Macos devices

[bayedieng]

This project would have a significant speed boost on mac if the demucs model were to use mps instead of cpu in a similar way that the windows binary uses the cuda backend.

[iffyloop]

Yes, absolutely. Last time I checked, Demucs wasn't compatible with the MPS backend; has this changed?

[bayedieng]

Yes it does I've tried it quite recently.

[iffyloop]

That's excellent, thanks for letting me know. Bringing those updates into StemRoller will be a hassle, since I'll need to repackage the Demucs standalone binaries. I need to merge in some PRs that have been open for a couple of weeks; after that, I'll look into packaging an updated Demucs with MPS support.

[hrueger]

Hi @iffyloop,
thanks for this great app. I was also looking into this, and it doesn't seem to difficult: running demucs I can use the -d mps flag to get gpu acceleration. However, when cxfreezing the current version and running it, I get this:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/cx_Freeze/initscripts/__startup__.py", line 124, in run
    module_init.run(name + "__main__")
  File "/opt/homebrew/lib/python3.11/site-packages/cx_Freeze/initscripts/console.py", line 16, in run
    exec(code, module_main.__dict__)
  File "main.py", line 4, in <module>
  File "/opt/homebrew/lib/python3.11/site-packages/demucs/separate.py", line 173, in main
    sources = apply_model(model, wav[None], device=args.device, shifts=args.shifts,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/demucs/apply.py", line 189, in apply_model
    out = apply_model(sub_model, mix, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/demucs/apply.py", line 216, in apply_model
    shifted_out = apply_model(model, shifted, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/demucs/apply.py", line 250, in apply_model
    chunk_out = future.result()
                ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/demucs/utils.py", line 129, in result
    return self.func(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/demucs/apply.py", line 271, in apply_model
    out = model(padded_mix)
          ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/demucs/htdemucs.py", line 588, in forward
    x = rearrange(x, "b c f t-> b c (f t)")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/einops/einops.py", line 591, in rearrange
    return reduce(tensor, pattern, reduction="rearrange", **axes_lengths)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/einops/einops.py", line 518, in reduce
    backend = get_backend(tensor)
              ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/einops/_backends.py", line 53, in get_backend
    backend = BackendSubclass()
              ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/einops/_backends.py", line 221, in __init__
    from . import _torch_specific  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/einops/_torch_specific.py", line 127, in <module>
    allow_ops_in_compiled_graph()
  File "/opt/homebrew/lib/python3.11/site-packages/einops/_torch_specific.py", line 106, in allow_ops_in_compiled_graph
    from torch._dynamo import allow_in_graph
  File "/opt/homebrew/lib/python3.11/site-packages/torch/_dynamo/__init__.py", line 2, in <module>
    from . import allowed_functions, convert_frame, eval_frame, resume_execution
  File "/opt/homebrew/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 34, in <module>
    from .eval_frame import always_optimize_code_objects, skip_code, TorchPatcher
  File "/opt/homebrew/lib/python3.11/site-packages/torch/_dynamo/eval_frame.py", line 62, in <module>
    from . import config, convert_frame, external_utils, skipfiles, utils
  File "/opt/homebrew/lib/python3.11/site-packages/torch/_dynamo/skipfiles.py", line 72, in <module>
    ] + [
        ^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/_dynamo/skipfiles.py", line 74, in <listcomp>
    _module_dir(m)
  File "/opt/homebrew/lib/python3.11/site-packages/torch/_dynamo/skipfiles.py", line 62, in _module_dir
    return _strip_init_py(m.__file__)
                          ^^^^^^^^^^
AttributeError: module 'abc' has no attribute '__file__'. Did you mean: '__name__'?

Any idea? Thanks in advance.

[iffyloop]

@hrueger Please could you test this build? https://github.com/stemrollerapp/demucs-cxfreeze/releases/download/release-26a2baeb0058444b3cf87028d9df721d37c78dfb/demucs-cxfreeze-mac.zip
Planning to do a new release soon with this version, but unfortunately I don't have any Apple Silicon hardware to test on. You'll need to download the files listed at https://github.com/stemrollerapp/demucs-cxfreeze?tab=readme-ov-file#models and put them in a directory named htdemucs_ft, then run demucs-cxfreeze with the arguments -n htdemucs_ft --repo path/to/your/directory/htdemucs_ft

[hrueger]

Hi @iffyloop,
thanks for the update! I tried to test it, but I run into macos singing errors (I believe).
First time running demucs-cxfreeze, I had to allow it in the Security Settings. However, now I get the following popup:

Sorry for the German, it translates to:

"Python" cannot be opened
because Apple cannot search for
malware in it.
This software needs to be updated.
Contact the developer for more information.
information.
Firefox loaded this file today at 10:02.
Show in Finder                    OK

Is that binary signed differently than the release version?

[hrueger]

Turns out, if you run lib/Python manually once, you can allow running that binary and you get a bit further.
However, then you need to approve every single .so file. I just spend 20 minutes and it's still going on, I thing that does not make sense ;-)

[iffyloop]

@hrueger whoops, forgot to mention: run xattr -dr com.apple.quarantine /path/to/demucs-cxfreeze-mac
I'm not sure why this happens now but wasn't happening before. Maybe an issue with cxfreeze, or a macOS update?

[hrueger]

Thanks! That works and is so much faster 😅
When now using -d mps it uses almost all of my CPU and GPU 🎉 and it is sooo much faster!

I'm not sure why this happens now but wasn't happening before. Maybe an issue with cxfreeze, or a macOS update?

No idea... I'm running macOS Sonoma 14.4.1.

[iffyloop]

Thanks @hrueger, glad it works for you! Released in #59, but feel free to re-open this issue if anyone encounters problems.

[iffyloop]

@hrueger please could you test the full StemRoller package available here? https://github.com/stemrollerapp/stemroller/releases/download/2.0.5/stemroller-2.0.5-mac.zip
Another user with an Apple Silicon device is reporting that it's not running any faster for them and not using GPU at all, am wondering if we fixed demucs-cxfreeze but for some reason there is a bug in StemRoller which is preventing it from being launched with the correct environment/settings.

UPDATE: the user had changed the preferences to Always use CPU. Sorry for the false alarm!