M1 native install dependencies

[sudocurse]

M1 native install dependencies

• I read contributing guideline
• I didn't find a similar pull request already open.
• My PR is related to Spleeter only, not a derivative product (such as Webapplication, or GUI provided by others)

Description

Updates instructions for M1, and adds Darwin dependencies to pyproject.toml. The platform marker lets us pin a tf version specific for Darwin platforms to maintain compatibility.

How this patch was tested

I've tested these instructions on my M1, but could use testing on other machines (Darwin and otherwise) just to make sure everything's 100p.

Documentation link and external references

Mostly just put this together from other comments floating around on the repo.

[d-dawg78]

Hello, thank you for your PR! We are currently testing it out on MacOS, and will get back to you asap.

[d-dawg78]

Hi, we tried installing spleeter on a Mac M1 using your PR code...and haven't been able to make it work. Here are the key points:

1. Poetry installs packages from the .lock file. The one in your PR does not contain any installs for TF-metal and macOS.
2. When deleting the .lock file, the command poetry install, which generates the .lock file and installs the appropriate packages, does not work. The specific bug is:

Resolving dependencies... (828.6s)<debug>PackageInfo:</debug> Invalid constraint (importlib-metadata (>='4.4') ; python_version < "3.10") found in markdown-3.3.5 dependencies, skipping

Let us know if we are doing anything wrong. Again, thank you for the PR!

[sudocurse]

Ah, I may have found the issue: we're installing spleeter from local files, since this commit isn't in PyPI yet, so we need to do pip install . from the spleeter directory (I believe pip install spleeter will install from PyPI)

• I'm downloading a clean install of miniforge
• git checkout m1
• conda create -n spleeter
• conda activate spleeter
• conda install -c conda-forge ffmpeg libsndfile
• conda install -c apple tensorflow-deps==2.5.0 llvmlite==0.36.0 numba pandas==1.3.3
• pip install .

[sudocurse]

As for the poetry install/poetry update error: not sure that the markdown-3.3.5 error is related to my changes, see python-poetry/poetry#4777. I believe that markdown 3.3.4 and 3.3.6 should work.

[sudocurse]

markdown is being pulled in by tensorboard which gets pulled in by tensorflow so we should be able to just pin a version

[sudocurse]

I'm having trouble with poetry add markdown==3.3.4 because of some other issue with numpy. After I install markdown manually, I can generate this poetry lockfile. My only hesitancy is that it seems to want to pin importlib-metadata to Python 3.6/3.7. (and python 3.6 was EOL'd at the end of last year).

diff --git a/poetry.lock b/poetry.lock
index c71b6de..358042d 100644
--- a/poetry.lock
+++ b/poetry.lock
@@ -547,12 +547,15 @@ python-versions = ">=3.6,<3.10"
 
 [[package]]
 name = "markdown"
-version = "3.3.5"
+version = "3.3.4"
 description = "Python implementation of Markdown."
 category = "main"
 optional = false
 python-versions = ">=3.6"
 
+[package.dependencies]
+importlib-metadata = {version = "*", markers = "python_version < \"3.8\""}
+
 [package.extras]
 testing = ["coverage", "pyyaml"]
 
@@ -1132,6 +1135,45 @@ category = "main"
 optional = false
 python-versions = "*"
 
+[[package]]
+name = "tensorflow-macos"
+version = "2.5.0"
+description = "TensorFlow is an open source machine learning framework for everyone."
+category = "main"
+optional = false
+python-versions = "*"
+
+[package.dependencies]
+absl-py = ">=0.10,<1.0"
+astunparse = ">=1.6.3,<1.7.0"
+flatbuffers = ">=1.12.0,<1.13.0"
+gast = "0.4.0"
+google-pasta = ">=0.2,<1.0"
+grpcio = ">=1.34.0,<1.35.0"
+h5py = ">=3.1.0,<3.2.0"
+keras-nightly = ">=2.5.0.dev,<2.6.0"
+keras-preprocessing = ">=1.1.2,<1.2.0"
+numpy = ">=1.19.2,<1.20.0"
+opt-einsum = ">=3.3.0,<3.4.0"
+protobuf = ">=3.9.2"
+six = ">=1.15.0,<1.16.0"
+tensorboard = ">=2.5,<3.0"
+tensorflow-estimator = ">=2.5.0rc0,<2.6.0"
+termcolor = ">=1.1.0,<1.2.0"
+typing-extensions = ">=3.7.4,<3.8.0"
+wrapt = ">=1.12.1,<1.13.0"
+
+[[package]]
+name = "tensorflow-metal"
+version = "0.4.0"
+description = "TensorFlow acceleration for Mac GPUs."
+category = "main"
+optional = false
+python-versions = "*"
+
+[package.dependencies]
+six = ">=1.15.0,<1.16.0"
+
 [[package]]
 name = "termcolor"
 version = "1.1.0"
@@ -1266,7 +1308,7 @@ evaluation = ["musdb", "museval"]
 [metadata]
 lock-version = "1.1"
 python-versions = ">=3.6.2,<3.10"
-content-hash = "9888d4c62cd916e50a29f0cc78ff829e101f6af1bf04f100321a7927e412ff39"
+content-hash = "d652f0942837452a41e80a83f30601e73a7b9d6ca064df333439cbe91c3d5dab"
 
 [metadata.files]
 absl-py = [
@@ -1597,8 +1639,8 @@ llvmlite = [
     {file = "llvmlite-0.36.0.tar.gz", hash = "sha256:765128fdf5f149ed0b889ffbe2b05eb1717f8e20a5c87fa2b4018fbcce0fcfc9"},
 ]
 markdown = [
-    {file = "Markdown-3.3.5-py3-none-any.whl", hash = "sha256:0d2d09f75cb8d1ffc6770c65c61770b23a61708101f47bda416a002a0edbc480"},
-    {file = "Markdown-3.3.5.tar.gz", hash = "sha256:26e9546bfbcde5fcd072bd8f612c9c1b6e2677cb8aadbdf65206674f46dde069"},
+    {file = "Markdown-3.3.4-py3-none-any.whl", hash = "sha256:96c3ba1261de2f7547b46a00ea8463832c921d3f9d6aba3f255a6f71386db20c"},
+    {file = "Markdown-3.3.4.tar.gz", hash = "sha256:31b5b491868dcc87d6c24b7e3d19a0d730d59d3e46f4eea6430a321bed387a49"},
 ]
 musdb = [
     {file = "musdb-0.3.1-py2.py3-none-any.whl", hash = "sha256:db8f65b40c56938dab9e1b042254c6e3a0d75b139f98a96d6edbb646c0d25c7d"},
@@ -2156,6 +2198,16 @@ tensorflow = [
 tensorflow-estimator = [
     {file = "tensorflow_estimator-2.5.0-py2.py3-none-any.whl", hash = "sha256:d1fe76dee8b1dcab865d807a0246da0a9c4a635b1eba6e9545bf216c3aad6955"},
 ]
+tensorflow-macos = [
+    {file = "tensorflow_macos-2.5.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:b3049da572fbc636e1bfdde36de1d569ba424af6618debeeb1440e64a98f2b7c"},
+    {file = "tensorflow_macos-2.5.0-cp38-cp38-macosx_11_0_x86_64.whl", hash = "sha256:005dce89192f29c088ec835dffe0e08f0239a16383d5033b81ea52eaa961962b"},
+    {file = "tensorflow_macos-2.5.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:6391419f54a6935e7f8a215d8eaa1828e74494e0255646401aa79f01213a4a62"},
+]
+tensorflow-metal = [
+    {file = "tensorflow_metal-0.4.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:85c59b1b2c3c9ec8a08db3987741b0cd2ed4813e8786e3d6ac8c674ad11d49a8"},
+    {file = "tensorflow_metal-0.4.0-cp38-cp38-macosx_11_0_x86_64.whl", hash = "sha256:9c02e410e9d5ef833f82817b4cc1776826b929d5bc2b47f30df08ce64c65d789"},
+    {file = "tensorflow_metal-0.4.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:be7b38ec6f0bfb06cc7869200d6664b61b3073820f7bea9215f47f3376d4455f"},
+]
 termcolor = [
     {file = "termcolor-1.1.0.tar.gz", hash = "sha256:1d6d69ce66211143803fbc56652b41d73b4a400a2891d7bf7a1cdf4c02de613b"},
 ]
diff --git a/pyproject.toml b/pyproject.toml
index 3feb098..50c1712 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -55,6 +55,7 @@ museval = {version = "0.3.0", optional = true}
 tensorflow = {version = "2.5.0", markers = "sys_platform != 'darwin'"}
 tensorflow-macos = {version = "2.5.0", markers = "sys_platform == 'darwin'"}
 tensorflow-metal = {version = "0.4.0", markers = "sys_platform == 'darwin'"}
+markdown = "3.3.4"
 pandas = "^1.1.2"
 numpy = "<1.20.0,>=1.16.0"
 importlib-resources = {version = "^4.1.1", python = "<3.7"}

[sudocurse]

Wondering if the issues with poetry may be more related to installing different versions via conda vs poetry.lock

[d-dawg78]

We're going to close this PR. We have a working solution for installing Spleeter of Mac M1: see here. We also have plans to update Spleeter packaging for automatic installation of the package on any/all OS. You will all be notified when this takes place :))

[sudocurse]

@d-dawg78 that's not a native install and will use significantly more power and cpu. there is a way to get this natively.

[jefeblaize]

Your working solution @d-dawg78 is not only not a native solution, but doesn't work as of 12/2022. Is there no other update for this?