support of audioread/librosa package

[rjt10]

Got error when trying to pip install these two packages with this:

            pip {
                install "six==1.10.0"
                install "audioread==2.1.5"
            }

The error message is:

Collecting audioread==2.1.5

  Could not find a version that satisfies the requirement audioread==2.1.5 (from versions: )
No matching distribution found for audioread==2.1.5 (NOTE: Chaquopy only supports wheels, not sdist packages)
Exit status 

Looking at the past feature requests, I have an impression that the python packages need to be manually ported to be available through Chaquopy, it is the case? If so, I am wondering if there's a way for community to help porting them?

Thanks.

[mhsmith]

Packages which are available on PyPI as pure Python wheels don't need to be rebuilt, but all others do. We have some unreleased scripts to assist in this, which work fine for packages which are pure Python or self-contained native code. However, we can't currently handle packages which depend on external native libraries or executables. In order to use these on Android, we'd also need a procedure for building and packaging the external components themselves.

[EDIT 2018-08: All pure-Python packages can now be installed directly, including sdists. Packages with native components, including external non-Python libraries, still need to be cross-compiled by us and added to the repository.]

This is the case with audioread, which (like many Python audio decoding libraries) depends on FFmpeg (#143). If you only need to read uncompressed audio files, it looks like it'll still work. [EDIT 2018-08: It can now be installed with the current version of Chaquopy.] Otherwise, you'll have to use a different approach. The obvious way would be to use Chaquopy to access the Android MediaCodec API from Python, though that looks pretty unfriendly.

As for librosa, we unfortunately can't yet build that because it depends on SciPy, which is difficult for other reasons (#15). The current version also depends on LLVM, though it looks this could be avoided by using an older version.

We are definitely considering making our package build system open-source so the community can help with porting packages. Please subscribe to #175 for updates on that.

[liangyue62]

Sorry for asking, but I wonder is librosa available now?
Because I have seen that SciPy is now available. Or is there still have other problems?

[mhsmith]

The current version of librosa (0.6.1) still can't be installed because it depends on numba (#139) and llvmlite. However, you should be able to install the older version 0.4.2, which doesn't have those dependencies. I'm not familiar with librosa so I haven't tested whether it works: please give it a try and report back.

[lorena-garcia-fernandez]

Hi,
I get an error even using the older vesion 0.4.2 of librosa.

I include classpath "com.chaquo.python:gradle:5.0.0" on the project graddle.

This is my app graddle

apply plugin: 'com.android.application'
apply plugin: 'com.chaquo.python'

android {
    compileSdkVersion 27
    buildToolsVersion '28.0.3'
    defaultConfig {
        applicationId [...]
        minSdkVersion 21
        targetSdkVersion 26
        compileSdkVersion 27
        versionCode 1
        versionName "1.0"
        ndk{
            abiFilters "armeabi-v7a"
        }
        python {
            buildPython "C:/Users/[...]/AppData/Local/Programs/Python/Python35/python.exe"
            pip {
                //install "librosa==0.4.2"
                //install "-r", "requeriments.txt"
                install "https://github.com/librosa/librosa/archive/0.4.2.zip"
            }
        }
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    productFlavors {
    }
    repositories {
        maven {
            url 'https://google.bintray.com/tensorflow'
        }
    }
    aaptOptions {
        noCompress "tflite"
        noCompress "lite"
    }
}

dependencies {
    [...]
}

I get the error after sucessfully built librosa:

Successfully built librosa
Installing collected packages: audioread, chaquopy-libgfortran, chaquopy-openblas, numpy, chaquopy-gnustl, scipy, scikit-learn, chaquopy-libpng, six, pyparsing, chaquopy-freetype, pytz, setuptools, kiwisolver, python-dateutil, cycler, matplotlib, joblib, decorator, librosa
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\\Users\\[...]\\AppData\\Local\\Programs\\Python\\Python35\\Lib\\site-packages\\tensorflow\\examples\\android\\LULLAAI\\app\\build\\generated\\python\\requirements\\debug\\armeabi-v7a\\matplotlib\\backends\\web_backend\\jquery\\css\\themes\\base\\images\\ui-bg_diagonals-thick_18_b81900_40x40.png'

Chaquopy: Exit status 1
:app:generateDebugPythonRequirements FAILED

It's a little bit strange. I cleaned the project. I restarted android studio as administrator (I'm developing my app in windows)

Can you help me ? Must I open a new issue?

Thanks

[mhsmith]

This is probably caused by the Windows filename length limit of 260 characters (pypa/pip#6279). Try moving your project to a location with a shorter path.

[lorena-garcia-fernandez]

Yes, this was the problem. I didn't realized. Thanks!

Now, I get a new error, but I think it is a problem with python.
´´´
if (!Python.isStarted()) {
Python.start(new AndroidPlatform(context));
}

py = Python.getInstance();
librosa = py.getModule("librosa");
´´´
When I make a cool to get the librosa module, an exception is thrown:
´´´
com.chaquo.python.PyException: ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.
´´´

Do i make an incorrect call to import the librosa lib, or is a problem of my python instalation? In latter, can you say me where I must ask for them? I can't find a solution googling.

I've only found this link termux/termux-packages#570 where someone say that "Android does not support sem_open() or related functions", and I'm developing an Android App

Thanks in advance.

[mhsmith]

>>> import librosa
/android_asset/chaquopy/requirements-common.zip/joblib/_multiprocessing_helpers.py:38: UserWarning: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770..  joblib will operate in serial mode
Traceback (most recent call last):
  File "stdlib/multiprocessing/synchronize.py", line 29, in <module>
ImportError: cannot import name 'sem_unlink'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  [...]
  File "/android_asset/chaquopy/requirements-common.zip/joblib/externals/loky/backend/__init__.py", line 13, in <module>
    from multiprocessing import synchronize
  File "import.pxi", line 59, in java.chaquopy.import_override
  File "/data/user/0/com.chaquo.python.pkgtest3/files/chaquopy/bootstrap.zip/java/_vendor/six.py", line 686, in reraise
    raise value
  File "import.pxi", line 18, in java.chaquopy.import_override
  File "stdlib/multiprocessing/synchronize.py", line 34, in <module>
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.

[mhsmith]

Despite the message, it appears that the current version of joblib (0.13.0) does not actually manage to fall back to serial mode (joblib/joblib#825).

Try installing an older version by adding the following line in build.gradle, at the top of the pip block:

install "joblib==0.11"

When I did that, I still received some warnings, but the librosa import completed successfully.

[lorena-garcia-fernandez]

Thanks, give some days to check it. On Monday i'll give you some response

[lorena-garcia-fernandez]

Finally, I found the librosa MFCC features extraction programmed on Java, which were the functionality I needed from librosa in my Android app. So, I didn't test whether your last solution could solve my problem. Thanks a lot, I hope it will resolve the problem to other programmers!

[mhsmith]

The current version of librosa can now be installed with Chaquopy 7.0.2. To upgrade, edit your app's top-level build.gradle file and change the version number of com.chaquo.python:gradle to 7.0.2.

Please review the release notes, especially the point about most native packages having been upgraded.

[neur1n]

Hello, I've tried to use Chaquopy 7.0.2 and 7.0.3 but still couldn't get it works. A ModuleNotFoundError: No module named 'librosa' exception was thrown. More information:

• OS: Windows 10
• librosa version: 0.4.2 (according to Please support librosa Library #185)
• joblib version: 0.11.0

[mhsmith]

Please post:

• The full exception trace
• The section of source code which caused it
• The python section of your build.gradle file

By the way, the note in #185 is out of date: you can now use any version of librosa. However, you may still need to use joblib 0.11.0, or the workaround at joblib/joblib#825 (comment).

[neur1n]

I just realize what does "The python section of your build.gradle file" mean, and I did miss this section. Now everything works.

[mhsmith]

I've just learned that although the current version of librosa can be installed with Chaquopy, it still fails to import. This will be fixed in the next version of Chaquopy.

[mhsmith]

This issue is fixed in Chaquopy 8.0.1. If you're using that version or later, there's no longer any need to use old versions of librosa or joblib.

To upgrade, edit your app's top-level build.gradle file and change the version number of com.chaquo.python:gradle.

[HughHMLee]

[Moved to #527]