Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support of audioread/librosa package #21

Closed
rjt10 opened this issue Jan 8, 2018 · 17 comments
Closed

support of audioread/librosa package #21

rjt10 opened this issue Jan 8, 2018 · 17 comments
Labels

Comments

@rjt10
Copy link

rjt10 commented Jan 8, 2018

Got error when trying to pip install these two packages with this:

            pip {
                install "six==1.10.0"
                install "audioread==2.1.5"
            }

The error message is:

Collecting audioread==2.1.5

  Could not find a version that satisfies the requirement audioread==2.1.5 (from versions: )
No matching distribution found for audioread==2.1.5 (NOTE: Chaquopy only supports wheels, not sdist packages)
Exit status 

Looking at the past feature requests, I have an impression that the python packages need to be manually ported to be available through Chaquopy, it is the case? If so, I am wondering if there's a way for community to help porting them?

Thanks.

@mhsmith
Copy link
Member

mhsmith commented Jan 9, 2018

Packages which are available on PyPI as pure Python wheels don't need to be rebuilt, but all others do. We have some unreleased scripts to assist in this, which work fine for packages which are pure Python or self-contained native code. However, we can't currently handle packages which depend on external native libraries or executables. In order to use these on Android, we'd also need a procedure for building and packaging the external components themselves.

[EDIT 2018-08: All pure-Python packages can now be installed directly, including sdists. Packages with native components, including external non-Python libraries, still need to be cross-compiled by us and added to the repository.]

This is the case with audioread, which (like many Python audio decoding libraries) depends on FFmpeg (#143). If you only need to read uncompressed audio files, it looks like it'll still work. [EDIT 2018-08: It can now be installed with the current version of Chaquopy.] Otherwise, you'll have to use a different approach. The obvious way would be to use Chaquopy to access the Android MediaCodec API from Python, though that looks pretty unfriendly.

As for librosa, we unfortunately can't yet build that because it depends on SciPy, which is difficult for other reasons (#15). The current version also depends on LLVM, though it looks this could be avoided by using an older version.

We are definitely considering making our package build system open-source so the community can help with porting packages. Please subscribe to #175 for updates on that.

@liangyue62
Copy link

liangyue62 commented Aug 9, 2018

Sorry for asking, but I wonder is librosa available now?
Because I have seen that SciPy is now available. Or is there still have other problems?

@mhsmith
Copy link
Member

mhsmith commented Aug 9, 2018

The current version of librosa (0.6.1) still can't be installed because it depends on numba (#139) and llvmlite. However, you should be able to install the older version 0.4.2, which doesn't have those dependencies. I'm not familiar with librosa so I haven't tested whether it works: please give it a try and report back.

@lorena-garcia-fernandez

Hi,
I get an error even using the older vesion 0.4.2 of librosa.

I include classpath "com.chaquo.python:gradle:5.0.0" on the project graddle.

This is my app graddle

apply plugin: 'com.android.application'
apply plugin: 'com.chaquo.python'

android {
    compileSdkVersion 27
    buildToolsVersion '28.0.3'
    defaultConfig {
        applicationId [...]
        minSdkVersion 21
        targetSdkVersion 26
        compileSdkVersion 27
        versionCode 1
        versionName "1.0"
        ndk{
            abiFilters "armeabi-v7a"
        }
        python {
            buildPython "C:/Users/[...]/AppData/Local/Programs/Python/Python35/python.exe"
            pip {
                //install "librosa==0.4.2"
                //install "-r", "requeriments.txt"
                install "https://github.com/librosa/librosa/archive/0.4.2.zip"
            }
        }
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    productFlavors {
    }
    repositories {
        maven {
            url 'https://google.bintray.com/tensorflow'
        }
    }
    aaptOptions {
        noCompress "tflite"
        noCompress "lite"
    }
}

dependencies {
    [...]
}

I get the error after sucessfully built librosa:

Successfully built librosa
Installing collected packages: audioread, chaquopy-libgfortran, chaquopy-openblas, numpy, chaquopy-gnustl, scipy, scikit-learn, chaquopy-libpng, six, pyparsing, chaquopy-freetype, pytz, setuptools, kiwisolver, python-dateutil, cycler, matplotlib, joblib, decorator, librosa
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\\Users\\[...]\\AppData\\Local\\Programs\\Python\\Python35\\Lib\\site-packages\\tensorflow\\examples\\android\\LULLAAI\\app\\build\\generated\\python\\requirements\\debug\\armeabi-v7a\\matplotlib\\backends\\web_backend\\jquery\\css\\themes\\base\\images\\ui-bg_diagonals-thick_18_b81900_40x40.png'

Chaquopy: Exit status 1
:app:generateDebugPythonRequirements FAILED

It's a little bit strange. I cleaned the project. I restarted android studio as administrator (I'm developing my app in windows)

Can you help me ? Must I open a new issue?

Thanks

@mhsmith
Copy link
Member

mhsmith commented Dec 18, 2018

This is probably caused by the Windows filename length limit of 260 characters (pypa/pip#6279). Try moving your project to a location with a shorter path.

@lorena-garcia-fernandez
Copy link

lorena-garcia-fernandez commented Dec 18, 2018

Yes, this was the problem. I didn't realized. Thanks!

Now, I get a new error, but I think it is a problem with python.
´´´
if (!Python.isStarted()) {
Python.start(new AndroidPlatform(context));
}

py = Python.getInstance();
librosa = py.getModule("librosa");
´´´
When I make a cool to get the librosa module, an exception is thrown:
´´´
com.chaquo.python.PyException: ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.
´´´

Do i make an incorrect call to import the librosa lib, or is a problem of my python instalation? In latter, can you say me where I must ask for them? I can't find a solution googling.

I've only found this link termux/termux-packages#570 where someone say that "Android does not support sem_open() or related functions", and I'm developing an Android App

Thanks in advance.

@mhsmith
Copy link
Member

mhsmith commented Dec 19, 2018

>>> import librosa
/android_asset/chaquopy/requirements-common.zip/joblib/_multiprocessing_helpers.py:38: UserWarning: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770..  joblib will operate in serial mode
Traceback (most recent call last):
  File "stdlib/multiprocessing/synchronize.py", line 29, in <module>
ImportError: cannot import name 'sem_unlink'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  [...]
  File "/android_asset/chaquopy/requirements-common.zip/joblib/externals/loky/backend/__init__.py", line 13, in <module>
    from multiprocessing import synchronize
  File "import.pxi", line 59, in java.chaquopy.import_override
  File "/data/user/0/com.chaquo.python.pkgtest3/files/chaquopy/bootstrap.zip/java/_vendor/six.py", line 686, in reraise
    raise value
  File "import.pxi", line 18, in java.chaquopy.import_override
  File "stdlib/multiprocessing/synchronize.py", line 34, in <module>
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.

@mhsmith
Copy link
Member

mhsmith commented Dec 19, 2018

Despite the message, it appears that the current version of joblib (0.13.0) does not actually manage to fall back to serial mode (joblib/joblib#825).

Try installing an older version by adding the following line in build.gradle, at the top of the pip block:

install "joblib==0.11"

When I did that, I still received some warnings, but the librosa import completed successfully.

@lorena-garcia-fernandez

Thanks, give some days to check it. On Monday i'll give you some response

@lorena-garcia-fernandez

Finally, I found the librosa MFCC features extraction programmed on Java, which were the functionality I needed from librosa in my Android app. So, I didn't test whether your last solution could solve my problem. Thanks a lot, I hope it will resolve the problem to other programmers!

@mhsmith
Copy link
Member

mhsmith commented Mar 5, 2020

The current version of librosa can now be installed with Chaquopy 7.0.2. To upgrade, edit your app's top-level build.gradle file and change the version number of com.chaquo.python:gradle to 7.0.2.

Please review the release notes, especially the point about most native packages having been upgraded.

@Neur1n
Copy link

Neur1n commented Jun 3, 2020

Hello, I've tried to use Chaquopy 7.0.2 and 7.0.3 but still couldn't get it works. A ModuleNotFoundError: No module named 'librosa' exception was thrown. More information:

@mhsmith
Copy link
Member

mhsmith commented Jun 3, 2020

Please post:

  • The full exception trace
  • The section of source code which caused it
  • The python section of your build.gradle file

By the way, the note in #185 is out of date: you can now use any version of librosa. However, you may still need to use joblib 0.11.0, or the workaround at joblib/joblib#825 (comment).

@Neur1n

This comment has been minimized.

@Neur1n
Copy link

Neur1n commented Jun 4, 2020

I just realize what does "The python section of your build.gradle file" mean, and I did miss this section. Now everything works.

@mhsmith
Copy link
Member

mhsmith commented Jul 25, 2020

I've just learned that although the current version of librosa can be installed with Chaquopy, it still fails to import. This will be fixed in the next version of Chaquopy.

@mhsmith mhsmith reopened this Jul 25, 2020
@mhsmith
Copy link
Member

mhsmith commented Jul 28, 2020

This issue is fixed in Chaquopy 8.0.1. If you're using that version or later, there's no longer any need to use old versions of librosa or joblib.

To upgrade, edit your app's top-level build.gradle file and change the version number of com.chaquo.python:gradle.

@mhsmith mhsmith closed this as completed Jul 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants