Problems when building on MacOS, Python3

[JonnyDaenen]

When building using the following command:
pip install https://github.com/LeeKamentsky/python-javabridge/archive/master.zip

I get the errors below. Could you provide a new binary with all the MacOS/python3 fixes? Or any suggestions on how to get it built?

Thanks!

Collecting https://github.com/LeeKamentsky/python-javabridge/archive/master.zip
  Downloading https://github.com/LeeKamentsky/python-javabridge/archive/master.zip (2.3MB)
    100% |████████████████████████████████| 2.4MB 513kB/s
Requirement already satisfied: numpy in /Users/jd/.pyenv/versions/3.5.2/envs/py3dev/lib/python3.5/site-packages (from javabridge==0.0.0)
Installing collected packages: javabridge
  Found existing installation: javabridge 1.0.14
    Uninstalling javabridge-1.0.14:
      Successfully uninstalled javabridge-1.0.14
  Running setup.py install for javabridge ... error
    Complete output from command /Users/jd/.pyenv/versions/3.5.2/envs/py3dev/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-mho32agy-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jd/.pyenv/versions/3.5.2/envs/py3dev/include/site/python3.5/javabridge:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.12-x86_64-3.5
    creating build/lib.macosx-10.12-x86_64-3.5/javabridge
    copying javabridge/__init__.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge
    copying javabridge/jutil.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge
    copying javabridge/locate.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge
    copying javabridge/noseplugin.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge
    copying javabridge/wrappers.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge
    creating build/lib.macosx-10.12-x86_64-3.5/javabridge/tests
    copying javabridge/tests/__init__.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge/tests
    copying javabridge/tests/test_cpython.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge/tests
    copying javabridge/tests/test_javabridge.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge/tests
    copying javabridge/tests/test_jutil.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge/tests
    copying javabridge/tests/test_wrappers.py -> build/lib.macosx-10.12-x86_64-3.5/javabridge/tests
    creating build/lib.macosx-10.12-x86_64-3.5/javabridge/jars
    copying javabridge/jars/rhino-1.7R4.jar -> build/lib.macosx-10.12-x86_64-3.5/javabridge/jars
    running build_ext
    javac -source 6 -target 6 /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java/org/cellprofiler/runnablequeue/RunnableQueue.java
    warning: [options] bootstrap class path not set in conjunction with -source 1.6
    1 warning
    jar cf /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/build/lib.macosx-10.12-x86_64-3.5/javabridge/jars/runnablequeue.jar -C /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java org/cellprofiler/runnablequeue/RunnableQueue$1.class -C /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java org/cellprofiler/runnablequeue/RunnableQueue.class
    javac -source 6 -target 6 /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java/org/cellprofiler/javabridge/test/RealRect.java
    warning: [options] bootstrap class path not set in conjunction with -source 1.6
    1 warning
    jar cf /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/build/lib.macosx-10.12-x86_64-3.5/javabridge/jars/test.jar -C /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java org/cellprofiler/javabridge/test/RealRect.class
    javac -source 6 -target 6 /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java/org/cellprofiler/javabridge/CPython.java /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java/org/cellprofiler/javabridge/CPythonInvocationHandler.java
    warning: [options] bootstrap class path not set in conjunction with -source 1.6
    Note: /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java/org/cellprofiler/javabridge/CPythonInvocationHandler.java uses unchecked or unsafe operations.
    Note: Recompile with -Xlint:unchecked for details.
    1 warning
    jar cf /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/build/lib.macosx-10.12-x86_64-3.5/javabridge/jars/cpython.jar -C /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java org/cellprofiler/javabridge/CPython$StackFrame.class -C /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java org/cellprofiler/javabridge/CPython$WrappedException.class -C /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java org/cellprofiler/javabridge/CPython.class -C /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java org/cellprofiler/javabridge/CPythonInvocationHandler.class -C /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/java org/cellprofiler/javabridge/CPythonInvocationHandler.class
    building 'javabridge._javabridge' extension
    creating build/temp.macosx-10.12-x86_64-3.5
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/jd/.pyenv/versions/3.5.2/envs/py3dev/lib/python3.5/site-packages/numpy/core/include -I/Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/include/darwin -I/Users/jd/.pyenv/versions/3.5.2/envs/py3dev/include -I/Users/jd/.pyenv/versions/3.5.2/include/python3.5m -c _javabridge.c -o build/temp.macosx-10.12-x86_64-3.5/_javabridge.o
    In file included from _javabridge.c:435:
    In file included from /Users/jd/.pyenv/versions/3.5.2/envs/py3dev/lib/python3.5/site-packages/numpy/core/include/numpy/arrayobject.h:4:
    In file included from /Users/jd/.pyenv/versions/3.5.2/envs/py3dev/lib/python3.5/site-packages/numpy/core/include/numpy/ndarrayobject.h:18:
    In file included from /Users/jd/.pyenv/versions/3.5.2/envs/py3dev/lib/python3.5/site-packages/numpy/core/include/numpy/ndarraytypes.h:1788:
    /Users/jd/.pyenv/versions/3.5.2/envs/py3dev/lib/python3.5/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: "Using deprecated NumPy API, disable it by "          "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
    #warning "Using deprecated NumPy API, disable it by " \
     ^
    In file included from _javabridge.c:437:
    /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/include/jni.h:1:1: error: expected identifier or '('
    <U+0008><U+0008>      <U+0008><U+0000><U+0008><U+0003><U+0008><U+0003><U+0008><U+0008>        <U+0008><U+0004><U+0008><U+0008><U+0008><U+0008>-<U+001A><U+0004><U+0000><B0><U+000E><U+0000><DA>7<U+001E><93>חcom.apple.finder8<87>U<U+001C><U+0000><U+0001>       <U+0008><U+0008><U+0008><U+0001><U+0008><U+0008><U+0008> <U+0008><U+0000><U+0008><U+0003><U+0008><U+0003><U+0008><U+0008>        <U+0008><U+0004><U+0008><U+0008><U+0008><U+0008>-<U+001A><U+0004><U+0000><B0>
    ^
...                                                                                                                                                                                                                                                                                                                                                                                     ^
    /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/include/jni.h:2:179: error: source file is not valid UTF-8
    <U+0000><DA>7<U+001E><93>חcom.apple.finder8<87>T<U+001C><U+0000><U+0001>   <U+0008><U+0008><U+0008><U+0001><U+0008><U+0008><U+0008> <U+0008><U+0000><U+0008><U+0003><U+0008><U+0003><U+0008><U+0008>        <U+0008><U+0004><U+0008><U+0008><U+0008><U+0008>-<U+001A><U+0004><U+0000><B0><U+0011><U+0000><DA>7<U+001E><93>חcom.apple.finder8<87>S<U+001C><U+0000><U+0001>       <U+0008><U+0008><U+0008><U+0001><U+0008><U+0008><U+0008> <U+0008><U+0000><U+0008><U+0003><U+0008><U+0003><U+0008><U+0008>        <U+0008><U+0004><U+0008><U+0008><U+0008><U+0008>-<U+001A><U+0004><U+0000><B0><U+001A><U+0000><DA>7<U+001E><93>חcom.apple.finder8<87>R<U+001C><U+0000><U+0001>       <U+0008><U+0008><U+0008><U+0001><U+0008><U+0008><U+0008> <U+0008><U+0000><U+0008><U+0003><U+0008><U+0003><U+0008><U+0008>        <U+0008><U+0004><U+0008><U+0008><U+0008><U+0008>-<U+001A><U+0004><U+0000><B0><U+000F><U+0000><DA>7<U+001E><93>חcom.apple.finderA<87>Q<U+001C><U+0000><U+0001>       <U+0008><U+0008><U+0008><U+0001><U+0008><U+0008><U+0008><U+0008><U+0008><U+0000><U+0008><U+0003><U+0008><U+0003><U+0008><U+0008>        <U+0008><U+0004><U+0008><U+0008><U+0008><U+0008>?<U+001A>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       ^
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    20 warnings and 20 errors generated.
    error: command 'clang' failed with exit status 1

    ----------------------------------------
  Rolling back uninstall of javabridge
Command "/Users/jd/.pyenv/versions/3.5.2/envs/py3dev/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-mho32agy-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jd/.pyenv/versions/3.5.2/envs/py3dev/include/site/python3.5/javabridge" failed with error code 1 in /private/var/folders/h2/3jt9lx8x7wxd0p51hmb9ffv00000gp/T/pip-bj631hie-build/

[LeeKamentsky]

I'd look at /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/include/jni.h
From the log, it looks like the file is corrupt. If so, maybe reinstalling the JDK would work.

[JonnyDaenen]

@LeeKamentsky Thanks for the feedback. Is it possible to provide a more recent build? My main problem is the following error when trying to run the pip package on Mac:

Traceback (most recent call last):
  File "/Users/jd/.pyenv/versions/py3dev/lib/python3.5/site-packages/javabridge/jutil.py", line 276, in start_thread
    vm.create_mac(args, RQCLS, library_path, libjli_path)
  File "_javabridge.pyx", line 650, in _javabridge.JB_VM.create_mac (_javabridge.c:6878)
TypeError: expected bytes, str found

[LeeKamentsky]

Unfortunately, I don't have access to a Mac right now, so I can't build for it.

[JonnyDaenen]

@LeeKamentsky No problem, just tried with newer JDK and it did the trick! jni.h indeed turned out to be corrupted. Thanks! :)

[achillesrasquinha]

You might have to install the bleeding-edge work now. Having this used for my own project. This seemed to work since it has been fixed, not deployed on pip yet.

[dschetel]

Hey,
i also have trouble installing javabridge in macOS (10.12.6)

When I try installing (pip install javabridge or pip install https://github.com/LeeKamentsky/python-javabridge/archive/master.zip), i get the following error:

Collecting https://github.com/LeeKamentsky/python-javabridge/archive/master.zip
  Downloading https://github.com/LeeKamentsky/python-javabridge/archive/master.zip (2.4MB)
    100% |████████████████████████████████| 2.4MB 14.5MB/s 
    Complete output from command python setup.py egg_info:
    Could not find Java JRE compatible with x86_64 architecture
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/4g/4jglxpwd1kz1pxl6040lg6xm0000gn/T/pip-req-build-2_riafj3/setup.py", line 405, in <module>
        ext_modules=ext_modules(),
      File "/private/var/folders/4g/4jglxpwd1kz1pxl6040lg6xm0000gn/T/pip-req-build-2_riafj3/setup.py", line 98, in ext_modules
        raise Exception("JVM not found")
    Exception: JVM not found
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/4g/4jglxpwd1kz1pxl6040lg6xm0000gn/T/pip-req-build-2_riafj3/

The currently installed Java Version is:

java -version
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

Do you have any tips on how to fix this?
Thank you!

[LeeKamentsky]

Hi @dschetel. Can you post the results of running the following from the command-line:

/usr/libexec/java_home --arch x86_64

That should give us a hint whether the problem is with the java_home program or the Java libraries.

[dschetel]

/usr/libexec/java_home --arch x86_64 returns:

/Library/Java/JavaVirtualMachines/jdk-10.0.2.jdk/Contents/Home

Is Java 10 a problem?
But my current system is a virtual environment (using miniconda), in which I installed java 8 - shouldn't this work?

[LeeKamentsky]

I think it won't work... ish. Javabridge will build using JDK 10 and run using JDK 8. The work-around is to define the environment variable, JAVA_HOME, to point to your miniconda-installed version of Java or to not install Java using Anaconda.

[LeeKamentsky]

I think I've reproduced the problem on Travis:
https://travis-ci.org/LeeKamentsky/python-javabridge/jobs/440135023

It looks like Javabridge isn't finding the libraries on JDK 10

[dschetel]

Thank you for your extensive help!
Do you have a recommendation/routine how to set this up?

I tried setting up a virtual environment, installing Java 8, setting the environment variable to the newly installed Java 8 and then to install Javabridge, but I still get error-messages..

[dschetel]

small update:
I used a fresh system and installed Java 8 and Javabridge (pip install javabridge)

/usr/libexec/java_home --arch x86_64
/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home

When executing Python and importing numpy and javabridge i get the following error:

>>> import numpy
>>> import javabridge
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dgs/miniconda3/envs/biochem/lib/python3.6/site-packages/javabridge/__init__.py", line 38, in <module>
    from .jutil import start_vm, kill_vm, vm, activate_awt, deactivate_awt
  File "/Users/dgs/miniconda3/envs/biochem/lib/python3.6/site-packages/javabridge/jutil.py", line 155, in <module>
    import javabridge._javabridge as _javabridge
ImportError: dlopen(/Users/dgs/miniconda3/envs/biochem/lib/python3.6/site-packages/javabridge/_javabridge.cpython-36m-darwin.so, 2): Symbol not found: _kCFAllocatorDefault
  Referenced from: /Users/dgs/miniconda3/envs/biochem/lib/python3.6/site-packages/javabridge/_javabridge.cpython-36m-darwin.so
  Expected in: flat namespace
 in /Users/dgs/miniconda3/envs/biochem/lib/python3.6/site-packages/javabridge/_javabridge.cpython-36m-darwin.so

[LeeKamentsky]

I think you might want to look at #136
I have been trying to get Anaconda and OS/X to work for a good part of the day. I think the problem as it says in the issue is that pythonw needs to be run in order to get a GUI application running. IMHO, a losing battle, I am still not getting it to work.
I would use the system Python and virtualenv to isolate your environment or maybe Homebrew.
https://github.com/LeeKamentsky/python-javabridge/blob/master/docs/installation.rst#macos-x