Update to Java 11 breaks Pyserini

[chriskamphuis]

When updating to Java 11 the path the JRE is not correctly configured for latest version of pyjnius that is installed with pip. This can be fixed by cloning the latest version from github and installing that version, but I assume this undesirable.

[lintool]

hey @chriskamphuis any idea why src/test/python/python_interface.py didn't fail then? it's part of Travis CI definition...

[chriskamphuis]

no clue .... it did not work for me locally because of this, so I assumed it was a problem in general, maybe it was my system specifically

[chriskamphuis]

btw your Travis uses Python 2. Dont think that will be the reason, but might be nice to change it to Python 3.

[lintool]

A bit more detail on this; I've replicated this bug. On macOS 10.14.6:

(python36) iMac-Pro:anserini jimmylin$ conda list | grep pyjnius
pyjnius                   1.2.0                     <pip>

The error I get is:

>>> from pyserini.search import pysearch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "./src/main/python/pyserini/search/pysearch.py", line 21, in <module>
    from ..pyclass import JSearcher, JString, JArrayList
  File "./src/main/python/pyserini/pyclass.py", line 28, in <module>
    from jnius import autoclass, cast
  File "/anaconda3/envs/python36/lib/python3.6/site-packages/jnius/__init__.py", line 13, in <module>
    from .reflect import *  # noqa
  File "/anaconda3/envs/python36/lib/python3.6/site-packages/jnius/reflect.py", line 15, in <module>
    class Class(with_metaclass(MetaJavaClass, JavaClass)):
  File "/anaconda3/envs/python36/lib/python3.6/site-packages/six.py", line 827, in __new__
    return meta(name, bases, d)
  File "jnius/jnius_export_class.pxi", line 114, in jnius.MetaJavaClass.__new__
  File "jnius/jnius_export_class.pxi", line 164, in jnius.MetaJavaClass.resolve_class
  File "jnius/jnius_env.pxi", line 11, in jnius.get_jnienv
  File "jnius/jnius_jvm_dlopen.pxi", line 90, in jnius.get_platform_jnienv
  File "jnius/jnius_jvm_dlopen.pxi", line 59, in jnius.create_jnienv
SystemError: Error calling dlopen(b'/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib': b'dlopen(/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib, 10): image not found'

Issue is documented here.

Indeed, the lib has moved:

$ ls /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib 
/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib

So a (janky) symlink will fix the issue:

$ sudo mkdir -p /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/

$ ls /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib 
/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib

$ sudo ln -s /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib

[lintool]

The colab demo doesn't seem to have this issue: https://colab.research.google.com/drive/1GOCyWyYW_fwJMKy5FVWdYHEC2IlvpqZN

Edit: nevermind, above colab notebook doesn't use pyjnius.

[lintool]

I just noticed, @Impavidity and @tuzhucheng mentioned the same issue (but different OS?) in #805. Unclear why colab works though...?

[lintool]

Colab demo illustrating issue and fix: https://colab.research.google.com/drive/1r1pRq_BfWS486kg2qwVH5iBfbhK_GPCg

[lintool]

Issue and resolution are documented in https://github.com/castorini/pyserini
Closing issue.