Elasticsearch 6+ packages are missing libjnidispatch.so which means only able to bind ES to localhost.

[gavinwill]

Hi.

Elasticsearch 6.1 package will by default try to bind to localhost unless the elasticsearch.yml file is modified. For testing and single node "cluster" this is fine. However when using a multi node cluster we naturally need for Elasticsearch to bind to an address other than localhost to allow inter node communication.

When changing the elasticsearch.yml file and restarting the service we see the errors in the logs:

unable to load JNA native support library, native methods will be disabled.

Elasticsearch service will not start up unless we change the config back to bind to localhost.

With the Elasticsearch 6.1 package there is a jar supplied jna-4.4.0-1.jar locate at /opt/local/lib/elasticsearch/lib which does not contain the required files for smartos and JNA to function (and as such to enable ES to bind to anything other than localhost).

Looking at the contents of JNA we can see the following:

jar tf jna-4.4.0-1.jar | grep lib
com/sun/jna/darwin/libjnidispatch.jnilib
com/sun/jna/linux-aarch64/libjnidispatch.so
com/sun/jna/linux-x86-64/libjnidispatch.so

We can manually update this jna-4.4.0-1.orig.jar file by extracting the contents, obtaining a prebuilt binary of the libjnidispatch.so for sunos and bundling the new jar together.

• Extract the jar: jar xf jna-4.4.0-1.jar

• Obtain prebuilt binary for sunos / smartos via: https://github.com/java-native-access/jna/blob/master/lib/native/sunos-x86-64.jar

• Extract sunos-x86-64.jar jar file to obtain libjnidispatch.so

• Move libjnidispatch.so into a new folder under com/sun/jna/ called sunos-x86-64

• Bundle the jar up again with jar cf jna-4.4.0-1.jar com/

• Sanity check the new jar to see the file is present in the new jar ok:

jar tf jna-4.4.0-1.jar | grep lib
com/sun/jna/darwin/libjnidispatch.jnilib
com/sun/jna/linux-x86-64/libjnidispatch.so
com/sun/jna/sunos-x86-64/libjnidispatch.so
com/sun/jna/linux-aarch64/libjnidispatch.so

• Move this new jna-4.4.0-1.jar back into the Elasticsearch default install directory at /opt/local/lib/elasticsearch/lib

At this point if we change elasticsearch config to listen to anything other that localhost the service will start up fine and inter node comms between nodes in an Elasticsearch cluster will function fine.

curl 10.24.100.189:9200

{
  "name" : "estest",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "E78fmJ9_TR63_8DicQNTGw",
  "version" : {
    "number" : "6.1.0"
........

We see the same problem in the trunk version of Elasticsearch version 6.2.2 but with a slight variation. We see that sunos is included in the jar file but x86 is not.

jar tf jna-4.5.1.jar | grep sunos
com/sun/jna/sunos-sparcv9/
com/sun/jna/sunos-sparcv9/libjnidispatch.so
com/sun/jna/sunos-sparc/
com/sun/jna/sunos-sparc/libjnidispatch.so

[mamash]

Thanks for the report. I'll have the elasticsearch package pull the sources for JNA and build the lib file from source rather than fetch it.

[gavinwill]

Point to note also if needing to bind to other than localhost is that Elasticsearch requires java options to be the same. This is part of the checks ES does and as such need to adjust the service manifest from :

  <propval name='min_heap' type='astring' value='256m' />
  <propval name='max_heap' type='astring' value='1g' />

to:

  <propval name='min_heap' type='astring' value='1g' />
  <propval name='max_heap' type='astring' value='1g' />

This is naturally a configurable the user can set. Its not a bug but just including it here for reference.

[mamash]

Thanks for the heads up. The docs suggest the heap size should be uniform for best performance too, so I'll set both values to 1g now.

[mamash]

I've backported this feature to 2017Q4.