mlockall fails on noexec /tmp

[jimmyjones2]

I've been debugging a system where I couldn't get memory locking working. The upshot is JNA extracts out the relevant native library to /tmp and trys to load it, however if you follow security best practice and disable execute in /tmp (noexec mount option), this fails silently. The JNA documentation recommends you extract out the libraries on systems with additional security constraints; in fact the JNA package that ships with RHEL puts the relevant library in /usr/lib64/jna.

I see two potential solutions:

• src/main/java/org/elasticsearch/common/jna/CLibrary.java - log with warning on UnsatisfiedLinkError (rather than debug), and hint at noexec being a potential issue
• extract out the various libjnidispatch.so versions into the elasticsearch lib directory and get JNA to load from there

[hhoffstaette]

I'm very sympathetic to not have JNA unpack, but unfortunately this has several implications for packaging and deployment. However I just looked into this and found that while our current version of JNA does not support a custom directoy, the latest version does (see https://github.com/twall/jna/blob/master/src/com/sun/jna/Native.java#L1014), so that might be a way forward.

I'm not the biggest fan of simply logging the failure since that does not provide an actual solution to the problem and will only confuse too many people even more (who will then ask about how to turn off noexec).

In the meantime you can try:

• set the java.io.tempdir system property to a user-writable directory (maybe ES_HOME/tmp or somesuch)
• sysctl vm.swappiness=0 with a recent kernel. If you want to know why this has the same effect and is in fact less harmful to the system than mlockall, see:
  http://www.quora.com/Linux/Why-does-Linux-swap-out-pages-when-I-have-many-pages-cached-and-vm-swappiness-is-set-to-0-Shouldnt-cached-pages-get-resized-and-no-swapping-should-occur

Hope this helps for now.

[jimmyjones2]

Hey, thanks for your response. I agree logging isn't ideal, but in its defence:

• warn if mlockall is not working once enabled in settings (instead of debug logging) #1194 already converted one class of mlockall failing to warn, but not this case (not sure why)
• Not many people have noexec /tmp, and those who do generally know they have, so I don't think it should confuse too many people

[hhoffstaette]

Please have a look at https://github.com/hhoffstaette/elasticsearch/commit/3cb270cf4f7e2326eb7e924275b5187d9b140e09 and let me know if the message is clear enough.

[jimmyjones2]

LGTM, thanks Holger!