[Meta] [FEATURE] Add OPENSEARCH_JAVA_HOME env to override JAVA_HOME

[chaitrahegde115]

Coming from here

• Add support for OPENSEARCH_JAVA_HOME in this repo
  [FEATURE] Add OPENSEARCH_JAVA_HOME env to override JAVA_HOME #2001
  [Backport] [1.x] [FEATURE] Add OPENSEARCH_JAVA_HOME env to override JAVA_HOME #2040
  [Backport] [1.2] [FEATURE] Add OPENSEARCH_JAVA_HOME env to override JAVA_HOME (#2040) #2042
• Add support for OPENSEARCH_JAVA_HOME security#1600
• [FEATURE] Add support for OPENSEARCH_JAVA_HOME performance-analyzer#132
• [FEATURE] Add support for OPENSEARCH_JAVA_HOME performance-analyzer-rca#127
• To JDK or not to JDK (and what version) project-website#628
• [FEATURE] Document OPENSEARCH_JAVA_HOME documentation-website#396
  [BUG] OPENSEARCH_JAVA_HOME is incorrect documentation-website#364

Related, opensearch-project/OpenSearch-Dashboards#1219

[anasalkouz]

OpenSearch doesn't use the environment variable OPENSEARCH_JAVA_HOME as mentioned on https://opensearch.org/docs/latest/opensearch/install/compatibility/. Closing this ticket, since we already have another issue opened to fix the documentation.

[dblock]

Should we turn this into a feature request? @chaitrahegde115 is there a reason why you'd want an ENV-set value vs. say replacing binaries in the distribution or consuming a no-JDK min version?

[jcgraybill]

To be clear, OpenSearch does support specifying a JDK with an environment variable, it's just that the environment variable is JAVA_HOME... the documentation is wrong in this case.

[reta]

@dblock @jcgraybill that is correct, but it could be useful for certain cases (migration fe), Elasticsearch has this feature elastic/elasticsearch#68954.

+1 to turn it into feature request, I actually do have an example from the field where it is useful. There are often several JVM colocated apps / services running on the same host which use different JDKs (fe apps go on edge with JDK-17 while services are still conservative on JDK-8/11). In this case, setting JAVA_HOME would not work (for all), but introducing OPENSEARCH_JAVA_HOME would actually make such deployments / migrations much simpler.

[dblock]

I'll reopen as a feature request.

[reta]

@dblock if there are no objections, you could assign it to me, thx!

[dblock]

@chaitrahegde115 original issue was as follows

Describe the bug
I am using Opensearch-1.2.3 tar which ships Java 15 with it. But I want to use Java 11. So I referred Compatibility - OpenSearch documentation which stated to “set OPENSEARCH_JAVA_HOME environment variable to the Java install location” . I tried it but Opensearch installation used bundled jdk-15 but not jdk-11 which I want to use. Logs are as below.

[opensearch-1.2.3]$ export OPENSEARCH_JAVA_HOME=/etc/alternatives/jre_openjdk
[opensearch-1.2.3]$ ./bin/opensearch
[2022-01-07T11:47:12,157][INFO ][o.o.n.Node               ] [chai-node] version[1.2.3], pid[16189], build[tar/8a529d77c7432bc45b005ac1c4ba3b2741b57d4a/2021-12-21T01:36:21.407473Z], OS[Linux/3.10.0-957.10.1.el7.x86_64/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2022-01-07T11:47:12,162][INFO ][o.o.n.Node               ] [chai-node] JVM home [/home/cloud-user/chaitra-opensearch/opensearch-1.2.3/jdk], using bundled JDK [true]
[2022-01-07T11:47:12,162][INFO ][o.o.n.Node               ] [chai-node] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-12173583538575190589, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/home/cloud-user/chaitra-opensearch/opensearch-1.2.3/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/home/cloud-user/chaitra-opensearch/opensearch-1.2.3, -Dopensearch.path.conf=/home/cloud-user/chaitra-opensearch/opensearch-1.2.3/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]

But when I set JAVA_HOME to my java installation directory like below, it picked up the java version which I want to use.

[opensearch-1.2.3]$ export JAVA_HOME=/etc/alternatives/jre_openjdk
[opensearch-1.2.3]$ ./bin/opensearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a futur
[2022-01-07T11:49:17,898][INFO ][o.o.n.Node               ] [chai-node] version[1.2.3], pid[16544], build[tar/8a529d77c7432bc453b2741b57d4a/2021-12-21T01:36:21.407473Z], OS[Linux/3.10.0-957.10.1.el7.x86_64/amd64], JVM[Red Hat, Inc./OpenJDK 64-Bit Server 11.0.13+8-LTS]
[2022-01-07T11:49:17,902][INFO ][o.o.n.Node               ] [chai-node] JVM home [/usr/lib/jvm/java-11-openjdk-11.0.13.0.8-1.el, using bundled JDK [false]
[2022-01-07T11:49:17,902][INFO ][o.o.n.Node               ] [chai-node] JVM arguments [-Xshare:auto, -Dopensearch.networkaddres=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=a.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recyclityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.loers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyO.io.tmpdir=/tmp/opensearch-3512813336349669626, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_eg, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allf=true, -Djava.security.policy=/home/cloud-user/chaitra-opensearch/opensearch-1.2.3/plugins/opensearch-performance-analyzer/pansearch_security.policy, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/home/cloud-user/chaitra-opensearch/opensearDopensearch.path.conf=/home/cloud-user/chaitra-opensearch/opensearch-1.2.3/config, -Dopensearch.distribution.type=tar, -Dopensed_jdk=true]

Hence OPENSEARCH_JAVA_HOME doesn't seem to work.

I turned it into a meta issue for adding support to OPENSEARCH_JAVA_HOME

[reta]

@dblock could you please add this one as well opensearch-project/performance-analyzer-rca#127, thank you

[reta]

@dblock I think we are done with it?