Skip to content
This repository has been archived by the owner on Mar 8, 2018. It is now read-only.

Elasticsearch 5.2.2 Cartridge (Dev notes)

Jump to bottom
AhmadFCheema edited this page Aug 14, 2017 · 2 revisions

Changelog

The following represents the code changes that were made from rbower's OpenShift ElasticSearch cartridge for ES v1.71, v2.3.3 (employing @unsalted's modifications) and ES core v5.2.2.

  • elasticsearch/bin/install

    • Disregarded @unsalted's commit to set "ES_HEAP_SIZE to 1/2 gear space" as it threw out the error: ... encountered environment variables that are no longer supported ... Use jvm.options or ES_JAVA_OPTS to configure the JVM...

  • elasticsearch/bin/control

    • Used @unsalted's updated version. Corrected an oversight which left callback _install_plugins unchanged.
    • Removed function _es_copy_config and its callbacks, as the directory structure had already been manually changed to compensate for the change of conf to hard-coded config.
    • Needed to add additional lines to direct towards correct Java version and give permission to run the plugin installation script (bin/elasticsearch-plugin).
    • Had to change PLUGIN_CMD, probably because path.plugins setting had been removed. Also, had to change old_plugins declaration to make the code work.
    • Additionally, encountered a bug while trying to download a plugin through a URL which contained an = sign. The _es_install_plugins function was using the cut command with = as a delimiter to separate a plugin name and its download URL from plugins.txt. Changed the code so as to include the URL portion even after the delimiter character = appeared in the URL (such as in this one). Another possible solution would've been to include an extraordinary string of characters that would have been unlikely to appear in a legitimate URL.

  • elasticsearch/matadata/managed_files.yml - Used @unsalted's updated version.

  • elasticsearch/template/plugins.txt

    • Completely updated the template by replacing comments regarding deprecated plugins (such as mobz/elasticsearch-head) with newer plugins.

  • elasticsearch/usr/bin/elasticsearch

    • Replaced this with the updated version from Elasticsearch core and tried to keep it unchanged.
    • The rbower3 version differed from Elasticsearch core at code related to Maven (no longer being used), path.conf and export HOSTNAME. These changes appeared to no longer be needed.

  • elasticsearch/usr/bin/elasticsearch.in.sh

    • Environment variables $ES_MIN_MEM, $ES_MAX_MEM, $ES_HEAP_SIZE, $ES_HEAP_NEWSIZE, $ES_DIRECT_SIZE, $ES_USE_IPV4 are no longer supported.
    • Aside from UseParNewGC and loggc settings, all other elasticsearch.in.sh settings are now transferred to elasticsearch/usr/config/jvm.options.
    • These UseParNewGC and loggc were not further manually added to the new jvm.options because although they had been present in Elasticsearch core v1.7 and v2.3.3 versions of file elasticsearch.in.sh, but they were absent from its corresponding Elasticsearch core v5.2.2 version (plus I didn't really know their exact purpose either).
    • OpenShift by default does not direct to the required Java version (i.e. Java 8), therefore, for the purpose of directing Elasticsearch to the correct Java version, appended the following line:
    export JAVA_HOME=/etc/alternatives/java_sdk_1.8.0; export PATH=$JAVA_HOME/bin:$PATH

  • elasticsearch/usr/bin/elasticsearch-systemd-pre-exec - Added.

  • elasticsearch/usr/bin/elasticsearch-translog

    • Added, apparently, variables in log4j2.properties do not resolve without this file.

  • elasticsearch/usr/bin/plugin

    • File plugin appears to have been updated to elasticsearch-plugin in ES core, thus it was replaced.

  • elasticsearch/conf

    • The $ES_JVM_OPTIONS (Path to file containing JVM options) was hard-coded into bin/elasticsearch in Elasticsearch core, so instead of changing it from there, changed the directory name and position to the one provided in Elasticsearch core (i.e. from /conf to usr/config). This hard-coding gets removed and replaced by $CONF_DIR variable in later 6.x branches of ES.

  • elasticsearch/usr/config/elasticsearch.yml

    • Removed network.publish_host - Error: Could not resolve placeholder 'PUBLISH_HOST'

    • Removed path.plugins - Error: ... unknown setting [path.plugins] please check that any required plugins are installed, or check the breaking changes documentation for removed settings...

    • Removed discovery.zen.ping.multicast.enabled - Error: ... unknown setting [path.plugins] please check that any required plugins are installed, or check the breaking changes documentation for removed settings...

    • Removed marvel.agent.enabled, marvel.agent.exporter.es.hosts, marvel.agent.interval - Error: ... unknown setting [path.plugins] please check that any required plugins are installed, or check the breaking changes documentation for removed settings...

    • The new elasticsearch.yml file did not by default include commented out lines for network.bind_host and transport.publish_port settings. Elasticsearch appeared to be working fine without using the transport.publish_port setting, however, removing the network.bind_host settings threw out the error: ... BindTransportException[Failed to bind to [9300-9400]]; nested: SocketException[Permission denied];...

    • According to @unsalted's commit,

      • Set, bootstrap.mlockall: true (which threw out the error ... unknown setting [bootstrap.mlockall]..., so updated it to, bootstrap.memory_lock: true). This probably didn't have any effect because the following error started appearing: Unable to lock JVM Memory: error=12, reason=Cannot allocate memory ... This can result in part of the JVM being swapped out...
      • network.publish_host, transport.publish_port and discovery.zen.ping.multicast.enabled, and settings in sections Marvel Logging and Security were not added as their commented out configurations did not exist in Ealsticsearch 5.2.2 core.

    • Final configurations are as follows:

    cluster.name: elasticsearch-${OPENSHIFT_APP_UUID}
node.name: ${OPENSHIFT_GEAR_UUID}
path.data: ${OPENSHIFT_DATA_DIR}elasticsearch/data
path.logs: ${OPENSHIFT_ELASTICSEARCH_DIR}/logs
network.bind_host: ${OPENSHIFT_ELASTICSEARCH_IP}
discovery.zen.ping.unicast.hosts: ${OPENSHIFT_ELASTICSEARCH_CLUSTER}
transport.publish_port: ${OPENSHIFT_ELASTICSEARCH_TRANSPORT_PROXY_PORT}

  • elasticsearch/usr/config/jvm.options

    • Added and because of limited allowed resources, initial and maximum heap space sizes were reduced to half of gear size (i.e. 306m). Change them as required.

  • elasticsearch/usr/config/logging.yml

    • Removed old logging configuration file and replaced it by the new log4j2.properties, now being used in ES core.

  • elasticsearch/usr/modules

    • Added, otherwise received the Unsupported transport.type [] error. Elasticsearch used to work previously without this directory.

  • elasticsearch/usr/NOTICE.txt - Updated (presence of NOTICE.txt is a requirement of Apache License, Version 2.0 Section 4d).

  • README.md - For the URL used in rhc app create command commit ID was also included. This was done to direct the Cartridge Reflector app towards the correct project branch. Otherwise, by default the master branch gets used to create the app on OpenShift.

so as to provide a static link to the cartridge, otherwise

Errors and Warnings

  • Creating a non-scalable app, for some reason, caused the cartridge to fail running and for the desktop client to not even be able to connect to the app through SSH.

  • Error: Could not resolve placeholder 'PUBLISH_HOST'

    • Removed the line from elasticsearch.yml
    • Originally was getting the error: /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:69:in `abort': can't convert OptionParser::InvalidOption into String. Got this error when trying to start the app by gear start -f through SSH. Spent a lot of time searching online for a solution to this issue, but couldn't find any relevant reference.

  • Error: Likely root cause: mapping values are not allowed here.

    • In elasticsearch.yml when an empty space was left in the beginning of a line behind some code.

  • Error: Classpath should not contain empty elements! (outdated shell script from a previous version?)

    • Replaced old elasticsearch.in.sh file with the new updated one.

  • Error: ...org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Unsupported transport.type []

    • This occurs when /modules directory is absent.

  • Error: ...org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks ... maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?...

    • Apparently, due to an 'orphaned' Elasticsearch process which could happen when another Elasticsearch process is already running in the background.

  • Error: Unable to create file ${sys:es.logs.base_path}/${sys:es.logs.cluster_name}.log java.io.IOException: No such file or directory

    • In the absence of elasticsearch-translog (and possibly elasticsearch-systemd-pre-exec too) variables ${sys:es.logs.base_path}, ${sys:es.logs.cluster_name}, ${sys:es.logs.node_name} in file log4j2.properties, did not resolve to their respective values and were getting used as is, which created "No such file or directory" errors.

  • Error: unable to install syscall filter ... seccomp unavailable...

  • Error: The provided downloadable cartridge '...' cannot be loaded: Cartridge-Vendor 'AhmadFCheema' does not match pattern /\Aa-z0-9\z/.

    • Apparently, allowed regex for cartridge vendor name, doesn't allow capital letters.

  • Error: ... elasticsearch/usr/bin/elasticsearch-plugin: Permission denied

    • Run command: chmod 700 elasticsearch/usr/bin/elasticsearch-plugin

  • Warning: ... ignoring unsupported logging configuration file [.../elasticsearch/usr/config/logging.yml], logging is configured via [.../elasticsearch/usr/config/log4j2.properties]

  • Warning: ... max file descriptors [16384] for elasticsearch process is too low, increase to at least [65536]...

  • Warning: ... max number of threads [350] for user [59775d822d5271aa8700006d] is too low, increase to at least [2048]...

  • Warning: ... max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]...

  • Warning: ... system call filters failed to install...

Clone this wiki locally