Skip to content
This repository has been archived by the owner on Mar 8, 2018. It is now read-only.

Home

Jump to bottom
AhmadFCheema edited this page Aug 15, 2017 · 2 revisions

Welcome to the openshift-elasticsearch-cartridge wiki!

Dev notes

Implementation on MediaWiki

OpenShift

Cartridges

  • Elasticsearch 5.2.2

  • Elasticsearch 5.3.2

  • Elasticsearch 1.5.0 (with Kibana dashboard at the frontend)

    • Didn't know how to connect the Elasticsearch part of the installation with MediaWiki.

  • Elasticsearch 1.7.1

    • Tested to be working. The original search index was created on MediaWiki v1.27 and the search updates seem to have been working fine even when upgraded to MediaWiki v1.28.

  • Elasticsearch 2.3.3 (with password authentication through Nginx at the frontend)

    • There doesn't appear to be any native way to send password authentication through the MediaWiki setup. Otherwise the setup appears to be working.
    • Password protection can be removed by removing the following lines from /nginx.conf.erb file:
auth_basic "closed api";
auth_basic_user_file <%= ENV['OPENSHIFT_REPO_DIR'] %>htpasswd;
  • Note that commenting out these lines through syntax "<!--" will cause deployment of the application in OpenShift to fail with the following error thrown out: [emerg] 409837#0: unknown directive "<!--" in .../app-root/runtime/repo/nginx.conf:6

OpenShift DIY app

The following is the relevant information noted down while attempting to deploy Elasticsearch on OpenShift through OpenShift's DIY app. The DIY app was abandoned because it didn't allow app scaling. This is only recorded for archival purposes and possible benefit to readers. The steps below might be incomplete and at times not self-explanatory.

  • Create a DIY app on OpenShift.
  • SSH into the created app.
  • nano app-root/data/elasticsearch/config/elasticsearch.yml,insert the following at the end,
  cluster.name: elasticsearch-${OPENSHIFT_APP_UUID}
  node.name: ${OPENSHIFT_GEAR_UUID}
  path.data: ${OPENSHIFT_DATA_DIR}elasticsearch/data
  # path.work: ${OPENSHIFT_TMP_DIR}elasticsearch
  path.logs: ${OPENSHIFT_DIY_LOG_DIR}elasticsearch_logs
  network.host: ${OPENSHIFT_DIY_IP}
  # transport.tcp.port: 3306
  http.port: ${OPENSHIFT_DIY_PORT}
  # discovery.zen.ping.multicast.enabled: false
  discovery.zen.ping.unicast.hosts: []
  • nano app-root/runtime/repo/.openshift/action_hooks/start
  #!/bin/bash
  # The logic to start up your application should be put in this
  # script. The application will work only if it binds to
  # $OPENSHIFT_DIY_IP:8080
  nohup $OPENSHIFT_DATA_DIR/elasticsearch/bin/elasticsearch > $OPENSHIFT_DIY_LOG_DIR/server.log 2>&1 &
  • nano app-root/runtime/repo/.openshift/action_hooks/stop
#!/bin/bash
source $OPENSHIFT_CARTRIDGE_SDK_BASH
# The logic to stop your application should be put in this script.
if [ -z "$(ps -ef | grep elasticsearch | grep -v grep)" ]
  then
    client_result "Application is already stopped"
  else
    kill `ps -ef | grep elasticsearch | grep -v grep | awk '{ print $2 }'` > /dev/null 2>&1
fi

  • Run command, $OPENSHIFT_DATA_DIR/elasticsearch/bin/elasticsearch -f and check whether Elasticsearch runs or not.

  • Relevant information and files,

    • $OPENSHIFT_DATA_DIR = app-root/data/
    • $OPENSHIFT_REPO_DIR = app-root/runtime/repo/diy/
    • Log file: app-root/data/elasticsearch/logs/elasticsearch.log
    • nano $OPENSHIFT_DATA_DIR/elasticsearch/bin/elasticsearch.in.sh
    • nano $OPENSHIFT_DATA_DIR/elasticsearch/config/jvm.options

  • Common commands run during testing,

    cd $OPENSHIFT_DATA_DIR
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.2.tar.gz
tar xf elasticsearch-5.3.2.tar.gz; rm elasticsearch-5.3.2.tar.gz; mv elasticsearch-5.3.2/ elasticsearch
cd elasticsearch

cd ~
nano app-root/data/elasticsearch/config/elasticsearch.yml
nano app-root/data/elasticsearch/config/jvm.options
export JAVA_HOME=/etc/alternatives/java_sdk_1.8.0; export PATH=$JAVA_HOME/bin:$PATH
$OPENSHIFT_DATA_DIR/elasticsearch/bin/elasticsearch -f

Errors

export JAVA_HOME=/etc/alternatives/java_sdk_1.8.0; export PATH=$JAVA_HOME/bin:$PATH
  • Error: Failed to execute: 'control start' - Could be due to a syntax mistake.
  • Error: bash: .../app-root/runtime/repo//diy/elasticsearch/bin/elasticsearch: Permission denied
    • Had to change from using the $OPENSHIFT_REPO_DIR to $OPENSHIFT_DATA_DIR.
Elasticsearch 5.x related
  • Limiting JVM heap size (error only occurred for the Elasticsearch 5.x versions)
    • Error: closed by remote host. - probably a resources used more than maximum allowed issue (1), (2)
      • nano app-root/data/elasticsearch/config/jvm.options
      • Change -Xms2g to -Xms256mXms306m & -Xmx2g to -Xmx256mXmx306m. 306m was used because when unsalted's branch method for calculating half gear memory was tried, the 306m value was given.
  • Warning: ... seccomp unavailable... - seccomp does not appear to be absolutely necessary
  • Warning: max file descriptors [16384] for elasticsearch process is too low, increase to at least [65536]
  • Warning: max number of threads [350] for user [59775d822d5271aa8700006d] is too low, increase to at least [2048]
  • Warning: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
  • Warning: system call filters failed to install... - System call filter check (Security risk), don't know how to resolve this for the present use-case.

Relevant OpenShift CLI commands

# To clone the repository locally, use the SSH URL located on an application's console page on OpenShift, under the heading Source Code.
git clone <SOURCE_CODE_SSH_URL>
git remote set-url origin <SOURCE_CODE_SSH_URL>
git push --set-upstream origin master
git add .
git commit -am "Some message"
git push
# To SSH into an application
ssh <SSH_URL>
# Git push to OpenShift without automatically deploying the application
rhc app-configure <APP-NAME> --no-auto-deploy
# To manually deploy the application again
rhc deploy <GIT_COMMIT_ID>
# To get commit ID
git rev-parse HEAD
# To make application auto-deploy again
rhc app-configure <APP-NAME> --auto-deploy
# Check disk usage for gears
rhc app show <APP-NAME> --gears quota
# To see all gears and their SSH URLs
rhc app show <APP-NAME> --gears
# To see what directories and files are taking up the most space, in SSH
du -h * | sort -rh | head -50
# To see inode quota, in SSH
quota -s
# Clean up the application’s logs, tmp directories, and tidy up the git repo
rhc app-tidy <APP-NAME>
# To download a file from OpenShift
rhc scp <APP-NAME> download . <FILE_ABSOLUTE_PATH>
# File path can be obtained by running the pwd command through SSH
# To delete app: rhc app-delete <APP-NAME>
rhc app-delete <APP-NAME> --confirm

References

Clone this wiki locally