Deploying openLooKeng Automatically

In addition to the manual deployment of openLooKeng Sever, you can follow below guide to complete deployment faster and easier. The script is friendly to most of linux OS. However, to Ubuntu, you need to manually install the following dependencies:

sshpass1.06 or above

Deploying openLooKeng on a Single Node

Execute the command below can help you download the necessary packages and deploy openLooKeng server in one-click:

bash <(wget -qO- https://download.openlookeng.io/install.sh)

or:

wget -O - https://download.openlookeng.io/install.sh | bash

Normally, you don't need to do anything, except for the installation to complete. It will automatically start the service.

Execute the command below to stop openLooKeng service.:

/opt/openlookeng/bin/stop.sh

Execute the command below to start openLooKeng Command Line client:

/opt/openlookeng/bin/openlk-cli

Deploying openLooKeng to Cluster online

Execute the command below to install openLooKeng cluster:

bash <(wget -qO- https://download.openlookeng.io/install.sh) -m

or:

bash <(wget -qO- https://download.openlookeng.io/install.sh) --multi-node

First of all, this command will download scripts and packages required by openLooKeng service. After the download is completed, it will check whether the dependent packages expect and sshpass are installed. If not, those dependencies will be installed automatically.

Besides, jdk version is required to be greater than 1.8.0_151. If not, jdk1.8.0_201 will be installed in the cluster. It is recommended to manually install these dependencies before installing openLooKeng service.

Secondly, the script will download openLooKeng-server tarball and copy that tarball to all the nodes in the cluster. Then install the openLooKeng-server by using this tarball.

Lastly, the script will setup openLooKeng server with the standard configurations, including configurations for JVM, Node and also for build-in catalogs like tpch, tpcds and memory connector.

By design, the script will check if there are existing configuration under directory: /home/openlkadmin/.openlkadmin/cluster_node_info

If this file is missing, the install script will ask user to input the nodes information.

Optionally, you can add user openlkadmin and create a file /home/openlkadmin/.openlkconf/cluster_node_info.

In the cluster_node_info, you should list appropriate values for your cluster.

Please refer to the template below, and replace the variables denoted with brackets <> with actual values.

COORDINATOR_IP=<master_node_ipaddress>
WORKER_NODES=<ip_address_1>[,<ip_address_2>,...,<ip_address_n>]

The general configurations for openLooKeng's coordinator, workers are taken from the configuration file /home/openlkadmin/.openlkadmin/cluster\_config\_info and configurations for connectors are taken from the directory /home/openlkadmin/.openlkadmin/catalog respectively. If these directories or any required configuration files are absent during the deploy script running, default configuration files will be generated automatically and deployed to all nodes.

Which means, alternatively, you can add those configuration files before running this deploy script, if you want to customized the deployment.

If above process all succeed, the deploy script will automatically start the openLooKeng Service for you. Execute the command below to stop openLooKeng service.:

/opt/openlookeng/bin/stop.sh

Execute the command below to start openLooKeng Command Line client.:

/opt/openlookeng/bin/openlk-cli

Tips:

If you are going to deploy openLooKeng on a big cluster with lots of nodes, instead of inputting the nodes' IP address one by one. It is better to prepare a file containing all nodes' IP address then pass this file as parameter to the installation script. Here is the command:

bash <(wget -qO- https://download.openlookeng.io/install.sh) -f <cluster_node_info_path> or: bash <(wget -qO- https://download.openlookeng.io/install.sh) --file <cluster_node_info_path>

For more help message,execute the command below to deploy single node cluster: bash <(wget -qO- https://download.openlookeng.io/install.sh) -h
or: bash <(wget -qO- https://download.openlookeng.io/install.sh) --help

Upgrade openLooKeng Service

Execute the command below to Upgrade openLooKeng Service:

bash <(wget -qO- https://download.openlookeng.io/install.sh) -u <version>

This command will upgrade the current openLooKeng Service to target version, preserving all the existing configurations on current cluster. Execute the command below to list all available versions:

bash <(wget -qO- https://download.openlookeng.io/install.sh) -l

or:

bash <(wget -qO- https://download.openlookeng.io/install.sh) --list

Deploying Configuration to openLooKeng Cluster

Modify configuration file (/home/openlkadmin/.openlkadmin/cluster_config_info) and then execute the command below to deploy the configurations to openLooKeng cluster:

sh /opt/openlookeng/bin/configuration_deploy.sh

Note, if you want to add more configrations or customize the configurations, you can add properties to the templates into the file located at /home/openlkadmin/.openlkadmin/.etc_template/coordinator

or /home/openlkadmin/.openlkadmin/.etc_template/worker.

The property format has to be key=<value>, where value is wrapped with '<' and '>', which means it is a dynamic value. For example:

http-server.http.port=<http-server.http.port>
exchange.client-threads=<exchange.client-threads>

Next, you need to add the actual value into configuration file /home/openlkadmin/.openlkadmin/cluster_config_info. For example:

http-server.http.port=8090
exchange.client-threads=8

Uninstall openLooKeng Service

It is very easy and straight forward to uninstall openLooKeng Service, simply run the command below:

bash /opt/openlookeng/bin/uninstall.sh

This will uninstall openLooKeng Service by removing directory /opt/openlookeng and all files inside it. However, the openlkadmin user and all the configuration files under/home/openlkadmin/ will not be removed. If you wan to delete user and configuration files, you need to run the below command:

bash /opt/openlookeng/bin/uninstall.sh --all

Deploying openLooKeng to Cluster offline

If you can't access the download URL from the machine where you want to install openLooKeng, you can download offline tarball and unpack to /opt directory. Execute the command below to deploy single node cluster:

bash /opt/openlookeng/bin/install_offline.sh`

Execute the command below to deploy multi-node cluster:

bash /opt/openlookeng/bin/install_offline.sh -m

or:

bash /opt/openlookeng/bin/install_offline.sh --multi-node

execute the command below to get help on all available options:

bash /opt/openlookeng/bin/install_offline.sh --help

Adding Node to Cluster

If you want to add node to make the cluster bigger, execute the command below:

bash /opt/openlookeng/bin/add_cluster_node.sh -n

or:

bash /opt/openlookeng/bin/add_cluster_node.sh --node

or:

bash /opt/openlookeng/bin/add_cluster_node.sh -f <add_nodes_file_path>

or:

bash /opt/openlookeng/bin/add_cluster_node.sh --file <add_nodes_file_path>

If there are multiple nodes, separate them by commas(,).

Removing Node to Cluster

If you want to remove node to make the cluster smaller, execute the command below:

bash /opt/openlookeng/bin/remove_cluster_node.sh -n

or:

bash /opt/openlookeng/bin/remove_cluster_node.sh --node

or:

bash /opt/openlookeng/bin/remove_cluster_node.sh -f <remove_nodes_file_path>

or:

bash /opt/openlookeng/bin/remove_cluster_node.sh --file <remove_nodes_file_path>

If there are multiple nodes, separate them with commas(,).