Skip to content
This repository was archived by the owner on Feb 8, 2024. It is now read-only.

Cosbench: setup and test run

Jump to bottom
Andrei Zheregelia edited this page May 6, 2020 · 3 revisions

Intro

Cosbench is a tool to test and measure cluster performance for data I/O. It supports S3 protocol.
Some parts of this guide may be redundant since real setup was deployed only once. Minimal configuration still has to be defined.

Prerequisites

Generic overview

Cosbench consists of controller and one or more drivers. Drivers do test workload. Controller provides management and reporting tools including WebUI.
Both controller and driver may be located on the same node.
EES cluster hosts mero and s3server services. Since test works via S3 protocol, required accounts should be created on EES cluster and credentials should be provided to cosbench configuration files. To do this - S3 client is needed only during setup step to actually create S3 account on EES cluster and generate required credentials for cosbench. That S3 client may be deployed on any node that has connection to EES cluster data network - so far it is 176.16.. network. It even can be configured on EES cluster node.
Following topology is suggested:

  • 2 EES Cluster nodes
  • 1 test node which hosts cosbench and S3 client.

It is planned to use s3.seagate.com domain name to access EES cluster. So, cosbench and client nodes have to resolve this name to data network Cluster IP. /etc/hosts file is used to do it.

S3 client configuration

This step incorporates account creation and EES cluster sanity check from the point of S3 data I/O.
Technically this will partially duplicate S3 guide. So, check it as well if something goes wrong.

Install S3 tools

AWS client tools are required

  • Choose required repo: http://ci-storage.mero.colo.seagate.com/releases/eos/
  • yum install -y eos-prvsnr-cli-<version>_el7.x86_64.rpm
  • /opt/seagate/eos-prvsnr/cli/setup-provisioner --singlenode
  • salt-call state.apply components.s3clients
  • pip install awscli
  • pip install awscli-plugin-endpoint

Make S3 server alias

Edit /etc/hosts and add following entry:

<Cluster IP>  s3.seagate.com sts.seagate.com iam.seagate.com sts.cloud.seagate.com

Note: in case S3 client and cosbench are located on different nodes - this step should be done on both of them.
Make sure that EES cluster nodes also has this entry in /etc/hosts.

Configure certificates

ca.crt file is installed during s3client component provisioning.

cp /etc/ssl/stx-s3-clients/s3/ca.crt /etc/pki/ca-trust/source/anchors/
cd /etc/pki/ca-trust/source/anchors/;
update-ca-trust enable
update-ca-trust extract

Configure AWS and create S3 account on EES cluster

This

s3iamcli createaccount -n cosbench_testing \
    -e cosbench@ees.com --ldapuser sgiamadmin --ldappasswd ldapadmin

Sample output:

AccountId = 828634098320, CanonicalId = c0d3ee09f15e4c63ba60a119e3aea27acb86e2d59d5b4fc09cbb21d3ca1dd2f5, RootUserName = root, AccessKeyId = TfyaGuDsT6-yICzzmlXoGw, SecretKey = 0+yyCrFRDt6NBQ1lQg9rvhuuLy1vOwyqdg35ifRW

AccessKeyId and SecretKey are required for cosbench config.

Installation

Install cosbench

Installation is done to /root/ folder - the same approach as QA team use. Other options are not tried yet.

cd /root/
wget https://github.com/intel-cloud/cosbench/releases/download/v0.4.2.c3/0.4.2.c3.zip
unzip 0.4.2.c3.zip
ln -s 0.4.2.c3 cos
cd cos
chmod +x *.sh
yum install -y java curl nc

Setup python virtual environment for cosbench_perfomance test

cd /root/eos-test
python3.6 -m venv virenv
pip install -r requirements.txt
python setup.py install
python setup.py develop

Configuration

Edit /root/cos/conf/controller.conf.
<ip address of first driver node> is normal network IP of current node since both controller and driver are located together.
Note: 127.0.0.1 is supposed to work but not tested yet.

[controller]
concurrency=1
drivers = 1
log_level = DEBUG
log_file = /root/cos/log/system.log
archive_dir = archive

[driver1]
name = DriverA
url = http://<ip address of first driver node>:18088/driver

Edit /root/cos/conf/driver.conf

[driver]
name=DriverA
url=http://127.0.0.1:18088/driver

Configure cosbench_perfomance test

Edit /root/eos-test/scripts/cosbench/config.ini file

[AWSCREDENTIALS]
s3_endpoint =  https://s3.seagate.com
access_key  =  <S3 AccessKeyId>
secret_key  =  <S3 SecretKey>

Start & stop

cd /root/cos
Use start-all.sh, stop-all.sh scripts to start/stop controller and driver.

Configure and run cosbench_perfomance test

By this time cosbench controller and driver should be running.
Check controller-boot.log and driver-boot.log. There should not be any errors there. To run test use:

python cosbench_performance.py -w <no of sessions> --buildver <any text>

/root/eos-test/scripts/cosbench/config.ini is used to configure test parameters.

Test config for EOS-7744

[WORKLOADPROPS]
no_of_buckets=1000
no_of_objects=1000
object_size_in_mb=1
run_time_in_seconds=3600

python cosbench_performance.py -w 100 --buildver <any text>

Logs and verification

  • WebUI
http://<controller-IP>:19088/controller/index.html
  • Every job has dedicated log in mission/ folder.
  • driver-boot.log and controller-boot.log are updated during test run. They are not only about boot.
  • system.log contains high-level information about overall system operation.

Clone this wiki locally