Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding perf script for galaxies #25

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions galaxies/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,51 @@ route.route.openshift.io/galaxies-service-1 exposed

```

# Run the load
Simulating the load on galaxies benchmarks using hyperfoil/wrk
`./scripts/galaxies-load.sh -c CLUSTER_TYPE [-i SERVER_INSTANCES] [--iter MAX_LOOP] [-n NAMESPACE] [-a IP_ADDR] [-t THREAD] [-R REQUEST_RATE] [-d DURATION] [--connection=CONNECTIONS]`

- **CLUSTER_TYPE**: docker|minikube|openshift
- **SERVER_INSTANCES**: Number of galaxies instances to which you want to run the load. It is optional, if is not specified then by default it will be considered as 1 instance.
- **MAX_LOOP**: Number of times you want to run the load. It is optional, if is not specified then by default it will be considered as 5 iterations.
- **NAMESPACE**: Namespace in which galaxies application is deployed(Required only in the case of openshift cluster and if the application is deployed in other namespaces except `openshift-monitoring`)
- **IP_ADDR**: IP address of the machine. It is optional, if it is not specified then the get_ip function written inside the script will get the IP address of the machine.
- **THREAD**: Number of threads
- **REQUEST_RATE**: Requests rate
- **DURATION**: Test duration
- **CONNECTIONS**: Number of connectionss

Example to run the load on 1 galaxies instances for 2 iterations in openshift cluster

**`$./scripts/galaxies-load.sh -c docker --iter=2`**
```
#########################################################################################
Starting Iteration 1
#########################################################################################

Running wrk load for galaxies instance 1 with the following parameters
CMD=./wrk2.sh --threads=10 --connections=700 --duration=60s --rate=2000 http://192.168.122.105:32000/galaxies
wrk logs Dir : /home/shruthi/galaxies-12-mar/benchmarks/galaxies/logs/galaxies-2021-03-12:12:30

#########################################################################################
Starting Iteration 2
#########################################################################################

Running wrk load for galaxies instance 1 with the following parameters
CMD=./wrk2.sh --threads=20 --connections=700 --duration=60s --rate=2000 http://192.168.122.105:32000/galaxies
wrk logs Dir : /home/shruthi/galaxies-12-mar/benchmarks/galaxies/logs/galaxies-2021-03-12:12:30
#########################################################################################
Displaying the results
#########################################################################################
RUN, THROUGHPUT, RESPONSE_TIME, MAX_RESPONSE_TIME, STDDEV_RESPONSE_TIME, ERRORS
1, 2008.90, 1.31, 150.99, 4.40, 0
2, 2001.05, 939.76, 84.93, 1.84, 0

```
Above image shows the logs of the load run, it processes and displays the output for each run. See Displaying the results section of the log for information about throughput, Number of pages it has retreived, average response time and errors if any.

To run the load for multiple instances in case of openshift cluster follow [README.md](/galaxies/scripts/perf/README.md)

# Cleanup
`$ ./scripts/galaxies-cleanup.sh -c CLUSTER_TYPE[docker|minikube|openshift] [-n NAMESPACE]`

Expand Down
73 changes: 65 additions & 8 deletions galaxies/scripts/galaxies-common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
### Script containing common functions ###

CURRENT_DIR="$(dirname "$(realpath "$0")")"
pushd "${CURRENT_DIR}" >> setup.log
pushd ".." >> setup.log
pushd "${CURRENT_DIR}" > /dev/null
pushd ".." > /dev/null

# Set the defaults for the app
export GALAXIES_PORT="32000"
Expand All @@ -42,6 +42,30 @@ function err_exit() {
fi
}

# Check if java is installed and it is of version 11 or newer
function check_load_prereq() {
echo
echo -n "Info: Checking prerequisites..." >> ${LOGFILE}
# check if java exists
if [ ! `which java` ]; then
echo " "
echo "Error: java is not installed."
exit 1
else
JAVA_VER=$(java -version 2>&1 >/dev/null | egrep "\S+\s+version" | awk '{print $3}' | tr -d '"')
case "${JAVA_VER}" in
1[1-9].*.*)
echo "done" >> ${LOGFILE}
;;
*)
echo " "
echo "Error: Hyperfoil requires Java 11 or newer and current java version is ${JAVA_VER}"
exit 1
;;
esac
fi
}

# Check for all the prereqs
function check_prereq() {
docker version > ${LOGFILE} 2>&1
Expand Down Expand Up @@ -69,7 +93,7 @@ function check_app() {
for status in "${CMD[@]}"
do
if [ -z "${status}" ]; then
echo "Application pod did not come up"
echo "Application pod did not come up"
exit -1;
fi
done
Expand All @@ -84,11 +108,11 @@ function build_galaxies() {
err_exit "Error: Building of docker image of galaxies."
}

# Run the galaxies application
# Run the galaxies application
# input:galaxies image to be used and JVM arguments if any
# output:Create network bridge "kruize-network" and run galaxies application container on the same network
function run_galaxies() {
GALAXIES_IMAGE=$1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable for JVM arguments is missing in this function.

GALAXIES_IMAGE=$1
INST=$2

# Create docker network bridge "kruize-network"
Expand All @@ -98,14 +122,47 @@ function run_galaxies() {
echo "Creating Kruize network: ${NETWORK}..."
docker network create --driver bridge ${NETWORK} 2>>${LOGFILE} >>${LOGFILE}
else
echo "${NETWORK} already exists..."
echo "${NETWORK} already exists..."
fi

# Run the galaxies app container on "kruize-network"
cmd="docker run -d --name=galaxies-app-${INST} -p ${GALAXIES_PORT}:8080 --network=${NETWORK} ${GALAXIES_IMAGE} "
${cmd}s 2>>${LOGFILE} >>${LOGFILE}
cmd="docker run -d --name=galaxies-app-${INST} -p ${GALAXIES_PORT}:8080 --network=${NETWORK} ${GALAXIES_IMAGE}"
${cmd} 2>>${LOGFILE} >>${LOGFILE}
err_exit "Error: Unable to start galaxies container"
((GALAXIES_PORT=GALAXIES_PORT+1))
# Check if the application is running
check_app
}

# Parse the Throughput Results
# input:result directory , Number of iterations of the wrk load
# output:Throughput log file (tranfer per sec, Number of requests per second, average latency and errors if any)
function parse_galaxies_results() {
RESULTS_DIR=$1
TOTAL_LOGS=$2
echo "RUN, THROUGHPUT, RESPONSE_TIME, MAX_RESPONSE_TIME, STDDEV_RESPONSE_TIME, ERRORS" > ${RESULTS_DIR}/Throughput.log
for (( iteration=1 ; iteration<=${TOTAL_LOGS} ;iteration++))
do
RESULT_LOG=${RESULTS_DIR}/wrk-${inst}-${iteration}.log
throughput=`cat ${RESULT_LOG} | grep "Requests" | cut -d ":" -f2 `
responsetime=`cat ${RESULT_LOG} | grep "Latency" | cut -d ":" -f2 | tr -s " " | cut -d " " -f2 `
max_responsetime=`cat ${RESULT_LOG} | grep "Latency" | cut -d ":" -f2 | tr -s " " | cut -d " " -f6 `
stddev_responsetime=`cat ${RESULT_LOG} | grep "Latency" | cut -d ":" -f2 | tr -s " " | cut -d " " -f4 `
weberrors=`cat ${RESULT_LOG} | grep "Non-2xx" | cut -d ":" -f2`
if [ "${weberrors}" == "" ]; then
weberrors="0"
fi
echo "${iteration}, ${throughput}, ${responsetime}, ${max_responsetime}, ${stddev_responsetime}, ${weberrors}" >> ${RESULTS_DIR}/Throughput.log
done
}

# Download the required dependencies
# output: Check if the hyperfoil/wrk dependencies is already present, If not download the required dependencies to apply the load
function load_setup(){
if [ ! -d "${PWD}/hyperfoil-0.13" ]; then
wget https://github.com/Hyperfoil/Hyperfoil/releases/download/release-0.13/hyperfoil-0.13.zip >> ${LOGFILE} 2>&1
unzip hyperfoil-0.13.zip
fi
pushd hyperfoil-0.13/bin > /dev/null
}

16 changes: 11 additions & 5 deletions galaxies/scripts/galaxies-deploy-openshift.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ function usage() {
echo
echo "Usage: $0 -s BENCHMARK_SERVER [-i SERVER_INSTANCES] [-n NAMESPACE] [-g GALAXIES_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] "
echo " "
echo "Example: $0 -s rouging.os.fyre.ibm.com -i 2 -g dinogun/galaxies:1.1-jdk-11.0.10_9 --cpulim=4 --cpureq=2 --memlim=1024Mi --memreq=512Mi"
echo "Example: $0 -s rouging.os.fyre.ibm.com -i 2 -g dinogun/galaxies:1.2-jdk-11.0.10_9 --cpulim=4 --cpureq=2 --memlim=1024Mi --memreq=512Mi"
exit -1
}

Expand Down Expand Up @@ -72,16 +72,12 @@ do
;;
memreq=*)
MEM_REQ=${OPTARG#*=}
# check memory request for unit
check_memory_unit ${MEM_REQ}
;;
cpulim=*)
CPU_LIM=${OPTARG#*=}
;;
memlim=*)
MEM_LIM=${OPTARG#*=}
# check memory limit for unit
check_memory_unit ${MEM_LIM}
;;
*)
esac
Expand Down Expand Up @@ -119,6 +115,16 @@ if [ -z "${NAMESPACE}" ]; then
NAMESPACE="${DEFAULT_NAMESPACE}"
fi

# check memory limit for unit
if [ ! -z "${MEM_LIM}" ]; then
check_memory_unit ${MEM_LIM}
fi

# check memory request for unit
if [ ! -z "${MEM_REQ}" ]; then
check_memory_unit ${MEM_REQ}
fi

# Create multiple yamls based on instances and Update the template yamls with names and create multiple files
# input:galaxies and service-monitor yaml file
function createInstances() {
Expand Down
171 changes: 171 additions & 0 deletions galaxies/scripts/galaxies-load.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020, 2021 Red Hat, IBM Corporation and others.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
### Script to load test galaxies application on docker,minikube or openshift###
#

CURRENT_DIR="$(dirname "$(realpath "$0")")"
source ${CURRENT_DIR}/galaxies-common.sh

function usage() {
echo
echo "Usage: -c CLUSTER_TYPE[docker|minikube|openshift] [-i SERVER_INSTANCES] [--iter=MAX_LOOP] [-n NAMESPACE] [-a IP_ADDR] [-t THREAD] [-R REQUEST_RATE] [-d DURATION] [--connection=CONNECTIONS]"
exit -1
}

while getopts c:i:a:n:t:R:d:-: gopts
do
case ${gopts} in
-)
case "${OPTARG}" in
iter=*)
MAX_LOOP=${OPTARG#*=}
;;
connection=*)
CONNECTIONS=${OPTARG#*=}
;;
esac
;;
c)
CLUSTER_TYPE=${OPTARG}
;;
i)
SERVER_INSTANCES="${OPTARG}"
;;
a)
IP_ADDR="${OPTARG}"
;;
n)
NAMESPACE="${OPTARG}"
;;
t)
THREAD="${OPTARG}"
;;
R)
REQUEST_RATE="${OPTARG}"
;;
d)
DURATION="${OPTARG}"
;;
esac
done

if [ -z "${CLUSTER_TYPE}" ]; then
usage
fi

if [ -z "${SERVER_INSTANCES}" ]; then
SERVER_INSTANCES=1
fi

if [ -z "${MAX_LOOP}" ]; then
MAX_LOOP=5
fi

if [ -z "${NAMESPACE}" ]; then
NAMESPACE="${DEFAULT_NAMESPACE}"
fi

if [ -z "${THREAD}" ]; then
THREAD="10"
fi

if [ -z "${REQUEST_RATE}" ]; then
REQUEST_RATE="2000"
fi

if [ -z "${DURATION}" ]; then
DURATION="60"
fi

if [ -z "${CONNECTIONS}" ]; then
CONNECTIONS="700"
fi

case ${CLUSTER_TYPE} in
docker)
if [ -z "${IP_ADDR}" ]; then
get_ip
fi
;;
icp|minikube)
if [ -z "${IP_ADDR}" ]; then
IP_ADDR=$(minikube service list | grep "galaxies" | awk '{print $8}' | tr '\r\n' ' ')
IFS=' ' read -r -a IP_ADDR <<< ${IP_ADDR}
fi
;;
openshift)
if [ -z "${IP_ADDR}" ]; then
IP_ADDR=($(oc status --namespace=${NAMESPACE} | grep "galaxies" | grep port | cut -d " " -f1 | cut -d "/" -f3))
fi
;;
*)
echo "Load is not determined"
;;
esac

LOG_DIR="${PWD}/logs/galaxies-$(date +%Y-%m-%d:%H:%M)"
mkdir -p ${LOG_DIR}

# Check if java is installed and it is of version 11 or newer
check_load_prereq >> setup.log

# Create the required load setup
load_setup >> setup.log

for(( inst=1; inst<=${SERVER_INSTANCES}; inst++ ))
do
for iter in `seq 1 ${MAX_LOOP}`
do
echo
echo "#########################################################################################"
echo " Starting Iteration ${iter} "
echo "#########################################################################################"
echo

# Change these appropriately to vary load
USERS=$(( THREAD*iter ))

case "${CLUSTER_TYPE}" in
docker)
cmd="./wrk2.sh --threads=${USERS} --connections=${CONNECTIONS} --duration=${DURATION}s --rate=${REQUEST_RATE} http://${IP_ADDR}:${GALAXIES_PORT}/galaxies"
;;
minikube)
cmd="./wrk2.sh --threads=${USERS} --connections=${CONNECTIONS} --duration=${DURATION}s --rate=${REQUEST_RATE} ${IP_ADDR[inst-1]}/galaxies"
;;
openshift)
cmd="./wrk2.sh --threads=${USERS} --connections=${CONNECTIONS} --duration=${DURATION}s --rate=${REQUEST_RATE} http://${IP_ADDR[inst-1]}/galaxies"
;;
esac

# Check if the application is running
check_app
# Run the wrk load
echo "Running wrk load for galaxies instance ${inst} with the following parameters" | tee -a ${LOG_DIR}/wrk-${inst}-${iter}.log
echo "CMD=${cmd}" | tee -a ${LOG_DIR}/wrk-${inst}-${iter}.log
echo "wrk logs Dir : ${LOG_DIR}"
sleep 20
${cmd} >> ${LOG_DIR}/wrk-${inst}-${iter}.log
err_exit "can not execute the command"
done
((GALAXIES_PORT=GALAXIES_PORT+1))
# Parse the results
parse_galaxies_results ${LOG_DIR} ${MAX_LOOP}
echo "#########################################################################################"
echo " Displaying the results "
echo "#########################################################################################"
cat ${LOG_DIR}/Throughput.log
done
Loading