Permalink
Browse files

Closes #58 Now uses pre-built AMIs

  • Loading branch information...
oliverlloyd committed Apr 7, 2016
1 parent 00d7381 commit fa6914d8e7e2d19bad907003405c30a958ef160c
Showing with 139 additions and 125 deletions.
  1. +2 −2 README.md
  2. +0 −86 install.sh
  3. +57 −22 jmeter-ec2.properties
  4. +10 −15 jmeter-ec2.sh
  5. +70 −0 verify.sh
View
@@ -2,7 +2,7 @@
This shell script will allow you to run your local JMeter jmx files either using Amazon's EC2 service or you can provide it with a simple, comma-delimited list of hosts to use. Summary results are printed to the console as the script runs and then all result data is downloaded and concatenated to one file when the test completes ready for more detailed analysis offline.
By default it will launch the required hardware using Amazon EC2 (ec2 mode) using Ubuntu AMIs, dynamically installing Java and Apache JMeter. Using AWS it is much easier and cheaper to scale your test over multiple slaves but if you need to you can also pass in a list of pre-prepared hostnames and the test load will be distributed over these instead. Using your own servers can be useful when the target server to be tested can not be easily accessed from a location external to your test network.
By default it will launch the required hardware using Amazon EC2. Using AWS it is much easier and cheaper to scale your test over multiple slaves but if you need to you can also pass in a list of pre-prepared hostnames and the test load will be distributed over these instead. Using your own servers can be useful when the target server to be tested can not be easily accessed from a location external to your test network or you want to repeat a test iteratively.
The script does not use JMeter's Distributed Mode so you do not need to adjust the test parameters to ensure even distribution of the load; the script will automatically adjust the thread counts based on how many hosts are in use. As the test is running it will collate the results from each host in real time and display an output of the Generate Summary Results listener to the screen (showing both results host by host and an aggregated view for the entire run). Once execution is complete it will download each host's jtl file and collate them all together to give a single jtl file that can be viewed using the usual JMeter listeners.
@@ -54,7 +54,7 @@ The script does not use JMeter's Distributed Mode so you do not need to adjust t
###Advanced Properties
`AMI_ID="[A linix based AMI, eg. ami-f95ef58a]"`
Recommended AMI provided. Both Java and JMeter are installed by the script and are not required.
Recommended AMI provided. Both Java and JMeter are installed by the script dynamically and are not required.
`INSTANCE_TYPE="t2.micro"`
Note. Older generation instance types require a different type of AMI (paravirtual vs. hmv).
View
@@ -1,86 +0,0 @@
#!/bin/bash
#
# jmeter-ec2 - Install Script (Runs on remote ec2 server)
#
# Source the jmeter-ec2.properties file, establishing these constants.
. /tmp/jmeter-ec2.properties
REMOTE_HOME=$1
INSTALL_JAVA=$2
JMETER_VERSION=$3
function install_jmeter_plugins() {
wget -q -O $REMOTE_HOME/JMeterPlugins-Extras.jar https://s3.amazonaws.com/jmeter-ec2/JMeterPlugins-Extras.jar
wget -q -O $REMOTE_HOME/JMeterPlugins-Standard.jar https://s3.amazonaws.com/jmeter-ec2/JMeterPlugins-Standard.jar
mv $REMOTE_HOME/JMeterPlugins*.jar $REMOTE_HOME/$JMETER_VERSION/lib/ext/
}
function install_mysql_driver() {
wget -q -O $REMOTE_HOME/mysql-connector-java-5.1.16-bin.jar https://s3.amazonaws.com/jmeter-ec2/mysql-connector-java-5.1.16-bin.jar
mv $REMOTE_HOME/mysql-connector-java-5.1.16-bin.jar $REMOTE_HOME/$JMETER_VERSION/lib/
}
function install_java() {
sudo DEBIAN_FRONTEND=noninteractive apt-get -qqy install openjdk-7-jre
}
function install_jmeter() {
# ------------------------------------------------
# Decide where to download jmeter from
#
# Order of preference:
# 1. S3, if we have a copy of the file
# 2. Mirror, if the desired version is current
# 3. Archive, as a backup
# ------------------------------------------------
if [ $(curl -sI https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz | grep -c "403 Forbidden") -eq "0" ] ; then
# We have a copy on S3 so use that
echo "Downloading jmeter from S3"
wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz
elif [ $(echo $(curl -s 'http://www.apache.org/dist/jmeter/binaries/') | grep -c "$JMETER_VERSION") -gt "0" ] ; then
# Nothing found on S3 but this is the current version of jmeter so use the preferred mirror to download
echo "downloading jmeter from a Mirror"
wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz "http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/$JMETER_VERSION.tgz&action=download"
else
# Fall back to the archive server
echo "Downloading jmeter from Apache Archive"
wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz http://archive.apache.org/dist/jmeter/binaries/$JMETER_VERSION.tgz
fi
# Untar downloaded file
tar -xf $REMOTE_HOME/$JMETER_VERSION.tgz
}
cd $REMOTE_HOME
echo "Updating apt-get..."
sudo apt-get -qqy update
echo "Update of apt-get complete"
if [ $INSTALL_JAVA -eq 1 ] ; then
# install java on ubuntu
echo "Installing java..."
install_java
echo "Java installed"
fi
# install jmeter
echo "Installing jmeter..."
install_jmeter
echo "Jmeter installed"
# install jmeter-plugins [http://code.google.com/p/jmeter-plugins/]
echo "Installing plugins..."
install_jmeter_plugins
echo "Plugins installed"
# install mysql jdbc driver
echo "Installing mysql driver..."
install_mysql_driver
echo "Driver installed"
# Done
echo "software installed"
View
@@ -6,29 +6,65 @@
#
# See README.txt for more details about each property
#
# Pre Installed AMIs (Dont forget to use the right AMI for your region.)
#
# Suggested AMIs
#
# OS Type AMI id User Available sizes
# Ubuntu 64bit ami-f95ef58a ubuntu t2.micro, m1.small, m1.medium, c1.medium, m1.large, m1.xlarge, etc.
# Region OS AMI id User
# eu-west-1 Ubuntu ami-ae72f2dd ubuntu
# ap-southeast-1 Ubuntu ami-aacf1ac9 ubuntu
# ap-northeast-1 Ubuntu ami-f050439e ubuntu
# ap-northeast-2 Ubuntu ami-7140891f ubuntu
# ap-southeast-2 Ubuntu ami-73735110 ubuntu
# eu-central-1 Ubuntu ami-7c759413 ubuntu
# sa-east-1 Ubuntu ami-6f038c03 ubuntu
# us-east-1 Ubuntu ami-90d2c4fa ubuntu
# us-west-1 Ubuntu ami-10473b70 ubuntu
# us-west-2 Ubuntu ami-caf501aa ubuntu
#
AMI_ID="ami-ae72f2dd"
# Should match the AMI
# IMPORTANT - t2.micro is not recommend for anything beyond developement, it works from a shared resource pool that can fluctuate, skewing test results.
#
REMOTE_HOME="/tmp" # This can be left as /tmp - it is a temporary working location
AMI_ID="ami-f95ef58a" # A suitable AMI - 2 suggested AMIs are listed above. (Tested OK with SUSE Linux 32 & 64 bit.). Dont forget to find the right ID for your region.
INSTANCE_TYPE="t2.micro" # Should match the AMI - I do not recommend usng micros for live tests but it's useful for dev work
INSTANCE_SECURITYGROUP="jmeter" # The name OR id of *your* security group in *your* Amazon account - clearly this needs to give your local machine ssh access.
AMAZON_KEYPAIR_NAME="euwest1" # The name of the Amazon Keypair that you want to use. This kea/usery should exist in IAM.
PEM_FILE="euwest1.pem" # The full name of the pem file you downloaded from your Amazon account. Usualy .pem from AWS but you could generate your own and name it what you want.
PEM_PATH=$HOME/.ssh # The path to your pem file
USER="ubuntu" # Should match the AMI
EMAIL="" # Email to be used when tagging instances
REGION="eu-west-1" # Specify the region you will be working in. Required so that we can find the right ami in the right availability zone.
INSTANCE_AVAILABILITYZONE="eu-west-1b" # Should match the AMI and be available in the region above.
RUNNINGTOTAL_INTERVAL="3" # How often the script prints running totals to the screen (n * summariser.interval seconds)
ELASTIC_IPS="" # A list of static IPs that can be assigned to each ec2 host. Ignored if not set.
REMOTE_PORT="22" # The port number sshd is running on,
JMETER_VERSION="apache-jmeter-2.13" # The version of JMeter to be used. Must be the full name used in the dir structure. Does not work for versions prior to 2.5.1.
INSTANCE_TYPE="t2.micro"
# Do not change
REMOTE_HOME="/home/ubuntu"
# The name OR id of *your* security group in *your* Amazon account - the permissions for thius group need to give your local machine ssh access.
INSTANCE_SECURITYGROUP="jmeter"
# The name of the Amazon Keypair that you want to use. It should exist in *your* AWS account for the region you are using.
AMAZON_KEYPAIR_NAME="euwest1"
# The full name of the pem file you downloaded from your Amazon account. Usualy .pem from AWS but you could generate your own and name it what you want.
PEM_FILE="euwest1.pem"
# The path to your pem file
PEM_PATH=$HOME/.ssh
# Should match the AMI
USER="ubuntu"
# Email to be used when tagging instances
EMAIL=""
# Specify the region you will be working in. Required so that we can find the right ami in the right availability zone
REGION="eu-west-1"
# Should match the AMI and be available in the region above.
INSTANCE_AVAILABILITYZONE="eu-west-1b"
# How often the script prints running totals to the screen (n * summariser.interval seconds)
RUNNINGTOTAL_INTERVAL="3"
# A list of static IPs that can be assigned to each ec2 host. Ignored if not set
ELASTIC_IPS=""
# The port number sshd is running on
REMOTE_PORT="22"
# The version of JMeter to be used. Must be the full name used in the dir structure. Does not work for versions prior to 2.5.1
JMETER_VERSION="apache-jmeter-2.13"
#
@@ -43,7 +79,7 @@ JMETER_VERSION="apache-jmeter-2.13" # The version of JMeter to be used.
#
SUBNET_ID="" # The subnet that the instance will belong to.
# REMOTE_HOSTS
# Remote hosts
#
# If this is set then the script will ignore INSTANCE_COUNT passed in at the command line and read in this list of hostnames to run the test over
# instead. If it is not set then n number of hosts will be requested from Amazon.
@@ -54,7 +90,6 @@ SUBNET_ID="" # The subnet that the instance will belong to.
# REMOTE_HOSTS="myhost.com,antherhost.com"
# or:
# REMOTE_HOSTS="blahblah.corp.synergy:2020,10.213.45.6"
# etc.
# DATABASE SETTINGS
# If specified, then the script will import the results to the mysql database given here
View
@@ -371,14 +371,9 @@ function runsetup() {
echo "complete"
echo
fi
# Tell install.sh to attempt to install JAVA
attemptjavainstall=1
else # the property REMOTE_HOSTS is set so we wil use this list of predefined hosts instead
hosts=(`echo $REMOTE_HOSTS | tr "," "\n" | tr -d ' '`)
instance_count=${#hosts[@]}
# Tell install.sh to not attempt to install JAVA
attemptjavainstall=0
echo
echo " -------------------------------------------------------------------------------------"
echo " jmeter-ec2 Automation Script - Running $project.jmx over $instance_count predefined host(s)"
@@ -402,17 +397,17 @@ function runsetup() {
done
fi
# scp install.sh
# scp verify.sh
if [ "$setup" = "TRUE" ] ; then
echo "copying install.sh to $instance_count server(s)..."
echo "copying verify.sh to $instance_count server(s)..."
for host in ${hosts[@]} ; do
(scp -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-i "$PEM_PATH/$PEM_FILE" \
-P $REMOTE_PORT \
$LOCAL_HOME/install.sh \
$LOCAL_HOME/verify.sh \
$LOCAL_HOME/jmeter-ec2.properties \
$USER@$host:$REMOTE_HOME \
&& echo "done" > $project_home/$DATETIME-$host-scpinstall.out) &
&& echo "done" > $project_home/$DATETIME-$host-scpverify.out) &
done
# check to see if the scp call is complete (could just use the wait command here...)
@@ -424,28 +419,28 @@ function runsetup() {
# Count how many out files we have for the copy (if the file exists the copy completed)
# Note. We send stderr to dev/null in the ls cmd below to prevent file not found errors filling the screen
# and the sed command here trims whitespace
res=$(ls -l $project_home/$DATETIME*scpinstall.out 2>/dev/null | wc -l | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
res=$(ls -l $project_home/$DATETIME*scpverify.out 2>/dev/null | wc -l | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
sleep 1
done
progressBar $instance_count $res true
echo
echo
# Install test software
echo "running install.sh on $instance_count server(s) (this can take several minutes)..."
echo "running verify.sh on $instance_count server(s)..."
for host in ${hosts[@]} ; do
(ssh -nq -o StrictHostKeyChecking=no \
-i "$PEM_PATH/$PEM_FILE" $USER@$host -p $REMOTE_PORT \
"$REMOTE_HOME/install.sh $REMOTE_HOME $attemptjavainstall $JMETER_VERSION 2>&1"\
> $project_home/$DATETIME-$host-install.out) &
"$REMOTE_HOME/verify.sh $JMETER_VERSION 2>&1"\
> $project_home/$DATETIME-$host-verify.out) &
done
# check to see if the install scripts are complete
# check to see if the verify script is complete
res=0
while [ "$res" != "$instance_count" ] ; do # Installation not complete (count of matches for 'software installed' not equal to count of hosts running the test)
# Update progress bar
progressBar $instance_count $res
res=$(grep -c "software installed" $project_home/$DATETIME*install.out \
res=$(grep -c "software installed" $project_home/$DATETIME*verify.out \
| awk -F: '{ s+=$NF } END { print s }') # the awk command here sums up the output if multiple matches were found
sleep 1
done
View
@@ -0,0 +1,70 @@
#!/bin/bash
#
# jmeter-ec2 - Install Script (Runs on remote ec2 server)
#
function install_jmeter_plugins() {
echo "Installing plugins..."
wget -q -O ~/JMeterPlugins-Extras.jar https://s3.amazonaws.com/jmeter-ec2/JMeterPlugins-Extras.jar
wget -q -O ~/JMeterPlugins-Standard.jar https://s3.amazonaws.com/jmeter-ec2/JMeterPlugins-Standard.jar
mv ~/JMeterPlugins*.jar ~/$JMETER_VERSION/lib/ext/
}
function install_java() {
echo "Updating apt-get..."
sudo apt-get -qqy update
echo "Installing java..."
sudo DEBIAN_FRONTEND=noninteractive apt-get -qqy install openjdk-7-jre
echo "Java installed"
}
function install_jmeter() {
# ------------------------------------------------
# Decide where to download jmeter from
#
# Order of preference:
# 1. S3, if we have a copy of the file
# 2. Mirror, if the desired version is current
# 3. Archive, as a backup
# ------------------------------------------------
if [ $(curl -sI https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz | grep -c "403 Forbidden") -eq "0" ] ; then
# We have a copy on S3 so use that
echo "Downloading jmeter from S3..."
wget -q -O ~/$JMETER_VERSION.tgz https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz
elif [ $(echo $(curl -s 'http://www.apache.org/dist/jmeter/binaries/') | grep -c "$JMETER_VERSION") -gt "0" ] ; then
# Nothing found on S3 but this is the current version of jmeter so use the preferred mirror to download
echo "downloading jmeter from a Mirror..."
wget -q -O ~/$JMETER_VERSION.tgz "http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/$JMETER_VERSION.tgz&action=download"
else
# Fall back to the archive server
echo "Downloading jmeter from Apache Archive..."
wget -q -O ~/$JMETER_VERSION.tgz http://archive.apache.org/dist/jmeter/binaries/$JMETER_VERSION.tgz
fi
# Untar downloaded file
echo "Unpacking jmeter..."
tar -xf ~/$JMETER_VERSION.tgz
# install jmeter-plugins [http://code.google.com/p/jmeter-plugins/]
install_jmeter_plugins
echo "Jmeter installed"
}
JMETER_VERSION=$1
cd ~
# Java
if java -version 2>&1 >/dev/null | grep -q "java version" ; then
echo "Java is already installed"
else
install_java
fi
# JMeter
if [ ! -d "$JMETER_VERSION" ] ; then
# install jmeter
install_jmeter
else
echo "JMeter is already installed"
fi
# Done
echo "software installed"

0 comments on commit fa6914d

Please sign in to comment.