Testing

2. Introduction

2.1 Objectives.

This test plan focuses on testing the horizontal scalability of the Express project when deployed on a Kubernetes cluster. It supports the following objectives.

1. To

2.2 Background

This test plan was created to evaluate the ability

2.3 Scope

3. Test Items

The Express project will be evaluated holistically.

4. Features to be Tested

The Express project consists of multiple separate components. The correctness of these individual components will not be evaluated.

Express Controller

The Express Controller is responsible for deploying, updating and tearing down application containers. As a result, it is required to to complete the testing tasks. However, it will not be evaluated because unit and functional tests are included as part of the subproject.

6. Approach

8. Suspension Criteria and Resumption Requirements

8.1 Suspension Criteria.

9. Test Deliverables

The following documents shall be included:

1. Test Plan. This document is the test plan for the Express project.
2. Test Summary Reports. A brief summary will be included each time the testing tasks are performed to discuss any notable trends. The summary will not include analysis of these trends.
3. Test Input Data. This will be in the form of the number of users to simulate and the number of users spawned/second.
4. Test Output Data. Each time the testing tasks are performed the following will be produced.
   • distribution.csv containing the absolute number of each HTTP request sent during the test.
   • exceptions.csv containing any errors that occurred during the test.
   • requests.csv containing the requests per second, average sliding-window response time, and number of users.

10. Testing Tasks

The testing tasks are to performed four times. First on a Kubernetes cluster consisting of a single physical or virtual machine. The tasks must then be repeated on the same cluster after the addition of another machine. This process will be performed until the testing tasks have been completed on clusters consisting of one, two, three and four machines.

10.1 Preparation

All preparation will be performed from the single test machine. The preparation assumes that the environmental needs in section 11 have been met. It is also required that the express-controller executable and Express v1.0.0-alpha.1 archive have been downloaded to the single test machine from the Releases section of the Express Github repository.

Cluster Reset

The Kubernetes cluster must be in a consistent state when starting the tasks. This can be accomplished by performing the following tasks. These tasks are only necessary if the testing tasks have already been performed on the cluster.

1. Remove the previously deployed application using express-controller teardown testapi.
2. Remove the Express environment from the cluster using helm delete terrifying-bronco.
3. Remove all persistent volume claims using kubectl delete pvc terrifying-bronco-database-0.
4. Remove all persistent volumes using kubectl delete pv terrifying-bronco-database-0.
5. Wait until No resources found. is shown when using kubectl get pods.

Cluster Setup

The Kubernetes cluster must be prepared for running the testing tasks. This can be done by performing the following tasks.

1. Navigate to the root directory of the Express v1.0.0-alpha.1 download in bash.
2. Add the Express environment to the cluster using helm install --replace --name terrifying-bronco ./express/.
3. Wait until all pods are ready when using kubectl get pods.
4. Deploy the test CRUD API using express-controller deploy testapi allankerr/api --port 8080 --max X --endpoint-config ./containers/api/endpoints.yaml. The parameter max must be changed to the number of nodes in the cluster.
5. Wait until all pods are ready when using kubectl get pods.
6. Create a new admin OAuth2 client for the authorization service using ./bin/authorization-cli.sh client create admin demo-password.
7. Add the user scope to the admin OAuth2 client using ./bin/authorization-cli.sh scope add admin user.
8. Wait until the cluster is stable by ensuring the horizontal pod autoscalers do not have <unknown> targets when using kubectl get hpa.

10.2 Test Procedure

11. Environmental Needs

11.1 Hardware Needs

Four physical or virtual machines with identical hardware specifications are required to run the Kubernetes cluster. Each machine must have a minimum of 2 GB of memory and the following minimum CPU requirements.

1. 2 AWS vCPU
2. 2 GCP Core
3. 2 Azure vCore
4. 2 Hyperthread on a bare-metal Intel processor with Hyperthreading

A single physical or virtual test machine is required to run the tests and interface with the Kubernetes cluster. It is recommended that the machine's CPU is at minimum an Intel Core i5-3330 or equivalent.

The test machine must be able to communicate with the four physical machines over a public or private network using HTTP.

11.2 Software Needs

The four machines must meet the following software requirements.

1. Kubernetes cluster. A Kubernetes cluster running version 1.8.5 must be deployed on the machines. Documentation on setting up a Kubernetes cluster can be found here.
2. Helm. The Helm server must be initialized on the Kubernetes cluster using the helm init command after installing the Helm client and kubectl on the single test machine.

The single test machine must meet the following software requirements.

1. kubectl. kubectl must be installed to interface with the Kubernetes cluster. Documentation on setting up kubectl can be found here. The kubectl config must be configured to access the deployed Kubernetes cluster.
2. Helm. The Helm client must be installed. Documentation on setting up the Helm client can be found here.
3. Python 3.0.
4. Bash.

11.3 Description of Actual Testing Environment

For executing the tests, Kubernetes Engine on the Google Cloud Platform will be utilized. The node pool will be setup in the us-central1-a zone with autoscaling disabled. Four Compute Engine VM instances with 2 vCPU cores and 4 GB of memory will be utilized. The cluster will be setup to use Kubernetes v1.8.5.

To execute the tests on the Kubernetes cluster, a Mid 2014 13-inch MacBook Pro running macOS Sierra Version 10.12.6 with 16 GB of memory and a 3 GHz Intel Core i7 processor will be utilized. The University of Saskatchewan's wireless network will be used to perform the tests.