Skip to content
Guidelines of Performance Testing using nGrinder
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
doc
src/main
.gitignore
README.md
ngrinder-guideline.iml
pom.xml

README.md

1. nGrinder Introduction

What is NGrinder?

nGrinder is a platform for stress tests that enables you to execute script creation, test execution, monitoring, and result report generator simultaneously. nGrinder allows you running test scripts written in Jython(Python running on JVM) or Groovy across a number of machines. It's internal engine is based on Grinder.

nGrinder consists of two major components.

Controller

  • Provides web interface for performance testing.
  • Coordinates test processes.
  • Collates and displays test statistics.
  • Let user create and modify script.

Agent

  • Runs processes and threads that put loads on target machines when running in agent mode.
  • Monitors target system performance (e.g.: CPU/memory) when running in monitor mode.

nGrinder Architecture

nGrinder Architecture

2. nGrinder Installation

2.1. Install nGrinder Controller

# Pull the docker image
$ docker pull seedotech/ngrinder-controller:3.4.2

# Create and run docker container
# NGRINDER Controller ports
# 8080: Default controller web UI port
# 16001: Connection between controller & agent
# 12000-12009: controllers allocate stress tests through these ports
$ docker run -itd \
                   -p 8080:8080 \
                   -p 16001:16001 \
                   -p 12000-12009:12000-12009 \
                   --name ngrinder-controller \
                   --hostname ngrinder-controller \
                   seedotech/ngrinder-controller:3.4.2 &> /dev/null

Access to the http://localhost:8080 with the default account admin/admin to use nGrinder's dashboards.

2.2. Install nGrinder Agent

# Pull the docker image
$ docker pull seedotech/ngrinder-agent:3.4.2

# Enter the host's IP of the nGrinder Controller container
$ docker run -itd \
                    -e NGRINDER_CONTROLLER_IP=$NGRINDER_CONTROLLER_IP \
                    -e NGRINDER_CONTROLLER_PORT=$NGRINDER_CONTROLLER_PORT \
                    --name ngrinder-agent-vtrd-raycad \
                    --hostname ngrinder-agent-vtrd-raycad \
                    seedotech/ngrinder-agent:3.4.2 &> /dev/null
                    
e.g
$ docker run -itd \
                      -e NGRINDER_CONTROLLER_IP=10.221.71.71 \
                      -e NGRINDER_CONTROLLER_PORT=16001 \
                      --name ngrinder-agent-vtrd-raycad \
                      --hostname ngrinder-agent-vtrd-raycad \
                      seedotech/ngrinder-agent:3.4.2 &> /dev/null

You can install many agents to create an nGrinder cluster.

nGrinder Cluster

You can set an agent on/off by clicking on the Approved/Disapproved buttons

3. Script Creation

For easy test scripts you can create separated/individual file to run. For the complex test scripts (e.g using external libs) you should create a test script project to implement/debug scripts.

Below are the steps to create a Groovy Maven Project:

  • Go to the nGrinder dashboard and select the Script menu item.

  • Click on the Create a script button.

Create Script

  • Select Groovy Maven Project in the Script Name combobox.

Configure Script

  • Tick on the Create Lib and Resource folders then click on the Create button.

Create Script

  • Open IntelliJ IDEA and check out the test project from the Subversion.

Import From Subversion

  • Add a new repository and select dbperf-benchmark.

Import From Subversion

  • Groovy Script Structure
class Test1 {

    public static GTest test;
    public static HTTPRequest request;

    // This method is executed once per a process.
    @BeforeProcess
    public static void beforeProcess() {
        test = new GTest(1, "${name}");
        request = new HTTPRequest();
        test.record(request);
        grinder.logger.info("before process.");
    }

    // This method is executed once per a thread.
    @BeforeThread
    public void beforeThread() {
        grinder.statistics.delayReports=true;
        grinder.logger.info("before thread.");
    }

    // This method is continuously executed until you stop the test
    @Test
    public void test(){
        HTTPResponse result = request.GET("${url}");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }
    
    // This method is executed once per a process.
    @AfterProcess
    public static void afterProcess() {
        // Close the global variables here if we have...
        grinder.logger.info("after process.");
    }
}

Groovy Script Structure

Annotation Description Applied to Usage
@BeforeProcess Define behaviors which should be executed before the process is invoked static method Load resource files which are shared by threads. Define GTest and instrument the test target using record method.
@AfterProcess Define behaviors which should be executed after the process is terminated static method Close resource files.
@BeforeThread Define behaviors which should be executed before the each thread is executed. member method Login the target system. Set up the thread bounded values. (For example, Cookie Handler)
@AfterThread Define behaviors which should be exeucted after the each thread is exeucted member method Logout the target system
@Before Define behaviors which should be executed before every @Test annotated method are executed. member method Common logic which is shared by the multiple @Test annotated methods. Variable set up
@After Define behaviors which should be executed after every @Test annotated methods are executed. member method Not used much.
@Test Define the test behavior. Excuted muliple times member method Test body

4. Script Debugging

  • Select a Groovy script you want to debug.
  • Right click and select Debug 'YOUR_SELECTED_SCRIPT'

After running you might get an error like that:

java.lang.RuntimeException: Please add -javaagent:/home/raycad/.m2/repository/net/sf/grinder/grinder-dcr-agent/3.9.1/grinder-dcr-agent-3.9.1.jar in 'Run As JUnit' vm argument.

To fix this you just edit the Run/Debug configuration and add -javaagent:/home/raycad/.m2/repository/net/sf/grinder/grinder-dcr-agent/3.9.1/grinder-dcr-agent-3.9.1.jar to the VM options then Run/Debug again.

[NOTE]: nGrinder Groovy code just runs with Java 1.8. To make sure your project was set using Java 1.8 please go to File/Project Structure.... In the Project menu item of the Project Settings select Project SDK 1.8.

nGrinder Debug

5. Performance Test Creation

  • Go to the nGrinder dashboard and select the Performance Test menu item.

  • Click on the Create Test.

  • Configure parameters.

Configure Performance Test

Using Ramp Up

Configure Performance Test

  • Analyze test result.

Performance Test Results

Ramp Up Test Results

Performance Test Results

6. Rules And Conventions For Performance Testing

  • Prepare Test Data (database, mocked files,...) on the STG Environment.

  • Create at least one Groovy Test Class for each API. Add the task link/information to the class's header for tracing tasks information in the future.

  • Package of the test script is the same with package or micro-service/backend's name of APIs.

  • Create Tests on nGrinder Dashboard after completing test scripts. Convention of the Test Name should be: [JIRA_TASK_CODE] API Description. E.g. [VF-1115] Request Customer Access Token

  • Run Tests to verify your created test cases (if the test failed please analyze logs).

  • Ensure the test results meet the requirements.

Examples:

Project Structure Template

nGrinder Dashboard Sample

7. Performance Testing Process

Performance Testing Process

8. References

https://github.com/naver/ngrinder/wiki/Architecture

https://github.com/naver/ngrinder/wiki/Groovy-Script-Structure

You can’t perform that action at this time.