Ballerina performance artifacts are used to continuously test the performance of Ballerina services.
These performance test scripts make use of Apache JMeter and a simple Netty Backend Service, which can echo back any requests and also add a configurable delay to the response.
In order to support a large number of concurrent users, two or more JMeter Servers can be used.
To fully automate the performance tests, an AWS CloudFormation template is used to create a deployment of 5 EC2 instances for Apache JMeter Client, 2 Apache JMeter Servers, Ballerina and Netty Backend Service.
Ballerina makes it easy to write microservices that integrate APIs.
A compiled, transactional, statically and strongly typed programming language with textual and graphical syntaxes. Ballerina incorporates fundamental concepts of distributed system integration and offers a type safe, concurrent environment to implement microservices.
Networked Type System
A type system that embraces network payload variability with primitive, object, union, and tuple types.
An execution model composed of lightweight parallel worker units that are non-blocking where no function can lock an executing thread manifesting sequence concurrency.
Run Performance Tests
You can run Ballerina Performance Tests from the source using the following instructions.
- Maven 3.5.0 or later
- AWS CLI - Please make sure to configure the AWS Cli
and set the output format to
Steps to run performance tests.
- Clone this repository using the following command.
git clone https://github.com/ballerina-platform/ballerina-performance
Run the Maven command
mvn clean installfrom the repository root directory.
Change directory to
cloudformation/targetand extract distribution
$ tar -xf ballerina-performance-distribution-*.tar.gz
- Estimate time to run the performance tests using
./jmeter/run-performance-tests.shscript. You can include or exclude scenarios. You can also change parameters as required.
$ ./jmeter/run-performance-tests.sh -t
./jmeter/run-performance-tests.sh -h Usage: ./jmeter/run-performance-tests.sh [-u <concurrent_users>] [-b <message_sizes>] [-s <sleep_times>] [-m <heap_sizes>] [-d <test_duration>] [-w <warmup_time>] [-n <jmeter_servers>] [-j <jmeter_server_heap_size>] [-k <jmeter_client_heap_size>] [-l <netty_service_heap_size>] [-i <include_scenario_name>] [-e <include_scenario_name>] [-t] [-p <estimated_processing_time_in_between_tests>] [-h] -u: Concurrent Users to test. Multiple users must be separated by spaces. Default "50 100 150 500 1000". -b: Message sizes in bytes. Multiple message sizes must be separated by spaces. Default "50 1024 10240". -s: Backend Sleep Times in milliseconds. Multiple sleep times must be separated by spaces. Default "0 30 500 1000". -m: Application heap memory sizes. Multiple heap memory sizes must be separated by spaces. Default "2g". -d: Test Duration in seconds. Default 900. -w: Warm-up time in minutes. Default 5. -n: Number of JMeter servers. If n=1, only client will be used. If n > 1, remote JMeter servers will be used. Default 1. -j: Heap Size of JMeter Server. Default 4g. -k: Heap Size of JMeter Client. Default 2g. -l: Heap Size of Netty Service. Default 4g. -i: Scenario name to to be included. You can give multiple options to filter scenarios. -e: Scenario name to to be excluded. You can give multiple options to filter scenarios. -t: Estimate time without executing tests. -p: Estimated processing time in between tests in seconds. Default 60. -h: Display this help and exit.
- Go back to
cloudformationdirectory and use
./run-performance-tests.shto run tests. Use flags used in step 4 without the
-tflags after specifying flags for the
./run-performance-tests.sh. You can use
--to indicate the end of command options.
./run-performance-tests.sh -f target/performance-ballerina-distribution-0.1.0-SNAPSHOT.tar.gz -k ~/keys/ballerina-perf-test.pem \ -u https://product-dist.ballerina.io/downloads/0.981.1/ballerina-platform-linux-installer-x64-0.981.1.deb \ -- -d 180 -w 1 -i passthrough_http -e https -u 100 -b 50 -s 0 -j 256m -k 256m -m 256m -l 256m
Please note that this script also needs an existing S3 Bucket name and the region of the S3 Bucket.
./run-performance-tests.sh -h Usage: ./run-performance-tests.sh -f <ballerina_performance_distribution> -k <key_file> -u <ballerina_installer_url> [-n <key_name>] [-b <s3_bucket_name>] [-r <s3_bucket_region>] [-J <jmeter_client_ec2_instance_type>] [-S <jmeter_server_ec2_instance_type>] [-B <ballerina_ec2_instance_type>] [-N <netty_ec2_instance_type>] [-h] -- [run_performance_tests_options] -f: The Ballerina Performance Distribution containing the scripts to run performance tests. -k: The Amazon EC2 Key File. -u: The Ballerina Installer URL. -n: The Amazon EC2 Key Name. Default: ballerina-aws. -b: The Amazon S3 Bucket Name. Default: ballerina-performance. -r: The Amazon S3 Bucket Region. Default: us-east-1. -J: The Amazon EC2 Instance Type for JMeter Client. Default: c5.xlarge. -S: The Amazon EC2 Instance Type for JMeter Server. Default: c5.large. -B: The Amazon EC2 Instance Type for Ballerina. Default: c5.xlarge. -N: The Amazon EC2 Instance Type for Netty (Backend) Service. Default: c5.xlarge. -h: Display this help and exit.
Contributing to Ballerina
As an open source project, Ballerina welcomes contributions from the community. To start contributing, read these contribution guidelines for information on how you should go about contributing to our project.
Check the issue tracker for open issues that interest you. We look forward to receiving your contributions.
Ballerina code is distributed under Apache license 2.0.