Skip to content

HowToUse

Jump to bottom
Peter Chang edited this page Sep 1, 2016 · 11 revisions

This document explains how to use pat to test your program and evaluate its performance.

Introduction

pat is a C++ performance analyzing and testing framework on Android and Linux platforms. We refer the concept of Google Test and extend its scope from testing framework into Performance Analysis Framework.

With the help of pat, programmers who wants to analyze their program, only need to write test cases via pat, and pat can analyze programs' correctness and performance.

The following section provides examples to show how to use pat.

Basic concept

The easiest steps to use pat are:

  1. Writing test cases including ASSERT, EXPECT, and PERFORM macros.
  2. Compiling these cases with your main program and the shared library of pat.
  3. Finally, to execute the binary file and you can get its correctness and performance.

Like Google Test, pat provides several macro to integrate your test cases and your main program. The next section introduces to each macro and explains how to use these macros to write your test-cases.

Macros

There are three kinds of macros in pat, ASSERT, EXPECT, and PERFORM. ASSERT and EXPECT are assertions for condition testing and PERFORM wraps a block of code for performance testing. The difference between ASSERT and EXPECT is whether it is fatal or not.

ASSERT is assertion for fatal condition testing. That is to say, if the condition of ASSERT_* fails, the test fails and stops immediately.

EXPECT is non-fatal assertion. When the condition of EXPECT_* fails, it displays on screen to indicate that is a non-fatal failure. The whole test keeps running and isn't considered as a failure.

PERFORM evaluates performance rather than correctness. With the help of PERFORM, you could measure the performance of code within a test.

The following sections give you examples to use these macros in your test program and explains the meanings of the output results.

Examples

All examples listed here are also in pat's source code, ${PAT_SRC}/examples.

Include PAT Stuff

Let's start from the most useful macro: ASSERT. All macros and definitions used in pat are located in the file <pat/pat.h>

For example, in assertion/main.cpp, it includes:

#include "pat/pat.h"

Declare A Test Function via PAT_F

PAT_F macro declare a test. It has two parameters: test case's name and test function's name. In pat, test logic is grouped into test cases. Every test case has a bunch of test functions. Here is an example to declare a test function.

PAT_F(AircraftCase, take_off_test)
{
   // My Wonderful Test Code
}
PAT_F(AircraftCase, landing_test)
{
   // My Wonderful Test Code
}

This example defines a test case AircraftCase who has two test functions take_off_test and landing_test. This is how does the "pat" organize the test codes. You should put logically related tests into the same test case.

Write A Test Function

By writing test functions, users can evaluate performance and reliability. For reliability, users can assert fatal or non-fatal conditions in a block of code; For performance, users can evaluate performance by using pat PERFORM macros. The following examples explains:

  • How to assert one fatal condition
  • How to assert multiple fatal conditions
  • How to assert non-fatal conditions
  • Evaluate performance
Assert one fatal condition

This example shows you how to use ASSERT_TRUE and ASSERT_FALSE to write a test function.

Here is an example of ASSERT_TRUE:

PAT_F(SeriesCase, fibonacci_test)
{
  int result = 0;

  ASSERT_TRUE(0 != fibonacci(10));
}

ASSERT is a macro resembling function calls. You test a class or a function by making assertions about its behavior. There are two kinds of ASSERT macros, ASSERT_TRUE and ASSERT_FALSE.

First of all, ASSERT_TRUE means that it will PASS only if the condition is TRUE. In this case, our condition is 0 != fibonacci(10) that is obviously TRUE. So this test is executed successfully.

If the condition of ASSERT_TRUE fails, the test will stop and exit the test immediately.

Here is an example to show a false condition:

PAT_F(SeriesCase, factorial_test_fail)
{
  ASSERT_TRUE(360 == factorial(5));

  cout<<"end of the test"<<endl;
}

Since the condition, 360 == factorial(5), obviously fails, ASSERT_TRUE(360 == factorial(5)) should be considered as a failure. As a result, you can see these on shell in run-time:

[ RUN      ] MyCase.factorial_test_fail
[  FAILED  ] 
main.cpp:62: fatal: failed to assert
Value of: 360 == factorial(5)
  Actual:   false
  Expected: true

Because ASSERT_TRUE is considered as a failure, the test stops and exits the test immediately without executing cout<<"end of the test"<<endl.

Second, ASSERT_FALSE means that it will PASS only if the condition is FALSE and vice versa.

PAT_F(SeriesCase, factorial_test)
{
  ASSERT_FALSE(360 == factorial(5));
}

In this test, the condition, 360 == factorial(5), obviously fails. Thus, it runs successfully.

In your test case, you can change the condition as you want. Pat provides rich testing macros, such as conditional assertions - ASSERT_TRUE/FALSE, and predicated assertions - ASSERT_EQ/NE/LT/LE/GT/GE.

Assert multiple fatal conditions

If you have more than one fatal conditions, you can use multiple ASSERT_* in one test. If there is any fatal condition considered as failure, the test will stop and exit immediately without executing the subsequent code.

Here is an example for multiple ASSERT_*:

PAT_F(MyCase, fibonacci_test)
{
  ASSERT_TRUE(0 != fibonacci(10));
  ASSERT_TRUE(2 == fibonacci(10));
  ASSERT_TRUE(0 != fibonacci(10));
}

Since the condition, 2 == fibonacci(10), is obviously FALSE, ASSERT_TRUE(2 == fibonacci(10)) should be considered as a failure. As a result, you can see these on shell in runtime:

[ RUN      ] MyCase.fibonacci_test
[  FAILED  ] 
main.cpp:53: fatal: failed to assert
Value of: 2 == fibonacci(10)
  Actual:   false
  Expected: true

Because ASSERT_TRUE(2 == fibonacci(10)) is considered as a failure, the test stops and exits the test immediately without executing the second ASSERT_TRUE(0 != fibonacci(10)).

Assert non-fatal conditions

EXPECT is non-fatal assertion. When the condition of EXPECT_* fails, it displays the result on screen and keeps running.

There are two kinds of EXPECT macros, EXPECT_TRUE and EXPECT_FALSE. EXPECT_TRUE means that it will PASS only if the condition is TRUE. EXPECT_FALSE means that it will PASS only if the condition is FALSE and vice versa. Like fatal conditions, you can insert multiple EXPECT_* macros in a test, too.

Here is an example for EXPECT_*:

PAT_F(MyCase, fibonacci_test)
{
  EXPECT_TRUE(0 != fibonacci(10));
  EXPECT_TRUE(2 == fibonacci(10));
  EXPECT_TRUE(10 == fibonacci(10));
  EXPECT_FALSE(10 != fibonacci(10));
}

In this example, there are three non-fatal assertions, EXPECT_TRUE(2 == fibonacci(10)), EXPECT_TRUE(10 == fibonacci(10)), and EXPECT_FALSE(10 != fibonacci(10)), are considered as a failure, the program executes all without stopping and show these three unsuccessful expectations on screen:

[ RUN      ] MyCase.fibonacci_test
[  FAILED  ] 
main.cpp:53: error: failed to expect
Value of: 2 == fibonacci(10)
  Actual:   false
  Expected: true
main.cpp:54: error: failed to expect
Value of: 10 == fibonacci(10)
  Actual:   false
  Expected: true
main.cpp:55: error: failed to expect
Value of: 10 != fibonacci(10)
  Actual:   true
  Expected: false
Evaluate performance

PERFORM macro measure the performance of a block of code wrapped by it. Here is an example of PERFROM:

PERFORM {
  fibonacci(10);
}

In this example, PERFORM macro measures the time of executing fibonacci(10).

It shows the time information as:

[ RUN      ] MyCase.fibonacci_test
[ TIME (ns)]    1000

In this test, it measures the time of executing factorial(5) and factorial(100) independently.

When executing, it shows the time information respectively on screen:

[ RUN      ] MyCase.factorial_test
[ TIME (ns)]       0    1000

Run All Test Functions via RunAll()

Call Initialize(&argc, argv) and RunAll().

int main(int argc, char* argv[])
{
  pat::Test::Initialize(&argc, argv);
  pat::Test::RunAll();
}

Initialize(&argc, argv) helps pat to initial the output interface of pat. If you do not call Initialize, you can not see the output of pat.

The RunAll() function magically knows about all the tests you defined. RunAll() runs all the tests you've declared, prints the results and returns zero only if all tests are succeed. Otherwise, it returns one.

Clone this wiki locally