Skip to content

Latest commit

 

History

History
43 lines (35 loc) · 3.44 KB

README.md

File metadata and controls

43 lines (35 loc) · 3.44 KB

How to use Test-Suite

Test-Suite is a micro-benchmark suite designed for validating various static analysis algorithms (particularly pointer analyses and static bug checkers) developed in SVF. It includes around 400 hand-written programs and code snippets. Test-Suite provides flexible and extendable interfaces for users to add their own tests for validating the correctness of different static analyses.

Download SVF and execute the following commandline under SVF's root directory. build.sh will build SVF and run SVF's analyses against the test cases if Test-suite folder is present.

git clone https://github.com/SVF-tools/SVF.git
cd SVF
git clone https://github.com/SVF-tools/Test-Suite
source ./build.sh
cd Release-build
ctest

The following gives the Test-Suite's folders and the corresponding SVF's options for validation.

Folder SVF option Description
basic_c_tests wpa -ander -stat=false basic test cases for C programs (flow-insensitive and field-sensitive analysis)
basic_cpp_tests wpa -ander -stat=false basic test cases for C++ programs (flow-insensitive and field-sensitive analysis)
fs_tests wpa -fspta -stat=false flow-sensitive tests
cs_tests dvf -cxt -print-pts=false -stat=false context-sensitive tests
path_tests path-sensitive tests
complex_tests wpa -ander -stat=false complex test cases simplified from real programs
mta multithreaded test cases
mem_leak saber -leak -valid-tests -mempar=inter-disjoint -stat=false memory leak test cases
double_free saber -dfree -valid-tests -stat=false double free test cases

Scripts

Travis CI is used to build all the test cases, It uses ./generate_bc.sh to do so and then pushes the compiled test cases into master branch.

All tests are specified in CMakeLists.txt file. The CMakeLists.txt file uses foreach loops to specify tests in bulk. To add new tests append a command and specified folder into the respective commands list and folders list in CMakeLists.txt An example of a test:

add_test(
  NAME basic_c_tests/CI-funptr.c
  COMMAND wpa -ander -stat=false ${CMAKE_CURRENT_SOURCE_DIR}/test_cases_bc/basic_c_tests/CI-funptr.c.bc
  WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/${CMAKE_BUILD_TYPE}-build/bin
)

For differential testing of two options a C++ script diff_tests.cpp is used. This script will run each option, compare the results and output. If the output is 0 then the test passed.