Skip to content

Commit

Permalink
test: shuffle tests
Browse files Browse the repository at this point in the history
luatest has an option --shuffle that allows to randomise the order of
tests. This can be useful to detect a test that passes just because it
happens to run after an unrelated test that leaves the system in a
favourable state. Patch enable tests shuffling with predefined random
seed generated by CMake [1]. Seed is useful for reproducing a problems
related to the specific test order.

1. https://cmake.org/cmake/help/latest/command/string.html#random
  • Loading branch information
ligurio authored and 0x501D committed Jul 25, 2022
1 parent f11a68f commit 8111743
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ include_directories(${TARANTOOL_INCLUDE_DIRS})
# Set CFLAGS
set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wall -Wextra")

string(RANDOM ALPHABET 0123456789 seed)

add_subdirectory(http)

add_custom_target(luacheck
Expand All @@ -31,7 +33,7 @@ add_custom_target(luacheck
)

add_custom_target(luatest
COMMAND ${LUATEST} -v --coverage
COMMAND ${LUATEST} -v --coverage --shuffle all:${seed}
BYPRODUCTS ${CODE_COVERAGE_STATS}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "Run regression tests"
Expand Down

0 comments on commit 8111743

Please sign in to comment.