Skip to content

Commit

Permalink
Simplify the capsicum-test wrapper script
Browse files Browse the repository at this point in the history
Instead of running tests one-by-one with the shell wrapper we now run
the full gtest testsuite twice (once as root, once as non root). This
significantly speeds up running tests despite running them twice.
This change also passes the missing -u flag to capsicum-test that caused
test failures (https://bugs.freebsd.org/250178)

Previously, running the testsuite with the wrapper script took ~3s per
test on aarch64 QEMU, i.e. a total of almost 5 minutes.
Now it takes 6 seconds to run all tests twice.

Before:
root@freebsd-aarch64:/usr/tests/sys/capsicum # /usr/bin/time kyua test functional
94/96 passed (2 failed)
      309.97 real        58.46 user       244.31 sys

After:
root@freebsd-aarch64:/usr/tests/sys/capsicum # /usr/bin/time kyua test functional
functional:test_root  ->  passed  [2.659s]
functional:test_unprivileged  ->  passed  [2.391s]
2/2 passed (0 failed)
        5.48 real         1.06 user         2.52 sys

This overhead is caused by kyua + atf-sh spawning lots of additional
processes and can be avoided by just running the googletest test binary.
syscall                     seconds   calls  errors
fork                   39.810229456    1275       0
sigprocmask            13.546928736     572       0

i.e. 1275 processes spawned to run a single test.

Test Plan:	All tests pass with D28907.
PR:		250178
Reviewed By:	lwhsu
Differential Revision: https://reviews.freebsd.org/D29014

(cherry picked from commit 53a535c)
  • Loading branch information
arichardson authored and ericbsd committed Mar 24, 2021
1 parent 48def31 commit f305a77
Showing 1 changed file with 17 additions and 53 deletions.
70 changes: 17 additions & 53 deletions tests/sys/capsicum/functional.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,69 +29,33 @@
#
# $FreeBSD$

SRCDIR=$(atf_get_srcdir)
CAPSICUM_TEST_BIN=capsicum-test

check()
{
local tc=${1}
atf_test_case "test_root"
test_root_head() {

atf_check -s exit:0 -o match:PASSED -e ignore \
${SRCDIR}/${CAPSICUM_TEST_BIN} --gtest_filter=${tc}
atf_set descr 'Run capsicum-test as root'
atf_set require.user root
}

skip()
{
local reason=${1}

atf_skip "${reason}"
test_root_body() {
atf_check -s exit:0 -o match:PASSED -e ignore \
"$(atf_get_srcdir)/${CAPSICUM_TEST_BIN}" -u "$(id -u tests)"
}

add_testcase()
{
local tc=${1}
local tc_escaped word

tc_escaped=$(echo ${tc} | sed -e 's/\./__/')

atf_test_case ${tc_escaped}

if [ "$(atf_config_get ci false)" = "true" ]; then
case "${tc_escaped}" in
ForkedOpenatTest_WithFlagInCapabilityMode___|OpenatTest__WithFlag)
eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/249960\"; }"
;;
PipePdfork__WildcardWait)
eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/244165\"; }"
;;
Capability__NoBypassDAC)
eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/250178\"; }"
;;
Pdfork__OtherUserForked)
eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/250179\"; }"
;;
*)
eval "${tc_escaped}_body() { check ${tc}; }"
;;
esac
else
eval "${tc_escaped}_body() { check ${tc}; }"
fi
atf_test_case "test_unprivileged"
test_unprivileged_head() {

atf_add_test_case ${tc_escaped}
atf_set descr 'Run capsicum-test as an unprivileged user'
atf_set require.user unprivileged
}

list_tests()
{
${SRCDIR}/${CAPSICUM_TEST_BIN} --gtest_list_tests | awk '
/^[^ ]/ { CAT=$0 }
/^[ ]/ { print CAT $1}'
test_unprivileged_body() {
atf_check -s exit:0 -o match:PASSED -e ignore \
"$(atf_get_srcdir)/${CAPSICUM_TEST_BIN}" -u "$(id -u)"
}

atf_init_test_cases()
{
local t
for t in `list_tests`; do
add_testcase $t
done
atf_init_test_cases() {
atf_add_test_case test_root
atf_add_test_case test_unprivileged
}

0 comments on commit f305a77

Please sign in to comment.