1.5.24: test suite freezes

[kloczek]

fragment of the ps auxwf output:

tkloczko 2892579  0.0  0.0 222196  3392 pts/12   S+   08:48   0:00              \_ /bin/sh -e /var/tmp/rpm-tmp.8RlqOO
tkloczko 2892580  0.0  0.0   3996  3344 pts/12   S+   08:48   0:00                  \_ /usr/bin/sh /usr/bin/xvfb-run -a /usr/bin/make -O -j48 V=1 VERBOSE=1 check DISABLE_GUI_TESTS=ibus-keypress
tkloczko 2892590  0.0  0.0 2794376 51420 pts/12  Sl+  08:48   0:00                      \_ Xvfb :100 -screen 0 640x480x24 -nolisten tcp
tkloczko 2892630  0.0  0.0   2864  2196 pts/12   S+   08:48   0:00                      \_ /usr/bin/make -O -j48 V=1 VERBOSE=1 check DISABLE_GUI_TESTS=ibus-keypress
tkloczko 2892631  0.0  0.0   4000  3116 pts/12   S+   08:48   0:00                          \_ /bin/sh -c fail=; \ if (target_option=k; case ${target_option-} in ?) ;; *) echo "am__make_running_with_option: internal error: invalid" "target option '${target_option-}' specified" >&2; exit 1;; esac; has_opt=no; sane_makeflags=$MAKEFLAGS; if { if test -z '0'; then false; elif test -n 'x86_64-redhat-linux-gnu'; then true; elif test -n '4.3' && test -n '/home/tkloczko/rpmbuild/BUILD/ibus-1.5.24'; then true; else false; fi; }; then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in *\\[\ \?]*) bs=\\; sane_makeflags=`printf '%s\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs $bs?]*//g"`;; esac; fi; skip_next=no; strip_trailopt () { flg=`printf '%s\n' "$flg" | sed "s/$1.*$//"`; }; for flg in $sane_makeflags; do test $skip_next = yes && { skip_next=no; continue; }; case $flg in *=*|--*) continue;; -*I) strip_trailopt 'I'; skip_next=yes;; -*I?*) strip_trailopt 'I';; -*O) strip_trailopt 'O'; skip_next=yes;; -*O?*) strip_trailopt 'O';; -*l) strip_trailopt 'l'; skip_next=yes;; -*l?*) strip_trailopt 'l';; -[dEDm]) skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in *$target_option*) has_opt=yes; break;; esac; done; test $has_opt = yes); then \   failcom='fail=yes'; \ else \   failcom='exit 1'; \ fi; \ dot_seen=no; \ target=`echo check-recursive | sed s/-recursive//`; \ case "check-recursive" in \   distclean-* | maintainer-clean-*) list='src bindings util conf client portal data m4 po docs engine ui tools bus ibus setup' ;; \   *) list='src bindings util conf client portal data m4 po docs engine ui tools bus   setup ' ;; \ esac; \ for subdir in $list; do \   echo "Making $target in $subdir"; \   if test "$subdir" = "."; then \     dot_seen=yes; \     local_target="$target-am"; \   else \     local_target="$target"; \   fi; \   (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && /usr/bin/make  $local_target) \   || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \   /usr/bin/make  "$target-am" || exit 1; \ fi; test -z "$fail"
tkloczko 2892639  0.0  0.0   4000  1832 pts/12   S+   08:48   0:00                              \_ /bin/sh -c fail=; \ if (target_option=k; case ${target_option-} in ?) ;; *) echo "am__make_running_with_option: internal error: invalid" "target option '${target_option-}' specified" >&2; exit 1;; esac; has_opt=no; sane_makeflags=$MAKEFLAGS; if { if test -z '0'; then false; elif test -n 'x86_64-redhat-linux-gnu'; then true; elif test -n '4.3' && test -n '/home/tkloczko/rpmbuild/BUILD/ibus-1.5.24'; then true; else false; fi; }; then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in *\\[\ \?]*) bs=\\; sane_makeflags=`printf '%s\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs $bs?]*//g"`;; esac; fi; skip_next=no; strip_trailopt () { flg=`printf '%s\n' "$flg" | sed "s/$1.*$//"`; }; for flg in $sane_makeflags; do test $skip_next = yes && { skip_next=no; continue; }; case $flg in *=*|--*) continue;; -*I) strip_trailopt 'I'; skip_next=yes;; -*I?*) strip_trailopt 'I';; -*O) strip_trailopt 'O'; skip_next=yes;; -*O?*) strip_trailopt 'O';; -*l) strip_trailopt 'l'; skip_next=yes;; -*l?*) strip_trailopt 'l';; -[dEDm]) skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in *$target_option*) has_opt=yes; break;; esac; done; test $has_opt = yes); then \   failcom='fail=yes'; \ else \   failcom='exit 1'; \ fi; \ dot_seen=no; \ target=`echo check-recursive | sed s/-recursive//`; \ case "check-recursive" in \   distclean-* | maintainer-clean-*) list='src bindings util conf client portal data m4 po docs engine ui tools bus ibus setup' ;; \   *) list='src bindings util conf client portal data m4 po docs engine ui tools bus   setup ' ;; \ esac; \ for subdir in $list; do \   echo "Making $target in $subdir"; \   if test "$subdir" = "."; then \     dot_seen=yes; \     local_target="$target-am"; \   else \     local_target="$target"; \   fi; \   (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && /usr/bin/make  $local_target) \   || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \   /usr/bin/make  "$target-am" || exit 1; \ fi; test -z "$fail"
tkloczko 2892640  0.0  0.0   3024  2412 pts/12   S+   08:48   0:00                                  \_ /usr/bin/make check
tkloczko 2892641  0.0  0.0   3024  2412 pts/12   S+   08:48   0:00                                      \_ /usr/bin/make check-recursive
tkloczko 2892642  0.0  0.0   4000  3300 pts/12   S+   08:48   0:00                                          \_ /bin/sh -c fail=; \ if (target_option=k; case ${target_option-} in ?) ;; *) echo "am__make_running_with_option: internal error: invalid" "target option '${target_option-}' specified" >&2; exit 1;; esac; has_opt=no; sane_makeflags=$MAKEFLAGS; if { if test -z '2'; then false; elif test -n 'x86_64-redhat-linux-gnu'; then true; elif test -n '4.3' && test -n '/home/tkloczko/rpmbuild/BUILD/ibus-1.5.24/src'; then true; else false; fi; }; then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in *\\[\ \?]*) bs=\\; sane_makeflags=`printf '%s\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs $bs?]*//g"`;; esac; fi; skip_next=no; strip_trailopt () { flg=`printf '%s\n' "$flg" | sed "s/$1.*$//"`; }; for flg in $sane_makeflags; do test $skip_next = yes && { skip_next=no; continue; }; case $flg in *=*|--*) continue;; -*I) strip_trailopt 'I'; skip_next=yes;; -*I?*) strip_trailopt 'I';; -*O) strip_trailopt 'O'; skip_next=yes;; -*O?*) strip_trailopt 'O';; -*l) strip_trailopt 'l'; skip_next=yes;; -*l?*) strip_trailopt 'l';; -[dEDm]) skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in *$target_option*) has_opt=yes; break;; esac; done; test $has_opt = yes); then \   failcom='fail=yes'; \ else \   failcom='exit 1'; \ fi; \ dot_seen=no; \ target=`echo check-recursive | sed s/-recursive//`; \ case "check-recursive" in \   distclean-* | maintainer-clean-*) list='. tests' ;; \   *) list='. tests' ;; \ esac; \ for subdir in $list; do \   echo "Making $target in $subdir"; \   if test "$subdir" = "."; then \     dot_seen=yes; \     local_target="$target-am"; \   else \     local_target="$target"; \   fi; \   (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && /usr/bin/make  $local_target) \   || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \   /usr/bin/make  "$target-am" || exit 1; \ fi; test -z "$fail"
tkloczko 2892652  0.0  0.0   4000  1808 pts/12   S+   08:48   0:00                                              \_ /bin/sh -c fail=; \ if (target_option=k; case ${target_option-} in ?) ;; *) echo "am__make_running_with_option: internal error: invalid" "target option '${target_option-}' specified" >&2; exit 1;; esac; has_opt=no; sane_makeflags=$MAKEFLAGS; if { if test -z '2'; then false; elif test -n 'x86_64-redhat-linux-gnu'; then true; elif test -n '4.3' && test -n '/home/tkloczko/rpmbuild/BUILD/ibus-1.5.24/src'; then true; else false; fi; }; then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in *\\[\ \?]*) bs=\\; sane_makeflags=`printf '%s\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs $bs?]*//g"`;; esac; fi; skip_next=no; strip_trailopt () { flg=`printf '%s\n' "$flg" | sed "s/$1.*$//"`; }; for flg in $sane_makeflags; do test $skip_next = yes && { skip_next=no; continue; }; case $flg in *=*|--*) continue;; -*I) strip_trailopt 'I'; skip_next=yes;; -*I?*) strip_trailopt 'I';; -*O) strip_trailopt 'O'; skip_next=yes;; -*O?*) strip_trailopt 'O';; -*l) strip_trailopt 'l'; skip_next=yes;; -*l?*) strip_trailopt 'l';; -[dEDm]) skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in *$target_option*) has_opt=yes; break;; esac; done; test $has_opt = yes); then \   failcom='fail=yes'; \ else \   failcom='exit 1'; \ fi; \ dot_seen=no; \ target=`echo check-recursive | sed s/-recursive//`; \ case "check-recursive" in \   distclean-* | maintainer-clean-*) list='. tests' ;; \   *) list='. tests' ;; \ esac; \ for subdir in $list; do \   echo "Making $target in $subdir"; \   if test "$subdir" = "."; then \     dot_seen=yes; \     local_target="$target-am"; \   else \     local_target="$target"; \   fi; \   (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && /usr/bin/make  $local_target) \   || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \   /usr/bin/make  "$target-am" || exit 1; \ fi; test -z "$fail"
tkloczko 2892653  0.0  0.0   2860  2292 pts/12   S+   08:48   0:00                                                  \_ /usr/bin/make check
tkloczko 2892654  0.0  0.0   2860  2340 pts/12   S+   08:48   0:00                                                      \_ /usr/bin/make check-TESTS
tkloczko 2892660  0.0  0.0   3868  3060 pts/12   S+   08:48   0:00                                                          \_ /bin/sh -c set +e; bases='ibus-bus.log ibus-config.log ibus-configservice.log ibus-factory.log ibus-inputcontext.log ibus-inputcontext-create.log ibus-keynames.log ibus-registry.log ibus-serializable.log ibus-share.log ibus-util.log ibus-engine-switch.log ibus-compose.log ibus-keypress.log'; bases=`for i in $bases; do echo $i; done | sed 's/\.log$//'`; bases=`echo $bases`; \ log_list=`for i in $bases; do echo $i.log; done`; \ trs_list=`for i in $bases; do echo $i.trs; done`; \ log_list=`echo $log_list`; trs_list=`echo $trs_list`; \ /usr/bin/make  test-suite.log TEST_LOGS="$log_list"; \ exit $?;
tkloczko 2892669  0.0  0.0   3112  2460 pts/12   S+   08:48   0:00                                                              \_ /usr/bin/make test-suite.log TEST_LOGS=ibus-bus.log ibus-config.log ibus-configservice.log ibus-factory.log ibus-inputcontext.log ibus-inputcontext-create.log ibus-keynames.log ibus-registry.log ibus-serializable.log ibus-share.log ibus-util.log ibus-engine-switch.log ibus-compose.log ibus-keypress.log
tkloczko 2892707  0.0  0.0   3868  2988 pts/12   S+   08:48   0:00                                                                  \_ /bin/sh ../../test-driver --test-name ibus-compose --log-file ibus-compose.log --trs-file ibus-compose.trs --color-tests no --enable-hard-errors yes --expect-failure no -- ./runtest ./ibus-compose
tkloczko 2892748  0.0  0.0 222332  3412 pts/12   S+   08:48   0:00                                                                      \_ /bin/sh ./runtest ./ibus-compose
tkloczko 2893934  0.0  0.0 3323196 67576 pts/12  Sl+  08:48   0:00                                                                          \_ /home/tkloczko/rpmbuild/BUILD/ibus-1.5.24/src/tests/.libs/lt-ibus-compose ../../../src/tests

and strace of the last process:

[tkloczko@barrel SPECS]$ strace -p 2893934
strace: Process 2893934 attached
restart_syscall(<... resuming interrupted read ...>

[fujiwarat]

tkloczko 2893934 0.0 0.0 3323196 67576 pts/12 Sl+ 08:48 0:00 _ /home/tkloczko/rpmbuild/BUILD/ibus-1.5.24/src/tests/.libs/lt-ibus-compose ../../../src/tests

ibus-compose requires to open display.

You could disable GUI tests if you cannot configure display correctly.

% make DISABLE_GUI_TESTS="ibus-compose ibus-keypress test-stress" check

[kloczek]

Issue is that I'm running test suite with $DSPLAY
Quote from my ibus.spec file:

%check
%{!?with_check:exit 0}
xvfb-run -a \
%make_build check \
          DISABLE_GUI_TESTS="ibus-keypress"

[fujiwarat]

OK, you already setup Display. Unfortunately you also need to setup a window manager since IBus needs to receive focus-in/out events to test IBus with GTK applications.
I think make check works if you run it in your desktop environment.
If you run it in a build system, probably I think it would be good to include ibus-compose to DISABLE_GUI_TESTS because the build system does not suppose to run desktop environments.

I run ibus-compose using gnome-desktop-testing-runner & ibus-desktop-testing-runner in Fedora CI but not Fedora build
https://src.fedoraproject.org/rpms/ibus/blob/rawhide/f/tests/roles/ibus-desktop-testing-role/tasks/main.yml
https://src.fedoraproject.org/rpms/ibus/blob/rawhide/f/ibus.spec#_369

[kloczek]

I think make check works if you run it in your desktop environment.

Issue is that it makes test suite useless in case typical build used on packaging software which uses env without human interraction.

[fujiwarat]

So you can use DISABLE_GUI_TESTS. I don't expect all users run ibus-compose.

[Apteryks]

@fujiwarat It'd be nice if we could declare that we want to skip all GUI tests, instead of having to know in advance which ones need to be added to DISABLE_GUI_TESTS. Something like: ./configure --disable-gui-tests. It seems ibus-engine-switch need to be added to the list mentioned above.

[fujiwarat]

Probably DISABLE_GUI_TESTS values are different by your environments.
If display is available but window manager is not, ibus-engine-switch works but ibus-compose does not.
So if we give the configure option, the option needs to have the argument likes --disable-gui-tests="ibus-compose ibus-keypress test-stress xkb-latin-layouts" and DISABLE_GUI_TESTS might be more useful not to require the rebuilt with configure.

[kloczek]

May I ask why you want to skip thise GUI test?
If they are broken they shpold be fixed .. isn't it?

[fujiwarat]

If they are broken they shpold be fixed .. isn't it?

When you enable window manager, the test should works.

[Apteryks]

Using the latest release, I could disable the problematic tests with DISABLE_GUI_TESTS=ibus-compose ibus-inputcontext-create xkb-latin-layouts . I tried running fluxbox or ratpoison on the same DISPLAY to see if that could work in the build container, but helas, it didn't.

[kloczek]

Using the latest release, I could disable the problematic tests with DISABLE_GUI_TESTS=ibus-compose ibus-inputcontext-create xkb-latin-layouts . I tried running fluxbox or ratpoison on the same DISPLAY to see if that could work in the build container, but helas, it didn't.

Just in case .. if you will look on top of this ticket on ps output u=you can find that I'm running test suite as xvfb-run -a parameter so $DISPLAY is provided 😋

[fujiwarat]

As I also noted above, openning display is not enough but a window manager is required for those testings.
Using DISABLE_GUI_TESTS is the easy way for you.

[kloczek]

OK so what exacly should be done to have for example FB session for test suite session? 🤔
Do you have proposition of exact steps which could be used during the test suite execution?

[fujiwarat]

ibus-desktop-testing-runner can run gnome-shell as a window manager and executes the tests using gnome-desktop-testring-runner.

Fedora CI runs ibus-desktop-testing-runner and those GUI test cases.