New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement: fix flaky tests #3009

Open
stapelberg opened this Issue Oct 1, 2017 · 2 comments

Comments

Projects
None yet
4 participants
@stapelberg
Member

stapelberg commented Oct 1, 2017

I got the following tests to fail while running the testsuite in a loop while stress-ng is running (ordered by frequency):

  • testcases/t/133-size-hints.t
  • testcases/t/293-focus-follows-mouse.t
#   Failed test 'second floating still on top'
#   at /usr/src/i3/build/../../i3/testcases/t/293-focus-follows-mouse.t line 85.
#          got: '6291461'
#     expected: '6291462'
#   Failed test 'first floating still behind'
#   at /usr/src/i3/build/../../i3/testcases/t/293-focus-follows-mouse.t line 86.
#          got: '6291462'
#     expected: '6291461'
  • Sometimes i3 cannot change the root window’s event mask (X11 error BadAccess), implying that the old i3 process on that x server hasn’t yet been cleaned up (even though we wait for the i3 process to terminate). We should look into sending SIGHUP to the X server, and possibly polling until the I3_* atoms have been cleared.

If there are any others which fail (even just occasionally), please mention them in this issue so that we can work on a fix.

@stapelberg stapelberg self-assigned this Oct 1, 2017

@i3bot i3bot added the enhancement label Oct 1, 2017

stapelberg added a commit to stapelberg/X11-XCB that referenced this issue Oct 3, 2017

Bugfix: use Newxz to zero-allocate size hints
Previously, the size hints data structure could contain arbitrary data,
resulting in i3/testcases/t/133-size-hints.t being flaky.

related to i3/i3#3009
@orestisf1993

This comment has been minimized.

Show comment
Hide comment
@orestisf1993

orestisf1993 Mar 25, 2018

Member

t/525-i3bar-mouse-bindings.t:

ok 1 - Connect to X11, verify XKB and XTEST are present (via Inline::C)
ok 2 - i3bar present
# i3bar window = 10485766
# Subtest: button 1 moves focus left
    ok 1 - tick nonce received
    ok 2 - button 1 moves focus left
    1..2
ok 3 - button 1 moves focus left
# Subtest: button 2 moves focus right
    ok 1 - tick nonce received
    ok 2 - button 2 moves focus right
    1..2
ok 4 - button 2 moves focus right
# Subtest: button 3 moves focus left
    ok 1 - tick nonce received
    not ok 2 - button 3 moves focus left

    #   Failed test 'button 3 moves focus left'
    #   at /home/orestis/Documents/programming/i3/build/../../i3/testcases/t/525-i3bar-mouse-bindings.t line 100.
    #     Structures begin differing at:
    #          $got->[0] = Does not exist
    #     $expected->[0] = '4194304'
    1..2
    # Looks like you failed 1 test of 2.
not ok 5 - button 3 moves focus left

#   Failed test 'button 3 moves focus left'
#   at /home/orestis/Documents/programming/i3/build/../../i3/testcases/t/525-i3bar-mouse-bindings.t line 127.
# Subtest: button 4 moves focus right
    ok 1 - tick nonce received
    ok 2 - button 4 moves focus right
    1..2
ok 6 - button 4 moves focus right
# Subtest: button 5 moves focus left
    ok 1 - tick nonce received
    ok 2 - button 5 moves focus left
    1..2
ok 7 - button 5 moves focus left
# Subtest: button 6 does not move focus while pressed
    ok 1 - tick nonce received
    ok 2 - button 6 does not move focus while pressed
    1..2
ok 8 - button 6 does not move focus while pressed
ok 9 - focus unchanged
# Subtest: button 6 release moves focus right
    ok 1 - tick nonce received
    ok 2 - button 6 release moves focus right
    1..2
ok 10 - button 6 release moves focus right
# Subtest: button 7 press moves focus left
    ok 1 - tick nonce received
    ok 2 - button 7 press moves focus left
    1..2
ok 11 - button 7 press moves focus left
# Subtest: button 7 release moves focus right
    ok 1 - tick nonce received
    ok 2 - button 7 release moves focus right
    1..2
ok 12 - button 7 release moves focus right
1..12

Convenient while loop:

cd build
while testcases/complete-run.pl t/525-i3bar-mouse-bindings.t
do
done
Member

orestisf1993 commented Mar 25, 2018

t/525-i3bar-mouse-bindings.t:

ok 1 - Connect to X11, verify XKB and XTEST are present (via Inline::C)
ok 2 - i3bar present
# i3bar window = 10485766
# Subtest: button 1 moves focus left
    ok 1 - tick nonce received
    ok 2 - button 1 moves focus left
    1..2
ok 3 - button 1 moves focus left
# Subtest: button 2 moves focus right
    ok 1 - tick nonce received
    ok 2 - button 2 moves focus right
    1..2
ok 4 - button 2 moves focus right
# Subtest: button 3 moves focus left
    ok 1 - tick nonce received
    not ok 2 - button 3 moves focus left

    #   Failed test 'button 3 moves focus left'
    #   at /home/orestis/Documents/programming/i3/build/../../i3/testcases/t/525-i3bar-mouse-bindings.t line 100.
    #     Structures begin differing at:
    #          $got->[0] = Does not exist
    #     $expected->[0] = '4194304'
    1..2
    # Looks like you failed 1 test of 2.
not ok 5 - button 3 moves focus left

#   Failed test 'button 3 moves focus left'
#   at /home/orestis/Documents/programming/i3/build/../../i3/testcases/t/525-i3bar-mouse-bindings.t line 127.
# Subtest: button 4 moves focus right
    ok 1 - tick nonce received
    ok 2 - button 4 moves focus right
    1..2
ok 6 - button 4 moves focus right
# Subtest: button 5 moves focus left
    ok 1 - tick nonce received
    ok 2 - button 5 moves focus left
    1..2
ok 7 - button 5 moves focus left
# Subtest: button 6 does not move focus while pressed
    ok 1 - tick nonce received
    ok 2 - button 6 does not move focus while pressed
    1..2
ok 8 - button 6 does not move focus while pressed
ok 9 - focus unchanged
# Subtest: button 6 release moves focus right
    ok 1 - tick nonce received
    ok 2 - button 6 release moves focus right
    1..2
ok 10 - button 6 release moves focus right
# Subtest: button 7 press moves focus left
    ok 1 - tick nonce received
    ok 2 - button 7 press moves focus left
    1..2
ok 11 - button 7 press moves focus left
# Subtest: button 7 release moves focus right
    ok 1 - tick nonce received
    ok 2 - button 7 release moves focus right
    1..2
ok 12 - button 7 release moves focus right
1..12

Convenient while loop:

cd build
while testcases/complete-run.pl t/525-i3bar-mouse-bindings.t
do
done
@stapelberg

This comment has been minimized.

Show comment
Hide comment
@stapelberg

stapelberg Mar 30, 2018

Member

Thanks for the report, I can reproduce the issue. Looking into it now.

Member

stapelberg commented Mar 30, 2018

Thanks for the report, I can reproduce the issue. Looking into it now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment