Some unittests fail: Automated clicks miss their target

[matthijskooijman]

I've been working on making the test suite a bit more usable (Will submit PRs for that next). With some fixes, most of the tests now work fine on my system, except for some of the tests that spawn the GUI and try to click things.

What seems to happen, is that the coordinates for something to be clicked get messed up. E.g. instead of opening a menu, it clicks somewhere in the bar above the menu (not straight above, a bit too far to the left too), failing to open the menu and showing e.g.:

    [junit] Clicking on menu item <javax.swing.JMenu[name='menuEdit', text='Edit', selected=false, enabled=true, visible=true, showing=true]> never showed a pop-up menu
    [junit] org.fest.swing.exception.ActionFailedException: Clicking on menu item <javax.swing.JMenu[name='menuEdit', text='Edit', selected=false, enabled=true, visible=true, showing=true]> never showed a pop-up menu
    [junit]     at org.fest.swing.exception.ActionFailedException.actionFailure(ActionFailedException.java:33)
    [junit]     at org.fest.swing.driver.JMenuItemDriver.ensurePopupIsShowing(JMenuItemDriver.java:137)
    [junit]     at org.fest.swing.driver.JMenuItemDriver.click(JMenuItemDriver.java:73)
    [junit]     at org.fest.swing.driver.JMenuItemDriver.activateParentIfIsAMenu(JMenuItemDriver.java:97)
    [junit]     at org.fest.swing.driver.JMenuItemDriver.show(JMenuItemDriver.java:79)
    [junit]     at org.fest.swing.driver.JMenuItemDriver.click(JMenuItemDriver.java:71)
    [junit]     at org.fest.swing.fixture.JMenuItemFixture.click(JMenuItemFixture.java:84)
    [junit]     at processing.app.ReduceIndentWith1CharOnLastLineTest.shouldJustCloseTheDialog(ReduceIndentWith1CharOnLastLineTest.java:51)
    [junit]     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [junit]     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    [junit]     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    [junit] 
    [junit] Test processing.app.ReduceIndentWith1CharOnLastLineTest FAILED

While waiting for the menu to open, the mouse pointer is somewhat locked in place (I can move it, but it immediately jumps back). It does only click once, it seems.

Normally, my system maximizes windows automatically. When I disable that, the behaviour changes and it clicks some distance to the left of the window, a bit lower than the menu bar, which seems quite random.

This is probably an issue caused by java, which sometimes already behaves weird on my system. When I open up a menu with the window maximized, it opens up over the menu bar rather than below it, and with the window not maximized, the menu pops up to the left of the window and too high. I've also seen in the past that the mouse pointer position does not match the currently highlighted menu item (you would have to move the mouse a bit further down and to the right to select an option).

I thought that updating the swing testing library might help. Currently the "fest" / "fest-swing" library is used, which seems to live here: https://github.com/alexruiz/fest-swing-1.x We use 1.2, and there is an 1.2.1, but that did not help. Also, this project has not seen any updates in 7 years.

I also found assertj, which seems to be a fork that has seen some more fixes and development (though the last commit on assertj-swing is also 3 years old). See https://github.com/joel-costigliola/assertj-swing and https://assertj.github.io/doc/ I started upgrading to this library to see if that helps, but ran into some compilation problems. I might try to finish that, but for now I needed to get on with other things.

At assertj-swing, I did find some other reports of related issues with clicking, sometimes caused by java 11 (which I have I think), multiple monitors (which I am not currently using) or GUI scaling (which I'm not using, this might be Win10-specific too). See assertj/assertj-swing#160, assertj/assertj-swing#158 and assertj/assertj-swing#229.

All this was with Ubuntu 19.10, gnome 3.34 and java 11.0.7, have not tried any other environments.

[cmaglie]

I've already ported the tests to jassert while trying to port the IDE to Java 11.
You may want to look at my java-11 branch here: https://github.com/cmaglie/Arduino/commits/java-11

[cmaglie]

Ah, I also made the tests pass on gh-actions Windows builder and almost did on macosx too, but they fails on... drumrolls... assertj :-). I tried to run on my macosx VM and when the test starts all kinds of permissions dialog pops up (I don't remember exactly which one, they asked something along the lines of "Do you want to allow Arduino to access Documents folder?"). That made the test stop and timeout and the same happens on gh-actions, this is another stopper for automatic GUI testing I think...