Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing GUI tests in GitHub Actions #87

Open
a-niem opened this issue May 29, 2024 · 0 comments
Open

Failing GUI tests in GitHub Actions #87

a-niem opened this issue May 29, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@a-niem
Copy link
Collaborator

a-niem commented May 29, 2024

Description
During automated testing of RodTracker the tests for user interactions with the GUI/whole program fail.
This was not observed when testing locally. So far I was also unable to reproduce it.

List of failed tests:

FAILED tests/test_userinteraction/test_usecases.py::test_typical - AssertionError
FAILED tests/test_userinteraction/test_usecases.py::test_open_rod_after_changes - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_usecases.py::test_undo_after_save - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_usecases.py::test_save_on_unloaded_img - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_position_operations[create rod] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_position_operations[delete rod] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_position_operations[change position] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_number_changing[switch number-abort] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_number_changing[switch number-all] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_number_changing[switch number-one cam] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_number_changing[switch number-one frame] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_number_changing[switch number-this frame/cam] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_length_adjustment[Lengthen-single] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_length_adjustment[Shorten-single] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_length_adjustment[Lengthen-all] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_length_adjustment[Shorten-all] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_display_operations[switch color] - TypeError: 'NoneType' object is not subscriptable
FAILED tests/test_userinteraction/test_useractions.py::test_amendment_operations[undo] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendment_operations[redo] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[undo-CreateRod(25)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[undo-DeleteRod(12)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[undo-ChangeRodPosition(12)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[undo-SwitchRodNumber(12->7)1] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[undo-SwitchRodNumber(12->7)2] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[undo-SwitchRodNumber(12->7)3] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[undo-SwitchRodNumber(12->7)4] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[redo-CreateRod(25)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[redo-DeleteRod(12)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[redo-ChangeRodPosition(12)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[redo-SwitchRodNumber(12->7)1] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[redo-SwitchRodNumber(12->7)2] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[redo-SwitchRodNumber(12->7)3] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_amendmendable_operations[redo-SwitchRodNumber(12->7)4] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-CreateRod(25)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-DeleteRod(12)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-ChangeRodPosition(12)] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-SwitchRodNumber(12->7)0] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-SwitchRodNumber(12->7)1] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-SwitchRodNumber(12->7)2] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-SwitchRodNumber(12->7)3] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-SwitchRodNumber(12->7)4] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-LengthAdjustment(12)0] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-LengthAdjustment(12)1] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-LengthAdjustment(12)2] - TypeError: 'NoneType' object is not iterable
FAILED tests/test_userinteraction/test_useractions.py::test_savable_operations[save-LengthAdjustment(12)3] - TypeError: 'NoneType' object is not iterable

Logs
There appear to be at least 2 failure modes:

  1.  tests/test_userinteraction/test_useractions.py:325: 
     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
     
     self = <gui_actions.LengthAdjustment object at 0x7f702c8f4700>
     main_window = <RodTracker.ui.mainwindow.RodTrackWindow object at 0x7f702c4a4670>
     qtbot = <pytestqt.qtbot.QtBot object at 0x7f702c3f71c0>
     monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f6f3bfd6e20>
     tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_savable_operations_save_L3')
     
         def run(
             self,
             main_window: RodTrackWindow,
             qtbot: QtBot,
             monkeypatch: MonkeyPatch = None,
             tmp_path: pathlib.Path = None,
         ) -> RodTrackWindow:
             if self.assertions:
                 state = aa.pre_length_adjustment(main_window)
             if self.rod is not None:
                 cam_idx = main_window.ui.camera_tabs.currentIndex()
                 cam = main_window.cameras[cam_idx]
                 rods = cam.rods
     >           for rod in rods:
     E           TypeError: 'NoneType' object is not iterable
     
     tests/gui_actions.py:537: TypeError
     ------------------------------ Captured log setup ------------------------------
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:63 /home/runner/.local/share/RodTracker/Session_ngdlf7lh
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:139 (camera_0) (gp3) 25 image file(s) loaded from: /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/RodTracker/resources/example_data/images/gp3
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:139 (camera_1) (gp4) 25 image file(s) loaded from: /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/RodTracker/resources/example_data/images/gp4
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:139 (RodData) Loaded rod file(s) from: /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/RodTracker/resources/example_data/csv
    
  1.   _________________________________ test_typical _________________________________
     
     main_window = <RodTracker.ui.mainwindow.RodTrackWindow object at 0x7f702c9a3ca0>
     qtbot = <pytestqt.qtbot.QtBot object at 0x7f702c4c7fd0>
     monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f702c374070>
     tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_typical0')
     
         @pytest.mark.filterwarnings("ignore:")
         def test_typical(
             main_window: RodTrackWindow,
             qtbot: QtBot,
             monkeypatch: MonkeyPatch,
             tmp_path: pathlib.Path,
         ):
             """Attempt a typical workflow."""
             try:
                 scenario = [
                     ga.OpenData(conftest.csv_data),
                     ga.OpenImage(conftest.cam1_img1),
                     ga.SwitchCamera(),
                     ga.OpenImage(conftest.cam2_img1),
                     ga.SwitchRodNumber(12, 7, lg.NumberChangeActions.ALL_ONE_CAM),
                     ga.SwitchCamera(),
                     ga.SwitchRodNumber(7, 12, lg.NumberChangeActions.CURRENT),
                     ga.SwitchFrame(1),
                     ga.SwitchFrame(1),
                     ga.ChangeRodPosition(12),
                     ga.CreateRod(25),
                     ga.SwitchCamera(),
                     ga.LengthAdjustment(QtCore.Qt.Key_T),
                     ga.SwitchColor("blue"),
                     ga.DeleteRod(12),
                     ga.SaveChanges(),
                 ]
                 for action in scenario:
     >               main_window = action.run(main_window, qtbot, monkeypatch, tmp_path)
     
     tests/test_userinteraction/test_usecases.py:75: 
     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
     tests/gui_actions.py:272: in run
         aa.post_number_switch(
     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
     
     main_window = <RodTracker.ui.mainwindow.RodTrackWindow object at 0x7f702c9a3ca0>
     rod_id = 12, new_id = 7, mode = <NumberChangeActions.ALL_ONE_CAM: 2>
     initial_state = {'dataset':             x1        y1        z1  ...  seen_gp4  particle  color
     0    -5.289254  3.563160  4.213875  ......3779839, 86.78520334109191], 'rod2_pos': [40.8029642065812, 23.430269667032004, 47.53805048682671, 29.433259962428203]}
     
         def post_number_switch(
             main_window: RodTrackWindow,
             rod_id: int,
             new_id: int,
             mode: Union[NumberChangeActions, None],
             initial_state: dict,
         ):
             # currently displayed/loaded
             rod1_new_pos = get_rod_position(main_window, rod_id)
             if mode is not None:
                 rod2_new_pos = get_rod_position(main_window, new_id)
     >           assert rod1_new_pos == initial_state["rod2_pos"]
     E           AssertionError
     
     tests/action_assertions.py:201: AssertionError
     ------------------------------ Captured log setup ------------------------------
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:63 /home/runner/.local/share/RodTracker/Session_ovx08pse
     ------------------------------ Captured log call -------------------------------
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:139 (RodData) Loaded rod file(s) from: /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/RodTracker/resources/example_data/csv
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:139 (camera_0) (gp3) 25 image file(s) loaded from: /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/RodTracker/resources/example_data/images/gp3
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:139 (camera_1) (gp4) 25 image file(s) loaded from: /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/RodTracker/resources/example_data/images/gp4
     INFO     RodTracker.ui.loggerwidget:loggerwidget.py:139 (gp4, 500, ) Changed rod #12 ---> #7 of color  in frames >= 500 of gp4.
    
@a-niem a-niem added the bug Something isn't working label May 29, 2024
a-niem added a commit that referenced this issue Jul 1, 2024
Temporarily deactivate minimum test coverage constraint until issue #87 is resolved.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant