Add test to check basic interactions with layer controls widgets #5757

dalthviz · 2023-04-21T19:46:31Z

Fixes/Closes

Initial effort towards improving GUI testing automation: https://napari.zulipchat.com/#narrow/stream/212875-general/topic/GUI.20testing

Description

This is an initial attempt to add more tests for code related with GUI elements. The new parametrized test proposed creates an instance of a layer (Image, Labels, Points, Shapes, Surface, Tracks or Vectors) and then operates over the layer control widgets simulating user usage (QCombox, QSpinBox and QSlider based widgets value changes, clicking QCheckbox and buttons widgets, QColorSwatch widgets color changes typing colors, etc)

This also fixes a little issue that I noticed while working on the test. The error is related with the value of the color mode combobox for a Vectors layer controls. The combobox was not showing the correct value after a fallback to a valid value is done (when a ValueError exception is raised, the color mode value of the layer is changed to the previous one, but the combobox value was missing being updated too):

Before:

After:

Besides that, the test structure also has some logic to skip some tracebacks from issues which are still open/were discovered while writing the test like #5798 and #4864

References

Type of change

Add more tests

How has this been tested?

all tests pass with my change
I check if my changes works with both PySide and PyQt backends
as there are small differences between the two Qt bindings.

Final checklist:

My PR is the minimum possible work for the desired functionality
I have commented my code, particularly in hard-to-understand areas

alisterburt

Great stuff @dalthviz - thank you

Would love to see this in but the test failures on pyside look real
https://github.com/napari/napari/actions/runs/4768301656/jobs/8477468793?pr=5757

Any ideas how we could get around those?

dalthviz · 2023-04-24T15:25:16Z

Hi @alisterburt, thank you for giving this a check! I think the problem comes from PySide2/6 not having the findChildren signature that supports passing a tuple of types (which seems like is only available for PyQt 5/6 bindings). Will push a fix for that 👍

codecov · 2023-04-24T15:49:55Z

Codecov Report

Merging #5757 (32c078c) into main (d91f4ee) will increase coverage by 0.14%.
The diff coverage is 78.01%.

@@            Coverage Diff             @@
##             main    #5757      +/-   ##
==========================================
+ Coverage   89.79%   89.93%   +0.14%     
==========================================
  Files         614      614              
  Lines       52196    52327     +131     
==========================================
+ Hits        46870    47062     +192     
+ Misses       5326     5265      -61

Impacted Files	Coverage Δ
napari/_qt/layer_controls/qt_vectors_controls.py	`80.62% <0.00%> (+7.96%)`	⬆️
...qt/layer_controls/_tests/test_qt_layer_controls.py	`87.34% <78.57%> (-12.66%)`	⬇️

... and 19 files with indirect coverage changes

…layer control

…ndom values

… stderr

kephale · 2023-05-02T11:29:53Z

Looks great!

@kne42 any thoughts on this one? In particular, are there some other parts of the GUI you were thinking needed more automated testing?

… are related to known unresolved issues and undo range changes for spinbox based widgets for the labels controls

GenevieveBuckley · 2023-05-10T03:21:25Z

napari/_qt/layer_controls/_tests/test_qt_layer_controls.py

+                expected_errors = [
+                    "MemoryError: Unable to allocate",  # See https://github.com/napari/napari/issues/5798
+                    "ValueError: array is too big; `arr.size * arr.dtype.itemsize` is larger than the maximum possible size.",  # See https://github.com/napari/napari/issues/5798
+                    "ValueError: Maximum allowed dimension exceeded",  # See https://github.com/napari/napari/issues/5798
+                    "IndexError: index ",  # See https://github.com/napari/napari/issues/4864
+                    "RuntimeWarning: overflow encountered",  # See https://github.com/napari/napari/issues/4864
+                ]


Can you make a note in each of these issues that any PR to close them should also remove these lines from this test? (You might have to do it after this PR is merged, in order to link to the line(s) directly)

Thanks for the feedback! Will do 👍

GenevieveBuckley · 2023-05-16T00:01:04Z

Nice work! Thank you @dalthviz 🎉

If you can follow up on this suggestion that would be fantastic.

Initial effort towards improving GUI testing automation: https://napari.zulipchat.com/#narrow/stream/212875-general/topic/GUI.20testing This is an initial attempt to add more tests for code related with GUI elements. The new parametrized test proposed creates an instance of a layer (`Image`, `Labels`, `Points`, `Shapes`, `Surface`, `Tracks` or `Vectors`) and then operates over the layer control widgets simulating user usage (`QCombox`, `QSpinBox` and `QSlider` based widgets value changes, clicking `QCheckbox` and buttons widgets, `QColorSwatch` widgets color changes typing colors, etc) This also fixes a little issue that I noticed while working on the test. The error is related with the value of the color mode combobox for a Vectors layer controls. The combobox was not showing the correct value after a fallback to a valid value is done (when a `ValueError` exception is raised, the color mode value of the layer is changed to the previous one, but the combobox value was missing being updated too): * Before: ![vectors_before](https://user-images.githubusercontent.com/16781833/234071199-5d22d1b4-0653-489a-be92-68ca138033a0.gif) * After: ![vectors_after](https://user-images.githubusercontent.com/16781833/234070708-52a41085-eea8-4cd9-bcde-03c5b71395b5.gif) Besides that, the test structure also has some logic to skip some tracebacks from issues which are still open/were discovered while writing the test like #5798 and #4864  - [x] Add more tests  - [x] all tests pass with my change - [x] I check if my changes works with both PySide and PyQt backends as there are small differences between the two Qt bindings. - [x] My PR is the minimum possible work for the desired functionality - [x] I have commented my code, particularly in hard-to-understand areas

# Fixes/Closes Initial effort towards improving GUI testing automation: https://napari.zulipchat.com/#narrow/stream/212875-general/topic/GUI.20testing # Description This is an initial attempt to add more tests for code related with GUI elements. The new parametrized test proposed creates an instance of a layer (`Image`, `Labels`, `Points`, `Shapes`, `Surface`, `Tracks` or `Vectors`) and then operates over the layer control widgets simulating user usage (`QCombox`, `QSpinBox` and `QSlider` based widgets value changes, clicking `QCheckbox` and buttons widgets, `QColorSwatch` widgets color changes typing colors, etc) This also fixes a little issue that I noticed while working on the test. The error is related with the value of the color mode combobox for a Vectors layer controls. The combobox was not showing the correct value after a fallback to a valid value is done (when a `ValueError` exception is raised, the color mode value of the layer is changed to the previous one, but the combobox value was missing being updated too): * Before: ![vectors_before](https://user-images.githubusercontent.com/16781833/234071199-5d22d1b4-0653-489a-be92-68ca138033a0.gif) * After: ![vectors_after](https://user-images.githubusercontent.com/16781833/234070708-52a41085-eea8-4cd9-bcde-03c5b71395b5.gif) Besides that, the test structure also has some logic to skip some tracebacks from issues which are still open/were discovered while writing the test like #5798 and #4864 ## Type of change  - [x] Add more tests # How has this been tested?  - [x] all tests pass with my change - [x] I check if my changes works with both PySide and PyQt backends as there are small differences between the two Qt bindings. ## Final checklist: - [x] My PR is the minimum possible work for the desired functionality - [x] I have commented my code, particularly in hard-to-understand areas

dalthviz added the tests Something related to our tests label Apr 21, 2023

dalthviz self-assigned this Apr 21, 2023

github-actions bot added the qt Relates to qt label Apr 21, 2023

dalthviz added the maintenance PR with maintance changes, label Apr 21, 2023

alisterburt reviewed Apr 22, 2023

View reviewed changes

Add test to check basic interactions with layer controls widgets

2a25e97

dalthviz force-pushed the layer_controls_tests branch from 122f47a to 2a25e97 Compare April 24, 2023 15:25

Add parameters to test also Surface and Tracks layers

4958009

Undo signal blocker comment on color mode fallback logic for Vectors …

73a9df7

…layer control

dalthviz marked this pull request as ready for review April 24, 2023 17:38

dalthviz marked this pull request as draft April 25, 2023 15:34

Use fixture to create layer and layer controls. Set numpy seed for ra…

9c69bb8

…ndom values

dalthviz marked this pull request as ready for review April 25, 2023 17:03

dalthviz marked this pull request as draft April 27, 2023 19:26

dalthviz added 5 commits April 28, 2023 13:09

Add section to test spinbox based widgets. Add asserts for stdout and…

5b19ddb

… stderr

Handle OverflowError on linux

aa18399

Testing

e8ba5a6

Merge branch 'main' into layer_controls_tests

da1ff70

Fix range generation for spinbox test

5cca432

Merge branch 'main' into layer_controls_tests

f9bf7e0

This was referenced May 2, 2023

IndexError when changing selected label on labels layer #4864

Closed

ValueError and MemoryError raise when setting a big value as label contour #5798

Open

dalthviz added 5 commits May 2, 2023 10:52

Validate spinbox based widgets possible error messages to see if they…

631c855

… are related to known unresolved issues and undo range changes for spinbox based widgets for the labels controls

Linux testing

1f03013

Linux testing

b015f7c

Linux testing

6be3ec8

Skip for min req setup (Python error: Aborted)

ca93fb8

dalthviz added 2 commits May 2, 2023 13:11

MacOS testing

a26d8d3

Linux testing

32c078c

dalthviz marked this pull request as ready for review May 2, 2023 18:45

GenevieveBuckley reviewed May 10, 2023

View reviewed changes

GenevieveBuckley approved these changes May 10, 2023

View reviewed changes

GenevieveBuckley merged commit 39b42e6 into napari:main May 15, 2023
32 checks passed

Czaki mentioned this pull request Jun 7, 2023

v0.4.18 - release process #5911

Closed

Czaki added this to the 0.5.0 milestone Jun 16, 2023

Czaki added the triaged-0.4.18 To mark PR that is triaged in 0.4.18 release process label Jun 16, 2023

Czaki modified the milestones: 0.5.0, 0.4.18 Jun 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add test to check basic interactions with layer controls widgets #5757

Add test to check basic interactions with layer controls widgets #5757

dalthviz commented Apr 21, 2023 •

edited

Loading

alisterburt left a comment

dalthviz commented Apr 24, 2023

codecov bot commented Apr 24, 2023 •

edited

Loading

kephale commented May 2, 2023

GenevieveBuckley May 10, 2023

dalthviz May 10, 2023

GenevieveBuckley commented May 16, 2023

Add test to check basic interactions with layer controls widgets #5757

Add test to check basic interactions with layer controls widgets #5757

Conversation

dalthviz commented Apr 21, 2023 • edited Loading

Fixes/Closes

Description

References

Type of change

How has this been tested?

Final checklist:

alisterburt left a comment

Choose a reason for hiding this comment

dalthviz commented Apr 24, 2023

codecov bot commented Apr 24, 2023 • edited Loading

Codecov Report

kephale commented May 2, 2023

GenevieveBuckley May 10, 2023

Choose a reason for hiding this comment

dalthviz May 10, 2023

Choose a reason for hiding this comment

GenevieveBuckley commented May 16, 2023

dalthviz commented Apr 21, 2023 •

edited

Loading

codecov bot commented Apr 24, 2023 •

edited

Loading