Overlays 2.0

[brisvag]

Description

This PR overhauls the "overlay" concept by unifying their behaviour and implementing a framework that allows us to expand it more easily and eventually expose it to plugins.

Here's a summary of the changes to help you navigate the diff:

General idea:

1. Overlays are no longer special cased and handled all through QtViewer. Instead, they go through a machinery very similar to that of Layers, where an VispyOverlay object gets autogenerated from an Overlay evented model by the overlay_to_visual in _vispy.utils.visual.py. This object takes care of hooking up events to the actual vispy backend visualisation.
2. Overlays can differ on 2 axes which can be combined in any way:
   • canvas/scene: canvas overlays live in canvas space (2D, unaffected by world transforms), while scene overlays live in scene space (2D or 3D, "displayed world")
   • viewer/layer: viewer overlays (scale_bar, text, axes) live on the ViewerModel, while layer overlays (bounding_box, transform_box, selection_box) live on each Layer.
3. I tried to improve the split between the "visual" logic and the "napari-to-vispy" logic in the _vispy.visuals and _vispy.overlays modules respectively, to reflect what we have for layers.
4. Both on ViewerModel and Layer they are kept in an EventedDict called _overlays. The ones meant for public use are exposed through properties.

Changes in individual overlays:

• text, scale_bar and axes are mostly unchanged, except for additional parameters such as opacity and order. They are exposed publicly on the viewer through properties, so this is basically backwards compatible.
• bounding_box is new, see Layer bounding box #4849. It's exposed publicly on every layer as layer.bounding_box and disabled by default.
• interaction_box was the main blocker here, cause it used a completely custom machinery. I had to remove it and reimplement it using the new system. It was completely removed from where it used to be (viewer.overlays.interaction_box), and is now a relatively simple overlay called transform_box (not publicly exposed, for now). The whole interactive transform logic was moved to proper layer callbacks, rather than using a custom stateful object on QtViewer. (it now lives in layers.base._base_mouse_bindings.py). This simplified the code a fair bit, but in turn I had to make changes to:
  • allow every layer to change mode and use the transform mode (previouysly only image could do it)
  • fix action manager and qt layer controls where they broke due to this.
  • maybe TODO: make transform mode simply non-selectable in 3D?
• selection_box is using the same overlay machinery from transform_box, but it's intended to replace the custom selection logic that Points and other layers implement with a unified system, while also allowing to transform the selected object (like interaction_box was meant to do before, at least for points). This is not yet implemented, but I think it should happen in a followup PR. @jojoelfe: as the one who implemented this the first time around, how do you feel about these changes? Particularly, what did I break that you might rely on?

Of course I'm also missing some dosctrings and comments, but I'll get around to it soon :)

Here's a snippet to test out everything:

import napari
import numpy as np

v = napari.Viewer(ndisplay=3)
img = v.add_image(np.random.rand(20, 10, 15), scale=[5, 15, 10])
pts = v.add_points(np.random.rand(20, 3) * 200, translate=[20, -10])

v.scale_bar.visible = True
v.axes.visible = True
v.text_overlay.visible = True
v.text_overlay.text = 'Cool stuff!'
v.text_overlay.position = 'top_center'

img.bounding_box.visible = True
img.bounding_box.line_color = 'green'
img.bounding_box.points = False

pts.bounding_box.visible = True

img.mode = 'transform'
v.reset_view()

To test the transform_box, switch to 2D and select the image layer.

overlays2.mp4

Type of change

• Bug-fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

References

• Overlay backend refactor #4907
• Layer bounding box #4849
• [WIP] Overlays #3763
• Transform mode for image layer #3611

[brisvag]

I added a few comments to help reviewing. Many of the changes inside each individual VispyOverlay are not strictly necessary for this PR (though they help a lot with reasoning about it) and could be factored out and merged beforehand, so you can ignore them for the purposes of understanding the overall changes. In practice, I split out the Visual component from the machinery that napari needs to communicate to the visual how to change (VispyOverlay).

[brisvag]

Good progress! I incorporated the bounding box code from #4849, so now we also have an example of LayerOverlay!

You can now test everything with this little snippet:

import napari
import numpy as np
v = napari.Viewer(ndisplay=3)
img = v.add_image(np.random.rand(20, 10, 15), scale=[5, 15, 10])
pts = v.add_points(np.random.rand(20, 3) * 200, translate=[20, -10])
v.overlays['scale_bar'].visible = True
v.overlays['axes'].visible = True
v.overlays['text'].visible = True
v.overlays['text'].text = 'Cool stuff!'
v.overlays['text'].position = 'top_right'
img.overlays['bounding_box'].visible = True
img.overlays['bounding_box'].line_color = 'green'
img.overlays['bounding_box'].points = False
pts.overlays['bounding_box'].visible = True
v.reset_view()

overlays.mp4

[github-actions]

Click here to download the docs artifacts
docs
(zip file)

[imagesc-bot]

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/creating-a-separate-cursor-in-the-view/88608/5

[imagesc-bot]

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/layer-legend-color/91512/6