Add NAP-6 - Proposal for contributable menus

[DragaDoncila]

Description

This PR proposes a set of contributable menus to expand the places where plugin developers can add
contributions.

The set of menus in this PR were generated based on existing discussion about the Tools menu, an analysis of a few current plugins with multiple widget contributions and some future thinking about new contribution types and napari objects.

There's a couple of things I still have questions about/haven't decided and I've summarized these here. Would appreciate as many eyes on this as we can get @napari/core-devs.

Questions/Decisions

• Should plugins be able to contribute whole new submenus to the proposed set of contributable menus? Nothing is stopping them at this stage.
• Should each plugin get its own submenu within menus if it contributes more than one item to that submenu? Currently in the exercise with the four plugins I've listed each one individually, because I think deeply nested menus are annoying. However as we can see, we would need to then list the plugin name next to each one which can be very annoying visually.
• We may want to move Filter, Transform and the analysis options from Generate to top level Layers menu. I didn't love this initially because it's not clear to the user whether the menu item edits existing layers or generates new ones. I feel like given we still only have widget contributions though, we can't enforce this anyway, so we may just want to move those out to the top level but group them under an "analysis" header or separator or something. Once we have new contribution types we can expose the edit vs. generate behavior a different way?
• I feel like Classification is way too specific and we need a broader menu for postprocessing/anaylsis but I didn't know what to call it, so plz help.
• Tracks feels like it may not be very domain agnostic.
• I considered adding a Layers -> New which would hold both builtin new empty layer actions for each of our layer types and allow plugin developers to contribute actions that generate new layers with specific properties. I don't know how common this is so I left it out, but we may just want it for our own layer types too.
• I considered but did not add a general top level Utilities menu because it feels too broad and like a shortcut to a dumping ground.
• I'm not sure what's supposed to go in the references/footnotes section, so I haven't changed that but I'll go back and look at some of the other NAPs for inspo. Just didn't want to hold it up for this reason

[brisvag]

Great work @DragaDoncila! As usual, approving to get this in quickly, so we can iterate via further PRs. I left a few comments, but feel free to address them in a later PR.

Also, not sure why, but I don't see this in the artifact... Maybe some other file needs to be updated?

And a general point about the napari.yaml: from the small piece you provided, it looks a bit unwieldy even with just 2 menu contributions. Are there plans to make this a bit more ergonomic? For example, a command could just have another field called tags and we could use those to populate menus accordingly... I feel like I saw discussions about this before, but maybe I'm wrong?

[brisvag]

What are the expected inputs/outputs for Visualisation and Measure?

[DragaDoncila]

I think these menus are the hardest to categorize based on input/output.

For Visualization I can imagine a widget that adds a new layer that provides a different "view" on the data. Mostly I was thinking of stuff like filtering what data is being shown in the current layer, auto adjustment of brightness/contrast, tools for quickly assigning/swapping preset colormaps... In general I think of it as "changing the way the layer is displayed in the viewer without changing the data of the layer".

Measure I think would typically contain items that generate text/summary output based on the layer, generate a new layer with information about the data (e.g. @psobolewskiPhD was mentioning he draws lines on Shapes layers to measure distances in Image layers or something like that - this would go here) or produce plots/figures about the layer data e.g. histograms, intensity plots, derived features, etc.

[brisvag]

For Visualization I can imagine a widget that adds a new layer that provides a different "view" on the data. Mostly I was thinking of stuff like filtering what data is being shown in the current layer, auto adjustment of brightness/contrast, tools for quickly assigning/swapping preset colormaps... In general I think of it as "changing the way the layer is displayed in the viewer without changing the data of the layer".

This feels like a combination of Generate and Edit to me, though.

Measure I think would typically contain items that generate text/summary output based on the layer, generate a new layer with information about the data (e.g. @psobolewskiPhD was mentioning he draws lines on Shapes layers to measure distances in Image layers or something like that - this would go here) or produce plots/figures about the layer data e.g. histograms, intensity plots, derived features, etc.

Makes sense!

[DragaDoncila]

This feels like a combination of Generate and Edit to me, though.

Perhaps Generate if we include adding new layers (which is debatable tbh I don't actually know of any widgets that do this I just could imagine they do) but definitely not Edit - nothing in the Visualization submenu should edit the data of your layer, it should always be non-destructive. It should only change how the layer is displayed. For example a widget that filters tracks displayed in a tracks layer based on certain properties shouldn't be deleting tracks from the data - just turning their visibility on and off.

[brisvag]

But due to how visualisation and data in napari are still coupled in the Layer object, to change visualisation you have to edit the layer object. But I see what you mean, Edit should only affect the data (and maybe features?), and Visualisation only the rest.

With this in mind,Visualisation says that it "generates" visualisations, but it sounds like you mean "change visualisation parameters"? But then what about something that creates a plot widget like napari-properties-plotter? Shoudl that live in Measure?

[DragaDoncila]

But I see what you mean, Edit should only affect the data (and maybe features?), and Visualisation only the rest.

Yes, I do think there's a difference between the Layer data & features (which I think are intrinsically coupled as fundamental attributes) and other Layer attributes that determine visualization and other napari specific features. It's probably worth us defining a clear line for this as we look towards user facing documentation.

With this in mind, Visualisation says that it "generates" visualisations, but it sounds like you mean "change visualisation parameters"

Yes I'd say the second is closer to a correct description of what I have in mind. I will update this. Thank you!

But then what about something that creates a plot widget like napari-properties-plotter? Shoudl that live in Measure?

I would say yes, as this is an auxiliary visualization, not changing the view of the layer itself.

[brisvag]

Shoudl this be inside I/O rather than on its own? It feels a bit specific for a top level menu at the same level of as Layers.

[alisterburt]

disagree, these plugins are in a class of their own, doesn't feel like it fits in IO - I'm fine with top level

[brisvag]

It would be nice to also see how the napari.yaml would look like for these plugins. For example, it's not clear to me how exactly a plugin would opt for the plugin submenu versus the napari menus.

Also, should/can a plugin put a command in multiple places?

[DragaDoncila]

For example, it's not clear to me how exactly a plugin would opt for the plugin submenu versus the napari menus.

This mechanism does not yet exist, nor does the mechanism for a plugin declaring submenus for its widgets menu. I will work with @nclack to see what that might look like and provide an example before we accept the NAP.

Also, should/can a plugin put a command in multiple places?

Plugins can do so currently. I do not personally believe they should - note I did not replicate widget contributions in the plugin's submenu in the example for that reason. I'm open to discussion on this, but I think given the command palette would always allow users to find everything a plugin offers, duplicating commands in menus would be confusing.

[alisterburt]

Had a quick first pass - love it, thanks for the work here @DragaDoncila ! Minor comments, +1 to merging and iterating in followup

[alisterburt]

❤️

[alisterburt]

disagree, these plugins are in a class of their own, doesn't feel like it fits in IO - I'm fine with top level

[alisterburt]

some grammatical inconsistencies here
should visualization become visualize? or should 'measure' become 'measurement'? I think I prefer the second

My immediate reaction to this list is that I don't know if I would intuitively look to 'layers -> generate' for registration/projection/segmentation/tracks/classification. Maybe I'm just not used to the mental model but 'measurement' feels at the same level as 'registration' or 'projection' for me

[DragaDoncila]

some grammatical inconsistencies here
should visualization become visualize? or should 'measure' become 'measurement'? I think I prefer the second

Yeah I don't like it either lol. I initially had visualize and then I changed it but I can't really remember why I changed it so it can't have been too compelling. I will change it back I think.

My immediate reaction to this list is that I don't know if I would intuitively look to 'layers -> generate' for registration/projection/segmentation/tracks/classification.

Yep, I can see that. Also I think someone would easily want to "generate" a transform, so the distinction isn't really meaningful. Initially I was thinking of it in terms of making clear to the user whether this will edit existing layer or add a new layer, but as I mention in the description, we can't guarantee that atm anyway with widgets. So I think that distinction should be made per plugin item once we have specific enough contributions to enforce it.

@alisterburt how would you feel about moving these items out to the top level Layers menu? Would you worry that it is too long?

[kevinyamauchi]

I think registration/projection/segmentation/tracks/classification can all be top level under Layers. I agree that it wouldn't be super intuitive to find them under Layers -> Generate.

[Czaki]

Thanks for working on this. Two days ago, I finally understood why I feel uncomfortable with deep menus. This pattern in the context of user interface design is the end of the XX century, maybe the first few years of the XXI century. You can look around and see that non of the easy-use and user-friendly software contains deep menus.

But the motivation described in this NAP is real, and we cannot keep our eyes closed.

So I prefer rather than name it "plugin menu contribution" but "structurize contributions by metadata".

Even without such a change, I would like to add the description field to the manifest (for example, next to the command). Such a field could be used to provide users with better information about contributions and could be really useful with trying to implement a more modern interface. For example, command palette.

[kevinyamauchi]

Thanks, @DragaDoncila ! Great stuff. I also think we should merge ASAP so we can begin iterating.

So I prefer rather than name it "plugin menu contribution" but "structurize contributions by metadata".

Even without such a change, I would like to add the description field to the manifest (for example, next to the command). Such a field could be used to provide users with better information about contributions and could be really useful with trying to implement a more modern interface. For example, command palette.

I think we can discuss this after merging, but I agree with @Czaki that we should consider if there are ways we can structure this such that the resulting annotations can be re-purposed later to organize other plugin access patterns (e.g., command pallette or other menu system).

[kevinyamauchi]

Suggested change

	│ ├─ Annotation Tools
	│ ├─ Annotate

I think this is more in line with the other Edit items.

Also, I think we should bring Annotate out to a top level Layers menu (or even it's own category). I think annotation is a pretty commun napari use case and thus should be easy to access. Happy to merge now and discuss later though.

[psobolewskiPhD]

Should Annotate maybe be under the Edit menu vs. the Layer menu?

[kevinyamauchi]

I think registration/projection/segmentation/tracks/classification can all be top level under Layers. I agree that it wouldn't be super intuitive to find them under Layers -> Generate.

[brisvag]

I forgot to address the questions/decisions at the top:

• Should plugins be able to contribute whole new submenus to the proposed set of contributable menus? Nothing is stopping them at this stage.

I would say no, especially to avoid duplicated menus with slightly different names; we can always change that later, but viceversa is going to be hard.

• Should each plugin get its own submenu within menus if it contributes more than one item to that submenu? Currently in the exercise with the four plugins I've listed each one individually, because I think deeply nested menus are annoying. However as we can see, we would need to then list the plugin name next to each one which can be very annoying visually.

My feeling is that this would hurt usability, cause it's not immediately clear where to find the thing you want, if you have multiple plugins.

What would be awesome is if we could have little icons to represent a plugin visually, in order to make it immediately clear that a menu entry is from a specific plugin (but then give the full plugin name when you hover). Plugins could provide their own icon (which btw we could use in all sorts of places), but we could otherwise generate unique ones similarly to how github does with user icons.

• We may want to move Filter, Transform and the analysis options from Generate to top level Layers menu. I didn't love this initially because it's not clear to the user whether the menu item edits existing layers or generates new ones. I feel like given we still only have widget contributions though, we can't enforce this anyway, so we may just want to move those out to the top level but group them under an "analysis" header or separator or something. Once we have new contribution types we can expose the edit vs. generate behavior a different way?

I agree that the generate/edit distinction is very important. Do we have any plans to somehow enforce such distinction with widgets as well? Maybe hard to do enforce it, but at least we could ask plugin devs to declare what kind of widget it is?

• I feel like Classification is way too specific and we need a broader menu for postprocessing/anaylsis but I didn't know what to call it, so plz help.

Is Analysis too generic?

• Tracks feels like it may not be very domain agnostic.

I'm not sure what it's supposed to be, actually. Literally Tracks layers?

• I considered adding a Layers -> New which would hold both builtin new empty layer actions for each of our layer types and allow plugin developers to contribute actions that generate new layers with specific properties. I don't know how common this is so I left it out, but we may just want it for our own layer types too.

I like this!

• I considered but did not add a general top level Utilities menu because it feels too broad and like a shortcut to a dumping ground.

Yeah, this sounds like what we were trying to avoid for Tools.

[DragaDoncila]

My feeling is that this would hurt usability, cause it's not immediately clear where to find the thing you want, if you have multiple plugins.

Yep I tend to agree here. Replicating per-plugin submenus within these new menus would just replicate the issue we have with the current Plugins menu.

What would be awesome is if we could have little icons to represent a plugin visually, in order to make it immediately clear that a menu entry is from a specific plugin ... we could otherwise generate unique ones similarly to how github does with user icons.

I really like this idea! The manifest already has an icon field though it's not exposed yet and I'm not exactly sure how it works. But certainly a lot of the machinery is already there, and as you say we could generate icons for plugins that don't provide it.

Do we have any plans to somehow enforce such distinction with widgets as well? Maybe hard to do enforce it, but at least we could ask plugin devs to declare what kind of widget it is?

Yeah I can't think of a good way to enforce it. Because widgets can take the Viewer, they can do just about whatever they want. I think we're not yet at the stage where we can stop providing the Viewer and still allow plugins to be useful. I definitely agree in a best practices doc we should strongly encourage that widget behaviour conforms to the specification for each menu.

Is Analysis too generic?

I feel like Analysis covers too much ground? I think Post-processing is probably closer to what we want? As these are utilities that are only useful to you once you've already done an initial analysis? But I worry that Post-processing will be odd when we don't have a Processing menu lol

I'm not sure what it's supposed to be, actually. Literally Tracks layers?

Basically, yes. A quick analysis of all widget contributions showed me about 10 widgets that perform some sort of tracking, so I gauged that was a significant enough number to warrant its own submenu, but I don't feel super strongly about it. If we don't have a Tracks category though, we do have to think about where these contributions would go.

[Czaki]

Yeah I can't think of a good way to enforce it. Because widgets can take the Viewer, they can do just about whatever they want. I think we're not yet at the stage where we can stop providing the Viewer and still allow plugins to be useful. I definitely agree in a best practices doc we should strongly encourage that widget behaviour conforms to the specification for each menu.

But still, we could allow tool menu entries could be only function contributions that do not get Viewer or ViewerModel as an argument?

But In general, the plugins should be able to get Viewer for more complex widgets.

[brisvag]

Ok, I'll merge so we don't hang on this for too long. As we've seen before, it's easier to discuss/review changes on follow-up PRs!