Proposal: add visibility field to manifest schema, similar to original preview

[DragaDoncila]

Description

The initial manifest schema contained a preview boolean field which was removed in #38. I'm proposing we bring it back, but change its name to visibility and support three different values (names up for discussion) as inspired by what is currently supported on the napari hub:

public - your plugin & its metadata is listed in the viewer. Your plugin comes up in search results and might be suggested to users e.g. if it can open a file type not supported by installed plugins
hidden - your plugin can be "found" and installed from the viewer by typing its full name. Your plugin does not come up in search results and will not be suggested to users.
none - your plugin cannot be installed from the viewer and never comes up in search results. Users can still find it on PyPI and if it's installed in your environment, it will be discovered by the viewer. Note - the napari hub calls this disabled but @jni very correctly pointed out that this has different connotations in the context of the manifest, and might imply that the plugin isn't even discovered if installed. @neuromusic do you think this change would be annoying for backwards compatibility on the hub side?

The hidden field may be the contentious one as we currently don't do a lot of metadata previews or suggestions in the viewer. However, I envision a much richer browsing experience in the viewer at some point, and suggestions will definitely be coming, so I think it's important to consider this moving forward.

Why?

Currently, the viewer shows all plugins declaring the Framework:: napari classifier. The cookiecutter also has this value by default (which is perfectly reasonable as it is afterall, a blueprint for creating a napari plugin). However, this being a default value in the cookiecutter plugin means that developers who just want to play around with a new plugin, or people creating a plugin as part of a tutorial or workshop, will immediately have this plugin listed in the viewer (and on the napari hub) even if it's just a playground, in very beginning stages, or never intended to be shared with users. @tlambert03, @neuromusic, various others (and I of course) have had conversations wishing that there could be an active choice made when people want to fully share their plugin with the world (versus it still being very much in development/testing phase). If the manifest supported this in the schema, we could change the default cookiecutter value to hidden, to support this development phase.

[neuromusic]

do you think this change would be annoying for backwards compatibility on the hub side?

not particularly. def agree w/ @jni that the context is different

my biggest concern with defining this metadata in the manifest is that it becomes permanently associated with a specific release (since releases are immutable in PyPI), not the plugin/package, which is the level that plugin developers are trying to control.

this isn't a huge problem for the use case you outline (transitioning from a hidden "in development" phase to a "now I want users" phase), but use case that motivated us adding the visibility flags on the napari hub was the inverse: a plugin that no longer wanted to be advertised, but still needed to be available to users that relied on it

the challenge with defining this at the level of a release is how clients like the viewer and hub should handle when a version bump comes with a more restrictive visibility (say 0.1.0 is "public" and 0.2.0 is "hidden"... does 0.1.0 still show up in search?) or a plugin developer wants to deprecate the entire plugin (if 0.1.0 is "public", do they need to release a patch just to toggle the visibility? then the same Q as before comes up)

My feeling here is that we need to give plugin developers a way to control visibility which is not constrained by PyPI's immutability constraints & therefore the manifest is not the right place to control visibility.

I'm curious what @alisterburt thinks, as he recently requested that the hub hide some plugins: chanzuckerberg/napari-hub#582

[neuromusic]

If folks disagree with me though and want plugin developers to control visibility at the level of releases, another option would be for the napari community to state best practices on the use of "development status" trove classifiers then let users filter plugins based on development status & hide planning/pre-alpha/inactive by default.

When the hub launched, we had initially supported this but since napari didn't observe it (everything showed up in napari), every plugin had the cookiecutter's "pre-alpha" default, so it provided no signal to users about the actual development status of a plugin.

[neuromusic]

a third release-centric option: add filters on the viewer and the hub based on PEP440/semver and filter out anything < 0.1.0 by default

[neuromusic]

by my count, 61.7% of plugins are >= 0.1.0

import json
from urllib import request
from packaging import version
from statistics import mean

index_endpoint = "https://api.napari-hub.org/plugins"

with request.urlopen(index_endpoint) as f:
    response = f.read()
    index = json.loads(response)

greater_than = [version.parse(v) >= version.parse("0.1.0") for p,v in index.items()]

print(mean(greater_than))

[tlambert03]

Thanks for the issue @DragaDoncila.

I don't think controlling visibility at the level of releases bothers me. It seems to be working fine for all of vscode extensions, which also control marketplace visibility with package.json in a release and honestly, it feels pretty natural to me.

I guess the primary objection is that someone might not want to have to make that "one more release" to change visibility? If that's it, I'm not terrible swayed. One doesn't even need to bump their core X.Y.Z version if that's a concern, you can just do a post-release.

I'm not opposed to filtering out anything less than <0.1.0, but I also don't think it's particularly clearer. We all know that extracting semantics from versioning is essentially broken 😂 ... and ultimately it then just comes down to manually telling packages "if you set your package version to less than .... etc". And once we're providing them with that sort of instruction, i feel like just saying "add visibility: hidden" is more explicit.

I feel like a similar argument goes for development status. It's still kinda/sorta co-opting a related-but-somewhat-independent concept (that of development status) for semantics about visibility in napari or the hub. I can also imagine a case where a package is truly not in alpha status, but they'd like to be unlisted in napari, and having manifest.visibility.hidden says that.

Same thing for yanks (like the issue linked by @neuromusic above). If you truly only want existing dependencies to be able to use your package, then you should yank the package on PyPI (and I think napari and/or the hub definitely could/should check yank status). If you only really want to yank it from napari, then cut a new post release and update the manifest. It would then not show up for any new users, (but those who explicitly install a specific version would not be affected)..

as for the spec here, I think I'm still slightly confused about hidden vs none. specifically:

your plugin can be "found" and installed from the viewer by typing its full name. Your plugin does not come up in search results and will not be suggested to users.

what does it mean "typing it's full name". but "doesn't come up in search results". Can you elaborate on that? Are you referring to typing it's name into the plugin search in napari? on the hub? Search results in napari? on the hub? Could use a little clarification on why we need both hidden and none

[neuromusic]

I guess the primary objection is that someone might not want to have to make that "one more release" to change visibility? If that's it, I'm not terrible swayed. One doesn't even need to bump their core X.Y.Z version if that's a concern, you can just do a post-release.

No, I don't think its to much to expect a version bump. My objection is just that its not clear to me how the logic should work in that case.

If I'm a plugin dev and my 0.1.0 plugin is visible, then I do a 0.1.1 release and toggle that to "hidden", does 0.1.0 still show up?

If napari or the napari hub eventually support indexing of multiple versions of plugins and not just the latest... then the other direction is important, too.

If I'm a plugin dev and my 0.0.x plugins are hidden, then I do a 0.1.0 release and toggle visibility to "public", do my previously hidden plugins suddenly show up?

Is the proposal then that the most recent release (not including dev releases?) controls the visibility for the entire package?

[tlambert03]

If I'm a plugin dev and my 0.1.0 plugin is visible, then I do a 0.1.1 release and toggle that to "hidden", does 0.1.0 still show up?

I'd say no? i.e. the latest release determines visibility.

Is the proposal then that the most recent release (not including dev releases?) controls the visibility for the entire package?

yep. I think that's consistent with lots of other marketplaces? The latests release determines the author's current intentions.

[tlambert03]

If napari or the napari hub eventually support indexing of multiple versions of plugins and not just the latest... then the other direction is important, too.

it's fine to index anything and everything. And if you wanted to somehow make visible old packages that once were visible but no longer want to be, then you could have a "show me old stuff that once was" checkbox in your search, right? in any case, this feels more like a UI question right? the information we need is still all there in the manifest(s)

[tlambert03]

If I'm a plugin dev and my 0.0.x plugins are hidden, then I do a 0.1.0 release and toggle visibility to "public", do my previously hidden plugins suddenly show up?

show up where? on the hub? I'd say that's a hub decision, but I'd vote "no". Similarly, on npm, if you change the private tag to public, that doesn't suddenly make all of your previous versions public.

[jni]

yep. I think that's consistent with lots of other marketplaces? The latest release determines the author's current intentions.

Cool, I didn't know that. I agree that makes sense and to me resolves the very valid issues raised by @neuromusic. It was definitely not a priori obvious what should happen if 0.1.0 had visibility=public and 0.2.0 had visibility=hidden. If there is prior art then for me that settles it. (And the behaviour described by Talley for npm/VSCode makes sense to me.)

what does it mean "typing it's full name". but "doesn't come up in search results".

For me this means the box at the bottom of the plugin installer, which could be integrated in the search box — if you press "enter" on a valid plugin name, you get it installed even if it's hidden.

[tlambert03]

@DragaDoncila, seems like there was reasonable agreement on this. do you want to add this?

I'm still a bit ambivalent about the need for none in addition to hidden. as I understand it, that would be a directive to our bundled installer to essentially throw an error even if someone manually typed in the name and tried to install it? that feels to me more like a user decision than a plugin decision. (saying "I'm hidden": please don't tell users about me seems like all a plugin should concern themselves with?)

anyway, would be great to add this field

[DragaDoncila]

Idk I still like the three tiers. I kinda see it as:

public: I'm declaring my intention for people to find me and I want to be found
hidden: I want people who know about me to find me, but don't show me off
none: I don't want to be found in any kind of napari browser (although I guess maybe then you'd just remove the framework classifier...).

Right now maybe the distinctions are kinda vague for the viewer since our browsing experience in the viewer isn't very rich. But I hope that in the future, people can filter plugins by capability and search by keyword etc. in the viewer, and also see more detail about individual plugins. In that case I can easily imagine wanting to partially opt out (people can browse my page but I don't show up in search results) or fully opt out (I'm not accessible from the browser at all), depending on how I'm choosing to share my plugin. The three tiers also definitely make sense for the napari hub imo and I'd like to have consistency there.

If you genuinely don't see a use case we could always start with public and hidden and add a third value later, right?

[tlambert03]

I want people who know about me to find me, but don't show me off

but what does it mean to "find something" that you already know about? i suspect it means, "if you type the exact string in, something happens", but I still don't understand what that "something" would be? the guess I was wagering above was that it's a "directive to our bundled installer to essentially throw an error even if someone manually typed in the name"... but are you thinking something different?

people can browse my page but I don't show up in search results

so, basically, in either the hub or the viewer, if you type in some exact string, then you can get more information? Yeah, still confused by this, since, if you had that string to begin with, it seems you probably found the information somewhere else already.

If you genuinely don't see a use case we could always start with public and hidden and add a third value later, right?

Yeah, it feels very YAGNI to me (not just in the viewer, also the hub). I suspect we would get a lot of "wait, so what is the difference again?" if we had to describe it to people. If you have a real world example of any other marketplace using a three-tier system, I might be more swayed... but npm doesn't have it, vscode doesn't have it, pypi doesn't have it, wasn't able to find any evidence of something like it in cargo, etc...

I'd like to start with visible, hidden, and open it up later if a compelling request comes along

[neuromusic]

First, napari is just Python so plugins should always be "installable" if they are available on PyPI/conda. That could happen by either typing in the name directly OR if the plugin is called as a dependency (::cough:: devbio ::cough::). Adding a "disabled" flag that blocks installation seems... odd and unpredictable and not a good idea.

PyPI is a cheese shop: You ask for a kind of cheese and PyPI says "yes, here it is" or "no".

For the napari hub, on the other hand, we're aggregating info about plugins and creating a webpage for each plugin. "Disabled" prevent this (but does not affect installation).

For "hidden" on the napari hub, we give plugin devs a URL but don't let it appear in search.

Part of the motivation here is to recognize that many plugin devs still want to use PyPI as a cheese shop, but don't necessarily want to make their thing easy to find.

In napari, I assume "hidden" means "don't appear in search results"... which would have the same in-napari functionality as "disabled" or dropping the trove classifier, right?

I'd like to start with visible, hidden, and open it up later if a compelling request comes along

This seems reasonable for the napari use cases: two states (does it show up in search or not). And TBH the trove classifier covers those two states... no additional metadata needed.

OTOH, if the goal is for this single field to span use cases across both napari and the napari hub, then we do still need a distinction between "hidden" and "disabled" even if napari doesn't make use of the distinction. We might be able to say "if you want to drop your plugin from the napari hub, then drop the trove classifier" 🤔 Need to think on that a bit.

[tlambert03]

PyPI is a cheese shop: You ask for a kind of cheese and PyPI says "yes, here it is" or "no".
For the napari hub, on the other hand, we're aggregating info about plugins and creating a webpage for each plugin.

Ok, then remove pypi from the list I gave above, and the point still stands that I'd like to see an example of any other plugin market place that needed something beyond visible or hidden

For "hidden" on the napari hub, we give plugin devs a URL but don't let it appear in search.

Sure, nothing wrong with that. Still only need two

"hidden" and "disabled"

The word disabled hasn't yet come up in this discussion, and has generally been reserved for a user decision (not a plugin decision). Disabled means "I don't want to uninstall this plugin, but I don't want it to load". Can you elaborate on what it means to you at the hub?

[neuromusic]

Sorry, "disabled" is what we use on the hub, analogous to none discussed here.

https://github.com/chanzuckerberg/napari-hub/wiki/Customizing-your-plugin's-listing#visibility

[neuromusic]

the point still stands that I'd like to see an example of any other plugin market place that needed something beyond visible or hidden

The important distinction here I think is that this marketplace is a subset of PyPI

[tlambert03]

ok, so can you elaborate on the distinction between hidden and none? From re-reading your post, the difference seems to be essentially whether you create a dev page for the plugin developer or not. Did get that correct? If so, I'm still struggling to see why we need a whole new flag for that. Why not just create a (hidden) "dev" page for everything? And then just not show them to end users if the plugin has stated itself as "hidden"? What have we lost by removing the none option? ... it kinda seems like all we've lost is the ability for a plugin dev to say "no, don't even make a development page (that only we could have seen anyway)". Just tell the plugin developer not to look at it if they don't want to?

[neuromusic]

From re-reading your post, the difference seems to be essentially whether you create a dev page for the plugin developer or not. Did get that correct?

Sorry, no... the distinction is whether or not we create a plugin page (e.g. https://napari-hub.org/plugins/<plugin_name>).

If visibility is set to "hidden", then this page is created for the plugin but it is not indexed for searching on https://napari-hub.org

If visibility is set to "disabled", then we do not create the plugin page at all.

[tlambert03]

ah ok, I understand now. sorry for being slow.

with that understanding, I guess i would like to hear some more opinions on the merits of just telling someone who doesn't event want their plugin to have a "hidden" plugin page on the hub to simply remove the napari framework classifier. If a plugin prefers to be neither discoverable via search ("hidden") nor have the napari hub or application be aware of them in any way, my first instinct would be to encourage them to remove the framework classifier.

[tlambert03]

@DragaDoncila would you like to implement this or should I? we all agree we need "public" and "hidden". There are no plugins waiting on "none". Let's just implement the thing we all agree on. It's trivial to add another valid key layer with no backwards incompatibility if anyone ever comes along and says "I don't want the hub to even be aware of my plugin, but I also don't want to remove the classifier"