Discussion about duplication of metadata in .napari-hub and python package metadata

[tlambert03]

The discussion of where plugin developers should enter specific metadata about their project has come up many times. While I know I've often requested that we try not to create new places to enter data that already has a "standard" place, I don't know that we have a running issue about it. So, I'd like to open this issue as a place to discuss it, and a somewhat more public request to remove some/all keys.

cc @neuromusic @potating-potato @jni @sofroniewn @DragaDoncila

My personal hope is that we can reduce the keys in .napari or .napari-hub to the bare minimum, and, preferably, to remove the file altogether. Below I discuss each key mentioned in the wiki at https://github.com/chanzuckerberg/napari-hub/wiki/Customizing-your-plugin's-listing#github

As you'll see below only 1 or two plugins have actually used any of these key in .napari or .napari-hub, so removing these from the spec and from the wiki should be quick and painless.

keys in .napari that are duplicated from core metadata that should be removed

I would propose that each of these be removed from .napari as soon as possible

• Summary

  only a single package (napari-features) has added a summary. unfortunately, they didn't include it in their setup.cfg metadata. (a good argument for removing it from .napari, since it should minimally be in setup.cfg). One PR would solve that.

Links

• Project Site

  this should be taken only from core metadata home page. only two plugins have used this so far.

Each of the following should just be a key that is looked for in core metadata project urls. See also the urls field in PEP621

• Documentation

  (only two plugins are using this, they should add it to core metdata)

• Support

  (three plugins are using this, they should add it to core metdata.)

• Report Issues

  (only one plugin is using this, they should add it to core metdata)

• Twitter

  (only one plugin is using this, they should add it to to core metdata)

keys in core metadata that might need to be duplicated

• Authors

  As I understand it, the motivation for this was to accept an orcid field, which is a nice idea. EP621 has made it possible to declare an inline table for project.authors... so it would be possible to declare something like this:

  [project]
  authors = [{ email = "mail@example.com", name = "My Name", orcid="01234-2354-345777..." }]

  setuptools will ignore that key, so it wouldn't make it into the dist-info of a wheel, but it's still probably the correct place to put it. Alternatively see [tool.napari] below.

• Description

  This is the one field that makes the most sense to potentially duplicate. Since it's certainly conceivable that someone would want to make their hub page look different then their readme and/or their pypi page... so, a single DESCRIPTION.md file remains the main thing I can't see a quick replacement for.

keys not in core metadata, that I don't think should go in .napari

• Conda

  see use npe2api in plugin install dialog napari/napari#4893 for some context. I'd prefer this not be in .napari at all, since it's not specific to the napari hub or display of a package on the hub. I would propose that it be moved either to a [tool.napari] section in pyproject toml (which is guaranteed to make it into the sdist) or to the manifest itself (which is guaranteed to make it into the wheel of a functioning plugin.)

• Visibility

  see Proposal: add visibility field to manifest schema, similar to original preview napari/npe2#196 for some context. I'd prefer this not be in .napari at all, since it's not specific to the napari hub or display of a package on the hub. I would propose that it be moved either to a [tool.napari] section in pyproject toml (which is guaranteed to make it into the sdist) or to the manifest itself (which is guaranteed to make it into the wheel of a functioning plugin.)

labels

Becuase of the PRs opened manually by the hub team, this is by far the most commonly used field, with about 20 plugins using it. This was originally in the npe2 manifest spec, but removed (the motivation for that removal is confusing to me, but in the past now). I'd propose this be readded to the manifest, but this would require opening new PRs to all those plugins that were previously instructed to put it in .napari.

pyproject.toml [tool.napari] or napari.yaml

For any plugin metadata that has a natural home in python's core metadata, i think we should absolutely use that field. There shouldn't be two places to enter the same thing.

For metadata that doesn't have a natural home in python's core metadata, we should probably use either a [tool.napari] section in pyproject.toml or add a new field to the napari.yaml manifest.

Looking forward to your feedback. thanks

[tlambert03]

Moving to discussion does make this a bit easier to get lost as a non actionable item. Still looking forward to your thoughts and really don't want this to get lost in the mix again.

[liu-ziyang]

Just want to assure you this won't be lost. Thank you for the detailed writing and very clear motivation, rationale, and proposals. I am supportive of this effort and can relate to the sentiment, I want to ensure we are digesting it well internally before I write anything down, hence the lack of response from me for the time being. (I am assuming the same for others as well)

[neuromusic]

Thoughts are coming.

Moving to discussion since this is a discussion and not immediately actionable. Our team tries to reserve the "issues" for concrete, actionable work and high-level plans and keep ideas and conversations regarding improvements in "discussions".

Once we've made some concrete decisions on any changes we decide to implement, we'll capture those in issues.

[neuromusic]

Thanks for reaching out @tlambert03 and centralize this conversation.

I want to make sure that we distinguish between preferred sources for metadata and allowed sources for metadata. In my mind, the greatest risk of confusion for plugin developers is if the preferred sources aren’t clear and the greatest risk of friction is if the preferred sources are too dispersed.

I understand that you don’t want .napari-hub to be an allowed location for plugin developers to modify their napari hub metadata, but first and foremost, let’s make sure we’re on the same regarding preferred locations by updating the documentation at napari.org/plugins to clearly articulate the napari community’s preferred sources for these metadata. This way, we can ensure that there’s no ambiguity and we can work toward consolidating the preferred sources into a single source to the extent possible. At the same time, our team can revise the language in “Customizing your plugin’s listing” to make it clear what the preferred sources are and why we support certain alternatives.

Improving plugin developer support and documentation is high priority for us over the coming months and we have bandwidth to help support and drive this effort.

This alone will get us far to addressing concerns about plugin developer confusion and friction, however, part of what I read in your recommendations is that you believe the napari hub’s only allowed sources should be the napari community’s preferred sources.

My personal hope is that we can reduce the keys in .napari or .napari-hub to the bare minimum, and, preferably, to remove the file altogether.

Is it fair to say that your goal is to make napari plugins “self-describing”?

For any plugin metadata that has a natural home in python's core metadata, i think we should absolutely use that field.

Agreed. Where there are relevant Python core fields available that meet the needs of napari users and plugin developers these should be the preferred fields. And for every piece of metadata that has a natural home in python’s core metadata, we support this (except perhaps the pyproject.toml URL spec that you referenced… but we can add that)

However, implied here is that you think any other fields for this metadata should not be allowed by the napari hub

There shouldn't be two places to enter the same thing.

This seems to be the heart of our disagreement.

When considering which allowed sources of metadata the napari hub supports, we have two goals for our work on the napari hub that I want to emphasize as they have implications for what we choose to support (or not).

We want to support as many plugins (and plugin developers) as reasonably possible. Part of what this means is that…

• We don’t want to force plugin developers to update to npe2 simply to add metadata that doesn’t affect the functionality of their plugin. The metadata under discussion here only serves the purpose of making it easier for user to find their plugin and understand what it does. It is fundamentally documentation & I don’t want to force plugin developers to refactor their plugin and migrate to npe2 simply to take advantage of documentation features of the napari hub (such as “labels”). By my count, over half of napari plugins are still using npe1.
• We don’t want to force plugin developers to update to pyproject.toml when Python still supports setup.cfg and any plugin developer that used the cookiecutter prior to Jan 2022 probably has a setup.cfg. I haven’t run the numbers on how many plugin devs are using setup.cfg

We want to enable napari users (of all levels of Python expertise) to find relevant and high quality plugins. Part of what this means is that…

• We want to encourage documentation (incl summary, description, metadata) that is relevant to napari users and tailored to their needs
• Since many “Python core” fields need to be relevant to all Python users, their napari hub analogs should be customized for napari users

To maintain these commitments to plugin developers and users, we want to allow multiple sources of some metadata.

For metadata that doesn't have a natural home in python's core metadata, we should probably use either a [tool.napari] section in pyproject.toml or add a new field to the napari.yaml manifest.

Agreed. This sounds great. I’m excited about your efforts to consolidate metadata into the manifest file and pyproject.toml, as you’ve proposed. On the napari hub side, we’re happy to scoop up the relevant metadata from these sources. Please keep me in the loop on your plans and timelines here so that we can make sure to queue up the work for our team and prevent a major lag between napari’s support for these sources and our support for these sources. As we identify these fields, I’m happy to make issues to track the work on our end and link to the relevant PRs or issues in npe2/napari/etc.

However, on the napari hub, we still want to offer an alternate path for plugin developers to document their plugins that doesn’t require a major upgrade to their core build infrastructure just to add some metadata that is useful for napari plugin users. When napari is ready to deprecate npe1 and/or we’re confident that most actively maintained and used plugins are all on npe2, I’m happy to reconsider whether we continue to support .napari-hub for metadata that has been consolidated into the napari.yaml manifest.

The path forward that I would recommend:

1. Let’s make sure we’re aligned on the preferred sources for these metadata by adding documentation to napari.org/plugins and updating the napari hub’s `Customizing your plugin listing” docs accordingly
2. The napari hub team can begin to deprecate support for .napari in favor of .napari-hub to prevent confusion with napari.yml... we can do a round of PRs for plugin developers that are using .napari to move them to .napari-hub then deprecate our support for .napari as a hub-specific configuration folder. I acknowledge that this doesn’t address your core issues here, but it will at least help to prevent confusion.
3. For fields which we support in the napari hub config that do not yet have analogs in the napari manifest or pyproject.toml, our team can add support for those sources once you have implemented them into the npe2 spec or formalized them as preferred sources in the documentation. We can also start to add notices that some of our allowed sources may be deprecated once they are implemented elsewhere.
4. When napari is ready to deprecate support for npe1 and/or we can reasonably expect that all actively supported and used plugins have manifest files, then the napari hub can consider deprecating support for those fields that have preferred analogs in the manifest.
5. Similarly, when the Python community is ready to deprecate setup.cfg and/or we can reasonably expect that all actively supported and used plugins have pyproject.toml files, then the napari hub can consider deprecating support for those fields that have preferred analogs in the manifest.

I’m going to break out my thoughts on each of the fields you highlight in separate sections below so they can be discussed separately.

[neuromusic]

Summary

@tlambert03: I would propose that each of these be removed from .napari as soon as possible

This falls under the same bucket as the DESCRIPTION for me: plugin developers should use a custom field. Currently, a large number of napari plugins feature summaries that go something like “A napari plugin that X” This is GREAT for PyPI.org, but rather redundant on the napari hub.

We’ll use the PyPI summary as a fallback (since something is better than nothing). Ideally, this should probably be in the manifest

Preferred:

• napari.yaml (needs to be added)

Allowed:

• Napari hub config file (until adopted by npe2 and npe1 is deprecated)
• Python core metadata

[neuromusic]

Links

@tlambert03: I would propose that each of these be removed from .napari as soon as possible

Agreed. Happy to start deprecating our support for links in .napari-hub/ & we can facilitate migrating the plugin devs who are using links here.

[neuromusic]

#629

[neuromusic]

Authors

@tlambert03: keys in core metadata that might need to be duplicated

Authors is surprisingly complicated (due largely to poor community standards around authorship of open source scientific software) and probably warrants a full discussion.

It's under-documented (which we are actively working to fix), but we also support CITATION.cff and we recently began pulling author info from the developers’ preferred citation, if it exists.

In our current implementation, this takes precedence over the Python core metadata if a CITATION.cff file is provided.

This has the pitfall, however, of scenarios where the citation author list is more than the folks who are considered “authors” of the plugin.

We’ve also done some work recently to improve our support for finding plugins by author (specifically filter-by-author) which has implications for where we source author info. Since the de facto standard for the napari community is to list authors in the Python core “author” field, we now parse this string to find individual authors, looking for commas, ampersands, and “and” as separators. This has come at the cost of false splitting of some institutions and authors who added their affiliations. For these folks, we’re planning to open PRs to encourage them to use .napari-hub/ where we don’t do any string splitting.

Because of these two issues (the need to support institutional authors and allowing an override of our citation author logic) we will override both Python core and CITATION.cff info with author information defined in .napari-hub/

Preferred:

• CITATION.cff

Allowed:

• Napari hub config file
• Python core metadata

I’m very curious if you have thoughts on the preferred locations to specify author information... your ideas around using the newer inline table in Python core looks promising.

Note: the ORCiD metadata is currently unused, but I can imagine tracking more author-level information (affiliation, Github handles, etc). That will probably not happen until we create user accounts for authors, though.

[tlambert03]

this is a good example of why I don't love the preferred/allowed distinction. core metadata should never be left unpopulated due to the presence of an additional file. if someone leaves out author/email in their core metadata, because we've told them it's ok to put it in the hub or citation file, then we've done them a disservice. Core metadata should always be preferred. In cases where the schema is too limited, we can allow them to add additional information in some other place, always with the clarification that they should also be populating their core metadata

[neuromusic]

In cases where the schema is too limited, we can allow them to add additional information in some other place, always with the clarification that they should also be populating their core metadata

This makes sense to me

[neuromusic]

Conda

@tlambert03 : see napari/napari#4893 for some context. I'd prefer this not be in .napari at all, since it's not specific to the napari hub or display of a package on the hub. I would propose that it be moved either to a [tool.napari] section in pyproject toml (which is guaranteed to make it into the sdist) or to the manifest itself (which is guaranteed to make it into the wheel of a functioning plugin.)

The primary intent with this field was to offer exactly what you’ve built with the npe2api service. We would (eventually) like to surface this info on the frontend (e.g. through a filter for plugins that are available in the desktop app), but we currently don’t use it.

If the napari community wants to replace the napari hub’s API with npe2api, then I’m happy to deprecate this and see where the dust settles. We may still end up needing to support such a field in .napari-hub one day (say, if we want to support conda-only plugins) but we can tackle that Q when we get there.

[tlambert03]

I don't think this depends on npe2api usage or lack thereof. That info could go in pyproject either way

[neuromusic]

I mean that with napari moving to npe2api to get conda info, we'll may drop it from our API and won't source it at all until we prioritize adding it to the frontend.

[neuromusic]

Visibility

@tlambert03 : see napari/npe2#196 for some context. I'd prefer this not be in .napari at all, since it's not specific to the napari hub or display of a package on the hub. I would propose that it be moved either to a [tool.napari] section in pyproject toml (which is guaranteed to make it into the sdist) or to the manifest itself (which is guaranteed to make it into the wheel of a functioning plugin.)

Depending on how that PR lands, we may be able to move most of our logic for granular visibility (public vs hidden, for example) to your proposed new napari.yaml metadata and, pending adoption of npe2 eventually deprecate some of this functionality.

However, I strongly suspect that we’ll still need to maintain some kind of “opt out” flag for plugin developers that don’t want to show up on the napari hub (even if they want to show up in napari). We can explore some other options for that, though.

[tlambert03]

as mentioned in napari/npe2#196 ... the main question to me here is why not just encourage them to remove the napari framework classifier. there is a lot of potential YAGNI going on here. Why don't we wait to hear from someone saying "please do let me be discovered in the napari application but please don't even create a hidden webpage for me in the hub". I personally can't imagine that happening. and at the very least it seems like something we don't need to create a new field for until someone specifically requests it.

[neuromusic]

Why don't we wait to hear from someone saying "please do let me be discovered in the napari application but please don't even create a hidden webpage for me in the hub".

Because adding the napari trove classifier to their PyPI metadata does not imply that the author has opted in to their PyPI and Github content being aggregated by the napari hub team.

We want to give plugin developers utmost control over their content and how we use it and share it. We've chosen this mechanism as an easy, reliable opt-out mechanism which is specific to the napari hub. There are a variety of ways that we might be able to do this, with a variety of legal and technical tradeoffs (and we may offer another mechanism or multiple mechanisms in the future), but this is how we've decided to offer an opt-out at this time.

[tlambert03]

Because adding the napari trove classifier to their PyPI metadata does not imply that the author has opted in to their PyPI and Github content being aggregated by the napari hub team.

Why not?

reliable opt-out mechanism which is specific to the napari hub.

If the hub is to be an index of napari plugins, with parity with the in app plugin installer, why not use the same opt in that we use? Again, as soon as you find a case of someone saying "please put me in the napari app, but NOT in the hub", I maintain that this is YAGNI

[alisterburt]

+1 here, let's use the same opt-in mechanism until someone requests something more powerful

[neuromusic]

Labels

This was originally in the npe2 manifest spec, but removed (the motivation for that removal is confusing to me, but in the past now). I'd propose this be readded to the manifest, but this would require opening new PRs to all those plugins that were previously instructed to put it in .napari.

I'm certainly open to our team sourcing this metadata from the manifest, however, our implementation here is much more constrained than a general purpose "label", as we only support a closed vocabulary from a subset of a single bioimaging-focused ontology.

Does the napari team want to support this? If so, we’d be happy to start pulling from the manifest and can facilitate PRs to help migrate plugins over and begin to deprecate this field.

[tlambert03]

then let's work together and define what the schema for npe2 categories should have been before removal in napari/npe2#38

label is just a word without a definition. we can apply whatever ontologies to it we want. i just didn't want to spread out where this information goes.

[neuromusic]

@nclack has explained to you why he advocated for their removal. As I understand, it was an effort to appropriately scope the initial npe2 implementation to fields that napari would actually use in the initial release of npe2, but he can speak better to the rationale than me, as I was not involved in the decision.

But as you said "its in the past now". How would you like to move forward with labels in npe2? In the napari hub, we only currently support a subset of the EDAM-Bioimaging ontology. Is this consistent with your vision for labels in napari?

[tlambert03]

My vision for labels in the manifest was literally just to support marketplace searching, so it was whatever you wanted it to be before it got removed

[tlambert03]

let’s make sure we’re on the same regarding preferred locations by updating the documentation at napari.org/plugins to clearly articulate the napari community’s preferred sources for these metadata.

my primary concern here is also for the plugin developer, and I don't want them to be confused about who to listen to when it comes to where they should put information. I strongly feel, (and it is my general impression that the core developer team shares the opinion), that we want to adhere as much as possible to python ecosystem standards as possible. So if something already exists in the python world, we don't go create another way to do it.

allowed vs preferred

I can get behind this distinction. I wouldn't insist that you remove all code that parses the .napari-hub file for these duplicated keys... however, I think it's just not a good idea, (and haven't yet heard any good reasoning for it). so, now that it already exists, I think we should explicitly discourage it. so I'd say we should

• remove any documented support for it in "customizing your plugin"
• open PRs encouraging plugins to put their data in core metadata

Is it fair to say that your goal is to make napari plugins “self-describing”?

my goal is to use python standards where they exist, and not needlessly create additional places to put the same information. emphasis on same here: these fields in .napari could potentially be used, but i would say they should only be used if someone explicitly wants their hub representation to differ from their core metadata info. and there, I suspect the "real world" occurence of such a difference to be extremely low. as mentioned in my original post, none of these duplicated fields (with the exception of description) have been used to provide different fields to the hub than in the core metadata... they've just added confusion, and split where a couple (misled) plugins have entered their data.

We don’t want to force plugin developers to update to npe2 simply to add metadata that doesn’t affect the functionality of their plugin.

they don't have to ... they can use pyproject.toml

We don’t want to force plugin developers to update to pyproject.toml when Python still supports setup.cfg and any plugin developer that used the cookiecutter prior to Jan 2022 probably has a setup.cfg

This is also not a problem. pyproject toml is not mutually exclusive with setup.cfg ... only the project field as defined in pep 621 is a replacement for setup.cfg ... but using pyproject in conjunction with setup.cfg is still a first class use case. indeed our cookiecutter still has both files. This isn't pre Jan 2022... its still there! Let me know if I can clarify anything more about pyproject vs setup.cfg

We want to encourage documentation (incl summary, description, metadata) that is relevant to napari users and tailored to their needs
Since many “Python core” fields need to be relevant to all Python users, their napari hub analogs should be customized for napari users

these are lovely abstract goals. but practically speaking, when we go down the list of actual fields in your schema, there are only two (description and maybe authors) that actually provide anything beyond the core metadata. So again, I'd say lets be a bit more conservative about not just adding more engineering until a very clear use case arises where the existing places for this info is unquestionably limiting.

Let’s make sure we’re aligned on the preferred sources for these metadata by adding documentation to napari.org/plugins and updating the napari hub’s `Customizing your plugin listing” docs accordingly

sounds good. I'd say preferred is definitely whatever the python ecosystem currently encourages, which is core metadata (wherever it may be specified depending on the package bundling tool used, eg setuptools or poetry or flit). For customizing your plugin listing. I'd request that you remove any documentation of support for these duplicated fields, until a compelling case arises where someone must enter something different in .napari for a field that they also entered in their core metadata.

The napari hub team can begin to deprecate support for .napari in favor of .napari-hub to prevent confusion with napari.yml...

yeah, I'm agnostic here... doesn't matter to me

For fields which we support in the napari hub config that do not yet have analogs in the napari manifest or pyproject.toml, our team can add support for those sources once you have implemented them into the npe2 spec or formalized them as preferred sources in the documentation.

this bit is slightly frustrating, since right now you have the opportunity to remove these fields like conda and visibility (not deprecate them), since no one has implemented them. That may not be that way forever, and then we'll be in a different situation.

When napari is ready to deprecate support for npe1 and/or we can reasonably expect that all actively supported and used plugins have manifest files, then the napari hub can consider deprecating support for those fields that have preferred analogs in the manifest.

the only field that has anything to do with npe2 and the manifest is possibly labels and visibility. conda can definitely go in pyproject and continue to support npe1. In fact all of these fields could have gone in pyproject.toml and backported support for npe1 plugins that didn't have a manifest. So I don't see deprecation of npe1 as related here.

Similarly, when the Python community is ready to deprecate setup.cfg and/or we can reasonably expect that all actively supported and used plugins have pyproject.toml files, then the napari hub can consider deprecating support for those fields that have preferred analogs in the manifest.

see point above. setup.cfg and pyproject.toml are not mutually exclusive. many many projects have both. perhaps the project field in pyproject is what's being confused here?

[tlambert03]

Just a general comment on all the points here. I understand where the usage of terms like "begin to deprecate" comes from. But when we can do a code search on github and see conclusively that, for example, only one or two plugins has actually implemented these things. Realistically speaking, an abstract deprecation cycle (as would be done when you have no idea who out there is depending on something) is likely overkill here. We could literally contact the 4-5 plugins that would be affected by the core metadata changes, give them a PR to fix it, and then just strip this stuff... no elongated deprecation process necessary

[neuromusic]

I tried to explain why we still intend to support these fields (even if they are largely unused) until npe1 and setup.cfg are deprecated.

Their sparse usage points to the fact that, for the vast majority of plugin developers, their presence as an allowed source is not confusing or adding undue cognitive burden.

[tlambert03]

And I tried to explain why setup.cfg isn't going to be deprecated. And pyproject sans manifest is 100% compatible with npe1

And all I'm asking is that you make it clear in your docs that these fields should ONLY be used in order to overwrite the core metadata when displayed specifically on the hub. Why is this difficult??

[neuromusic]

Why is this difficult??

Please don't mock me.

We have some reasonable disagreements here, Talley, which I'd like to work through with you. I'm working to understand your perspective and I've acknowledged areas where we are in agreement and shared my perspective and/or rationale in areas where we disagree.

I'm confident that we can find a path forward, but I'd like to request that you maintain respectful and professional discourse with me and our team as we do so.

[tlambert03]

there is no mocking intended there, and please don't attempt to cast me as doing so. I'm truly having a hard time understanding why you're so resistant to updating your documentation to encourage our plugin developers to first populate their metadata in the traditional python places, and then use .napari only if you need hub specific information that goes outside of the schema of traditional methods, and why you're resistant to checking in with the napari core development team when it comes to collaborating on how a plugin developer should provide this information.

[neuromusic]

I'm truly having a hard time understanding why you're so resistant to updating your documentation to encourage our plugin developers to first populate their metadata in the traditional python places, and then use .napari only if you need hub specific information that goes outside of the schema of traditional methods

This is precisely what I suggested as our first step in my initial response to your post: "Let’s make sure we’re aligned on the preferred sources for these metadata by adding documentation to napari.org/plugins and updating the napari hub’s "Customizing your plugin listing" docs accordingly"

[tlambert03]

Proposal

there's a lot of text on this page now, so i'd like to re-summarize/simplify my proposal, and invite specific concerns on these points:

1. remove documentation of links (Project site, documentation, support, report issues, twitter) from "customizing your plugin", and instruct users to put it in their core metadata. Help the three plugins that have used them move their data.
2. for summary, help the one plugin that has added it to their .napari file to put it in their core metadata instead. update "customizing your plugin" documentation to state that a plugin should add summary to their .napari file ONLY if they want their summary on the hub to be different than their core metadata summary. (otherwise, they should not use it).
3. for authors, make it clear in "customizing your plugin" that authors should only be used if you want to add an ORCID id, and that it should be used in addition to adding authors to the core metadata (not as a substitute)
4. conda, should be removed from the "customizing your plugin" page ASAP, since it hasn't yet been implemented by anyone, no deprecation is necessary, it can simply be removed. I'd say this field fits comfortably into the heuristic I proposed previously, which got general community support: which requests that "If a change/proposal being considered by the CZI/hub team impacts the actions or API that a napari plugin developer (or the napari program) is expected to perform or use, then that should be a broader discussion with the napari team". I would not have supported putting this sort of installation metadata into the napari-hub specific file.
5. visibility has also not been implemented by anyone but @DragaDoncila in nd2-dask. It should also be removed from the documentation until we come to a consensus on how that should be collected (Proposal: add visibility field to manifest schema, similar to original preview napari/npe2#196).

[neuromusic]

Thank you for the tighter summary.

For 1 and 2 we're agreed.

For 3, I think we need more discussion on preferred sources of author information and take stock of the variety of ways that our users want to report on author information (including institutions, groups, and labs as authors).

For 4, I'm happy to kill this (but not ready to commit to your proposed heuristic... that probably needs to be a separate bullet point)

For 5, happy to explore revising this (esp "hidden") once the dust settles on the npe2 thread, but as I note in the thread on visibility our team will need to maintain some kind of napari-hub-specific opt-out mechanism.

[alisterburt]

A point regarding 3 - authors of the plugin itself should IMO remain a separate concept from the citation associated with the plugin.

Talley's point about supplying author information as part of core metadata is a valid one and whilst adding an ORCiD is definitely cool I don't think it warrants moving us to a situation where we have two sources for such important metadata. Two sources which absolutely will get out of sync -> I would rather see users provide an ORCiD for their account on the hub and have accounts matched to author names on the hub side

4. the heuristic provided is totally reasonable and following it would mean discussions like the above about how to undo certain changes are necessary far less often - this seems like a much better situation to be in

5. reiterating support above for parity with the way napari itself sources package lists within the app, using the framework classifier, unless there is a specific reason this can't be implemented (you mentioned legal? more info would be useful if this is impossible for you)

[neuromusic]

For 1. #629

For 4. #630

[neuromusic]

re: @tlambert03's heuristic, what I can promise is that we won't add support for any new fields in .napari-hub at least until the CZI leads have a chance to meet with the napari steering council to discuss in more depth

[GenevieveBuckley]

Elsewhere, I've been complaining about the problems with data duplication in the docs between the README and description.md file the hub requires.

Talley suggested this is a better place to put that thought, since it's more likely to produce a useful change if the hub maintainers know about the problems the requirement for a separate description file causes.

Here's the full thread: jni/affinder#63 (comment)

GenevieveBuckley says:

And can I just say how much I hate having napari-hub require a separate description file, instead of using the README. You inevitably end up with this type of problem, where one is better than the other and only some of the information is in certain places.

I've heard all the arguments about how they're supposed to be two different documents aimed at different audience, but it just never seems to actually work out better in practice.

Draga replies:

I'm actually +1 to just getting rid of the DESCRIPTION.md and making it the README. Like Genevieve says, the original argument that you'd maintain a separate description for the napari hub is maybe, nice, but I don't think it's nicer than having complete documentation on the docs website and on the repo. Keeping both in sync manually is a nightmare

Draga also says:

The napari hub btw does use the README if one is available and no DESCRIPTION file is present.

But I still don't think that's very good, since it's clear the hub team really want to discourage that:

The napari hub linter tool (or preview tool? I don't know what you call it) absolutely complains at you if there is no description file present. It's absolutely considers my plugins "unfinished" for the napari hub in that respect.

[dgmccart]

Thanks for resurfacing this discussion @GenevieveBuckley! I am happy to help with making sure our documentation and utilities make it clear where plugin devs should put their metadata to avoid duplicating effort and confusion. It sounds like there has been some alignment about this in the thread about DESCRIPTION.md and the github README.

We had taken some steps already to improve our documentation, but we missed the Plugin Preview Page utility and we should audit the rest of our documentation and utilities to make sure they are clear and consistent. I’ve made an issue to track this, and I’ll be leading this effort. Please let me know if you find any other areas that need updating.