Process for orphaning/deprecating modules in community.general and community.network

[felixfontein]

Summary

There are a lot of modules and plugins in community.general (and I guess also community.network, and probably in some more collections) which are basically orphaned, partially/potentially broken (we don't know because we don't know if anyone actually uses them), and we have no idea what to do with them.

An example is the serverless module in community.general (ansible-collections/community.general#2558).

I think we need a process to mark such modules and plugins as orphaned, and if nobody is interested in some time, deprecate and finally remove them. Maybe orphaning it for one major version (i.e. mark it as orphaned, mention that in the changelog), then deprecate it, and two major versions later remove it in case nobody mentioned any interest in the module.

(For marking it as orphaned we can use ansible/proposals#68, though that implementation seems to be still in flux. It should be available for ansible-core 2.12 tough.)

[Andersson007]

I suggest moving orphaned modules to a special collection. I don't think they'll eat a lot of disc space there. If it seems to be sensible, the questions are:

1. Should the collection be shipped with Ansible?
2. If yes, should we redirect to the moved modules from community.genera, community.network to make the move transparent for users?

My personal view is that we shouldn't ship it. We could mention in the deprecation warnings that the content will be moved to the special collection. If there are users who use the modules, they can continue to use them by installing the collection manually.

[tadeboro]

I did not think about this much yet, but one question popped into my head: how is the orphaned status communicated to the user? We will probably document this, but is this "enough"? I have a feeling that if we only announce the orphaned status in documentation, quite a few users will miss it and only start to react when we deprecate the module. So why should we even bother with the orphan status in the first place?

[felixfontein]

Hmm, documentation is the only way we can announce this. We can also have a list of orphaned modules in the collection's README, though that's as easy to miss. Also, for collections that use the collection_bot, the bot can add a label or message to all PRs/issues for orphaned plugins informing that the plugin has been orphaned.

But even if it's "just" in the documention, I still think it is important that there's a way to declare content orphaned.

[aminvakil]

What would happen if a module has been marked as orphan, then show deprecation warnings in case someone uses them, and someone opens an issue that they're willing to maintain the module?
Remove deprecation message and make them maintainer of module?
If there was already one major version which this module has been marked as deprecated, ignore the new maintainer message?

My main concern is that specifying strict criteria for making a module orphaned, deprecated, removed is hard and during this process how can that be taken back and remove module from orphaned modules?

[mariolenz]

"orphaned" as a state between (fully) supported and removed sounds like a good idea.

[felixfontein]

We discussed this a bit today. Some ideas/... from the discussion:

1. For indicating orphaned plugins, attributes would be great. IMO we should think of separating maintenance information from usage information. Plugins should only be removed if they are neither maintained nor used (and of course there has to be a deprecation period).
2. To figure out which ones are no longer maintained and which ones are no longer used, this is somewhat harder...
   1. We can look at recent releases of roles and collections on Galaxy to get a first impression what's still in use
   2. The vscode plugin might be able to provide some telemetry. @ssbarnea do you know about that?
   3. We could offer a script which scraps playbooks and task files for plugin usages, and/or offer a special callback for that, (at least for modules, other plugins could be harder) and provide a service where folks can voluntary upload such information.
   4. Of course allow manual reporting.
   5. We could also count data on contributions to the plugin, and number of issues reported (that haven't been solved). Probably some triaging on which issues are relevant would be helpful (feature tracking issues or bugs that nobody can reproduce mean issues open for a long time, without indicating that something is wrong).
3. Regarding the procedure, we should start with documenting the findings, then maybe (at least one major release later) get ansible-lint to report such plugins, and only later (at least 1-2 major releases) start adding deprecation warning (with at least two major versions for removal). Here I'm assuming c.g/c.n with a major release every six months, other collections can of course use intervals more fitting to their release schedule :)

[felixfontein]

(ansible-core and AWX/AAP do not and will not provide telemetry, so it's not worth suggesting that, if anyone thinks about doing that :) )

[ssbarnea]

We plan to add some opt-in telemetry to vscode extension but I do not think that would be a the best place to gather info about production use of a collection or module.

We could add some telemetry to ansible-lint too and also easily implement a "pending-deprecation" warning for specific modules/collections. The big question is how we should manage the list of pending-deprecation stuff.

While I would personally love to have have enabled-by-default telemetry in linter, I think that Red Hat has a strong policy against doing this. This also means that we will almost never be able to get a good glimpse about which modules/collections are used in the wild or not, as most users will not have telemetry enabled. Not sure how we can address this part.

I personally do not want to manage this list inside the linter as it require lots of changes and releases, to add/remove stuff from that list.

Deprecating a plugin/collection by making a new release of it is also not sustainable, especially because this happens when that code is not well maintained

We need a very easy to update list of proposed deprecations and use a REST call to check them. The downside is that this feature will not be available when ansible-lint runs in offline mode.

We could add a file to a repository like schemas or devtools which do not ship anything by themselves, so making a change is just a PR, one that can also be done from within the browser.

[felixfontein]

We plan to add some opt-in telemetry to vscode extension

What kind of telemetry would the plugin collect? Would that include which plugins/modules are used in playbooks/roles?

but I do not think that would be a the best place to gather info about production use of a collection or module.

It's definitely not hte best place, but it's a lot better than nothing (what we have so far).

We could add some telemetry to ansible-lint too

As long as it is opt-in, getting information from there would be useful as well.

and also easily implement a "pending-deprecation" warning for specific modules/collections. The big question is how we should manage the list of pending-deprecation stuff.

For collections that are actively maintained, like community.general, I would let the collection manage this.

While I would personally love to have have enabled-by-default telemetry in linter, I think that Red Hat has a strong policy against doing this.

I personally would also stop using ansible-lint if it starts sending telemetry by default.

This also means that we will almost never be able to get a good glimpse about which modules/collections are used in the wild or not, as most users will not have telemetry enabled. Not sure how we can address this part.

I don't think we can ever get such a good glimpse, but I also don't think we really need that. What we need is a good enough approximation, and combining infomation from different sources (opt-in telemetry combined with combing roles/collections in Galaxy combined with combing GitHub / GitLab / ...) should give a good one IMO.

I personally do not want to manage this list inside the linter as it require lots of changes and releases, to add/remove stuff from that list.

I'm not sure what you mean with "this list". Are you talking about usage telemetry, or about the data derived from it?

Deprecating a plugin/collection by making a new release of it is also not sustainable, especially because this happens when that code is not well maintained

Deprecating a plugin must happen in a release. There is no other way to do it IMO, if you want a proper deprecation. (Warning that it is considered for deprecation, or seems to be unmaintained, these are totally different things.) Deprecating a collection (from Ansible) can be done without involvement of the collection.

[mariolenz]

This issue / discussion looks stale (last activity nearly 1 1/2 years ago). If someone wants to discuss this further, I suggest to open a topic in the forum.