Should we allow moving content out of community.general and community.network to collections that are not part of Ansible?

[felixfontein]

Summary

In the past we actively decided not to allow this, but I think it's time to rethink this, especially since we are removing collections from Ansible that are unmaintained, and having things in c.g/c.n that are not maintained in that place anymore moved somewhere else where it is maintained is a lot better IMO than keeping a unmaintained copy just for sake of not breaking playbooks when users do pip install --upgrade ansible. Users can easily install the additional collections they need.

A concrete case where this came up is ansible-collections/community.network#506 - the avi_* modules are pretty unmaintained there (the only change so far was ansible-collections/community.network#218), but vmware.alb is available on Galaxy with the same modules (apparently), and its actively worked on.

So my proposal would be: allow to move content out of c.g and c.n to other collections outside of Ansible, assuming it is announced in time (roughly half a year before the actual remove happens, i.e. announcing now that removal will happen in the next major release which is close to the next Ansible major release is fine).

What do you think, is this good enough to vote on, or do you have other ideas, comments on this, ...?

[ssbarnea]

Yes

[Andersson007]

Considering all what's written in the description:
+1 from me for allowing it. If users are informed in advance, should be OK. No issue with manual installation imo and less confusion with duplicated content one of copy of which being unmaintained

[mariolenz]

+1 from me to allow removing them, even if the new collection is not part of the community package

[mariolenz]

So my proposal would be: allow to move content out of c.g and c.n to other collections outside of Ansible, assuming it is announced in time (roughly half a year before the actual remove happens, i.e. announcing now that removal will happen in the next major release which is close to the next Ansible major release is fine).

Versioning and deprecation:

We recommend to make sure that if a deprecation is added in a collection version that is included in Ansible 3.y.z, the removal itself will only happen in a collection version included in Ansible 5.0.0 or later, but not in a collection version included in Ansible 4.0.0.

It's just a recommendation, but maybe we should eat our own dog food and target Ansible 9 to remove those modules instead of the next Ansible major version (8).

What do you think, is this good enough to vote on, or do you have other ideas, comments on this, ...?

I suggest to vote on allowing to deprecate things in c.g/c.n moved somewhere else, but not when they are really removed. Looks like two different issues to me.

[felixfontein]

And here it gets complicated ;-) As opposed to a regular deprecation / removal (i.e. tombstoning), we are replacing plugins with redirects to another collection. That means that a) we do not deprecate these plugins before, and b) you can run existing playbooks / roles with no change by simply installing the new collection (assuming there was no change in how the plugins/modules work). That's pretty different from regular deprecation/removal/behavior change that the above guideline is about.

(We did have problems with that in the past, but that was when we were still supporting Ansible 2.9 in these collections, and redirects did not work for 2.9 users. They had to actively change their playbooks/roles.)

[mariolenz]

And here it gets complicated ;-) As opposed to a regular deprecation / removal (i.e. tombstoning), we are replacing plugins with redirects to another collection.

Yes, "complicated" is the word. Or, maybe, "an accident waiting to happen" would be even better. What if a collection removes a module next year, or there are breaking changes to it? What if a collection isn't maintained anymore... would you keep the magic redirect to an unmaintained collection indefinitely, or would you move the content back?

you can run existing playbooks / roles with no change by simply installing the new collection

What's so bad about users having to change their playbooks? Happens all the time when collections remove deprecated features / modules or there are other breaking changes in a new major release.

The redirects might be a good way to get rid of stuff in c.g and c.n quickly, but I think deprecating the modules (with reference to the new collection, which the users have to install anyway) and removing them in Ansible 9 is cleaner and more future-proof.

I wouldn't vote agains redirecting, but my preference would be a regular deprecation and removal for what it's worth.

[mariolenz]

@ssbarnea @Andersson007 What's your opinion on redirecting vs. deprecation / removal?

[Andersson007]

@ssbarnea @Andersson007 What's your opinion on redirecting vs. deprecation / removal?

I'm agree with @mariolenz, so I'm personally for removal.

So, provided that the committee agrees on removal, maybe we should add the following points to corresponding policy?

• we can remove stuff only if the superseding collection is available on Galaxy
• we can remove stuff only if the original stuff does not show signs of maintenance by the community

We are not discussing details now, putting this above not to forget

[ssbarnea]

@Andersson007 is right, ok to remove stuff as long the is no replacement or the replacement is hosted under ansible-community organization, just because we want to be able to enable new maintainers to takeover in case current maintenaner becomes unavailable.

Basically, we want to avoid replacing some current modules from community collections with a 3rd party one that is totally outside our control. The last thing we want to see happening is for someone to takeover some modules just to make them later pay-only or include some other code that is not secure (basically security risks).

[mariolenz]

So, provided that the committee agrees on removal, maybe we should add the following points to corresponding policy?

• we can remove stuff only if the superseding collection is available on Galaxy

Yes, makes sense to me.

• we can remove stuff only if the original stuff does not show signs of maintenance by the community

I basically agree, but it depends a bit. If we're talking about a "Moving stuff out of c.g. / c.n." policy, that's a valid point. If we're talking about a "How to deal with unmaintained stuff in c.g. / c.n." it's already in the policy name that it doesn't apply to things that show signs of maintenance by the community ;-P

[Andersson007]

So, provided that the committee agrees on removal, maybe we should add the following points to corresponding policy?

• we can remove stuff only if the superseding collection is available on Galaxy

Yes, makes sense to me.

• we can remove stuff only if the original stuff does not show signs of maintenance by the community

I basically agree, but it depends a bit. If we're talking about a "Moving stuff out of c.g. / c.n." policy, that's a valid point. If we're talking about a "How to deal with unmaintained stuff in c.g. / c.n." it's already in the policy name that it doesn't apply to things that show signs of maintenance by the community ;-P

@mariolenz yep, i'm talking about moving out stuff, e.g., about cases when some folks copied stuff from a community collection to another collection and want to remove the same stuff from the community collection but there's alive community there and there's no consensus among all the folks to go to the new repo.
But it's definitely a detail, so let's decide on the main topic first:)

[Andersson007]

@Andersson007 is right, ok to remove stuff as long the is no replacement or the replacement is hosted under ansible-community organization, just because we want to be able to enable new maintainers to takeover in case current maintenaner becomes unavailable.

Basically, we want to avoid replacing some current modules from community collections with a 3rd party one that is totally outside our control. The last thing we want to see happening is for someone to takeover some modules just to make them later pay-only or include some other code that is not secure (basically security risks).

@ssbarnea good points!

[felixfontein]

There is a new case (well, not so new anymore): cloud.terraform which wants to take over the community.general.terraform module, but cloud.terraform is not part of Ansible. See also #192 (opened to discuss that specific case).

I think we should decide on this general case first before starting to actually remove or deprecate something.

[felixfontein]

Turns out we already have a similar discussion: #66. I'll close that one in favor of this one.

[felixfontein]

We really need to decide on this in some way or another.

[felixfontein]

Proposal:

We allow to move content out of c.g and c.n to other collections outside of Ansible under the following conditions:

1. The new collection is appropriately licensed and does not require a CLA to contribute.
2. None of the contributors who contributed to the content in the last 6 months objects in a four-weeks period after the plan to deprecate the module has been announced.
3. There is a deprecation period of at least 6 months during which deprecation warnings are shown. If community members or contributors bring up good reasons in these 6 months to not do the move, the Steering Committee will discuss these and vote on them before the content is removed.

Redirects are only added if full backwards compatibility can be ensured.

What do you think? Does this sound reasonable / acceptable / ...?

[samccann]

Should the deprecation notice also mention that the modules are moving to a collection NOT in the Ansible package and thus they need to install it separately?

[felixfontein]

Sounds like a good idea!

[felixfontein]

@ansible-community/steering-committee could you please take a look at #167 (comment) and @samccann's idea? Would be really nice if we can conclude this topic eventually.

[gundalow]

Would the error message displayed to a user be clear enough?
Can/do we we need to extend ansible-core's tombstoning code to give clearer guidance when it can't find a module?

[felixfontein]

If we use tombstoning, we should make sure that the error message contains enough information. Something like The module foo.bar.baz has been moved to the collection bam.bar. Please use bam.bar.baz instead. Note that the module might not be fully backwards compatible with the module from foo.bar. maybe?

If we use redirections, I'm not sure whether we actually should add a warning. Having a warning makes using the old short name annoying (you always get this warning!), and we alienate existing users. On the other hand, if the users do not have the new collection installed, they already now get a mostly useful error message: The module baz was redirected to bam.bar.baz, which could not be loaded. It doesn't say that the collection bam.bar isn't installed, so you cannot distinguish between the collection not being installed, and a version of the collection installed that does not have this module. This is something that can definitely be improved.

[Andersson007]

@felixfontein i like your conclusion, would you like to create a forum topic instead of this topic? I think we should allow to move the stuff out to avoid frustration and put it explicitly as it's not being maintained in c.network anyway.

[mariolenz]

@felixfontein Please close this issue if done, or open a new forum topic and then close the issue with a pointer to the new discussion: Community-topics: Archiving the repo

[felixfontein]

I created https://forum.ansible.com/t/should-we-allow-moving-content-out-of-community-general-and-community-network-to-collections-that-are-not-part-of-ansible/5738 so we can continue this discussion in the forum.