Develop more complete EOL policies

[tfoote]

Filing this to keep track of my discussion with @mikaelarguedas

When an Ubuntu Distro goes EOL if we remove it from the rosdistro it stops being resolvable via rosdistro APIs. #18151 (comment) and #18235 A potential solution for this is to take a snapshot of the rosdistro contents and make a static rosdep.yaml file with the contents for people to add to their sources.

Related we also noticed that rosdep update is pulling a lot of old content even on newer platforms. Due to groovy and newer being in the index.yaml We should also have a policy for retiring these keys.

And if we're snapshotting EOL distros, that might also be an opportunity to prune those keys from the main rosdep database. The snapshot could include both the rosdistro results and the distro specific rules from the main rosdep database. And then we could retire those from the main database to allow us to simplify.

[nuclearsandwich]

Related we also noticed that rosdep update is pulling a lot of old content even on newer platforms. Due to groovy and newer being in the index.yaml

As announced rosdep no longer pulls out-of-support distros by default. I like the idea of taking snapshots of rosdep keys when retiring distros. Perhaps we can start with just branches or tags on this repository?

[mikaelarguedas]

Gentle bump here, is it possible to create tag like indigo or indigo-eol ?

Now that some rules have been changed for some EOL ROS distros while their host Linux distros is still active, a tagged version of the rosdep database would be valuable for people still using the EOL ROS distro.

Similarly is would be useful for people still using e.g. trusty ELTS.

[mikaelarguedas]

@tfoote @nuclearsandwich friendly ping

[dirk-thomas]

@mjcarroll @tfoote Since Eloquent and Kinetic are the next ones to be EOLed.

[tfoote]

We now are tagging the rosdistro on every sync which includes the rosdistro state and rosdep keys that were aggregated at the time.

As such if you want to revive something on an EOL distro or EOL rosdistro you can checkout from those tags.

And it takes a little bit of work, but you can also setup rosdep sources to pull from the tags which will reenable them from the snapshot.

We have added the end-of-life status for the rosdistros which makes them clearly EOL. And the rosdistro keys can be removed when there are no longer any rosdistros supporting them. As a compromise we'll keep the keys for distros that go EOL during the lifetime of the rosdistro and only clean them after. It's a small window but shouldn't hurt. And note that I suggest that we only clean on touch of a rule and not expect to do bulk updates to remove rosdep keys.

[tfoote]

More discussion at: #31569

[mikaelarguedas]

We now are tagging the rosdistro on every sync which includes the rosdistro state and rosdep keys that were aggregated at the time.

I really like the idea of having a "final" tag in the snapshots apt repository. Would it be possible to follow the same logic here and tag {ROS_DISTRO}-final when a distro goes EOL ?

[tfoote]

Tagging the rosdistro with final probably does make sense using the safe ref as the last dated snapshot. I'd suggest ${ROSDISTRO}/final as the format to be consistent with the sorting as well as the snapshots.

[mikaelarguedas]

@tfoote do you think we could experiment this for the upcoming melodic EOL ?