Add a script for generating import configurations from the live Ignition repositories.

[nuclearsandwich]

This script reads data from the packages.osrfoundation.org repository and prints out a reprepro-updater import configuration.

Caveats and addendums

• Right now it only supports the -stable repositories, and hard codes the packages in each collection (taking the info at authorship time from ignition-tooling/gazebodistro but does not support fetching the data directly from it at runtime.
• Another limitation is that it looks only at the binary-amd64 package information so it implicitly assumes no disparities between architectures.
• The parsing of the Debian Packages file is extremely rudimentary but presently sufficient in order to avoid dependencies outside the Python standard library.

But that's enough with the caveats, what cool things does it do?

With a builtin list of igniton collection source packages, it will find and group up all of the binary packages generated by that source and create a valid reprepro import configuration for use in the ros_bootstrap repository. This will make it much easier to keep the bootstrap repository in sync with the current releases from the packages.osrfoundation.org repository and could eventually be run out-of-tree as part of the upstream team's release automation.

#143, #150, #152, and #153 are all created with iterations on this script

[nuclearsandwich]

No reviews yet but FYI @j-rivero @chapulina

[chapulina]

Right now it only supports the -stable repositories

This should be enough for our needs 👍

[nuclearsandwich]

@j-rivero I only just learned about gazebo-tooling/gazebodistro#4 today and it looks like we thought very similarly here.

What I like from the script in that repository is that it uses the gazebodistro repos files at runtime to get the base package information, and then includes all dependencies of selected packages from the packages.osrfoundation.org repository.

What I like about this script is that it generates the exact configuration file reprepro-updater wants to consume and it can be run on any host platform rather than being dependent on the system repositories for state.

I think we should talk through the current status of each script and see which one to concentrate our efforts on.

[j-rivero]

I'm giving a try to the tool and can not make it work with python 3.8 and 3.9.5:

❯ python3.9 generate_ignition_import_config.py 
Traceback (most recent call last):
  File "/home/jrivero/code/infra/reprepro-updater/scripts/generate_ignition_import_config.py", line 44, in <module>
    class PackageGroup:
  File "/home/jrivero/code/infra/reprepro-updater/scripts/generate_ignition_import_config.py", line 46, in PackageGroup
    packages: None|List[str] = None
TypeError: unsupported operand type(s) for |: 'NoneType' and '_GenericAlias'

Is this kind o hinting support in newer versions or there is a bug somehow?

@j-rivero I only just learned about gazebo-tooling/gazebodistro#4 today and it looks like we thought very similarly here.

What I like from the script in that repository is that it uses the gazebodistro repos files at runtime to get the base package information, and then includes all dependencies of selected packages from the packages.osrfoundation.org repository.

What I like about this script is that it generates the exact configuration file reprepro-updater wants to consume and it can be run on any host platform rather than being dependent on the system repositories for state.

I think we should talk through the current status of each script and see which one to concentrate our efforts on.

+1 Did not really like the gazebodistro PR so +1 to merge efforts here.

[j-rivero]

Suggested change

	exit(2)
	group.packages.remove(p)

The check is nice to have. However It can detect problems with versions for orphaned packages, like the bad named libgz-math6-eigen-dev now renamed to libgz-math6-eigen3-dev. If we keep the exit after the check, the script can not continue.

A possible path forward would be to leave the warning but remove the package from the expected imports.

[azeey]

@azeey