Allow meta-packages for distribution

[sebasguts]

Some projects, for example CAP and homalg, consist of multiple packages. Currently these packages are distributed independently.

It would be nice if one could produce meta-packages, containing all packages belonging to project. Benefits would be the following:

• Coherent versions among the packages in a project
• Easier distribution of projects
• More structure in the pkg dir, as all packages of a project would be in a subfolder.

[olexandr-konovalov]

To collect some aspects of this to think about:

• how such projects will be distributed and picked up for redistribution with GAP (individually, or as a single archive)
• which new fields in PackageInfo.g files will be needed to reflect this
• how projects will be presented on the GAP website

[olexandr-konovalov]

Instead of commenting on #2267 which I have no time to review now, I will post a quick note here.

My concern at the moment is that any changes which will make possible to build packages and to load packages located in subdirectories should also work with the existing package redistribution mechanism, or allow its adaptation.

If a meta-package would have PackageInfo.g file and will have packages in subdirectories, then the current system for checking for package update would not require any modification at all. I will simply add new package for the redistribution, and will remove from the redistribution all packages that will now be part of this meta-package.

Scripts that produce auto-generated pages with package overviews for the GAP website would require some adjustments, but that does not seem to be an obstacle.

[olexandr-konovalov]

(comes from #2267).

As I said above, one of the questions is which new field(s) in PackageInfo.g files will be needed to reflect this. For example, there could be an entry in PackageInfo.g which lists names of subpackages forming the project, for example

PackageComponents:=["ExamplesForHomalg", "GaussForHomalg", "GradedModules",...],

Then package loading will traverse subdirectories ("ExamplesForHomalg-2017.09.02", "GaussForHomalg-2018.02.05" etc.) , but will assume that they contain packages that have to be loaded if and only if they have PackageInfo.g file for a package whose name matches the one in PackageComponents. This will thus allow to reject gh-pages directory (since it will have PackageInfo.g which will provide a name of a meta-package, but not the one from PackageComponents).

[olexandr-konovalov]

We need also to identify things which have to be updated following the proposed change:

• bin/BuildPackages.sh if meta package has a component that requires compilation (e.g. Gauss which is a part of homalg)

[olexandr-konovalov]

Remark: Currently we test neither CAP nor homalg during releases, and we rely only upon the internal testing made by their development teams. When we will be able to handle meta packages, I would be glad to see the TestFile component in the "meta" PackageInfo.g file and a tst directory in the root directory of a meta-package, that will have tests to exercise all sub-packages forming a meta package.

[olexandr-konovalov]

To summarise, the following aspects should be decided for meta-packages:

• Why authors may want a meta-package (to help further package authors to decide)
• Loading in GAP
  • whole meta-package
  • individual component of a meta-package
• Structure of PackageInfo.g
  • versioning of the meta-package
  • fields to specify that this is a meta-package or a component of a meta-package (with some fields becoming unnecessary)
  • adjusting ValidatePackageInfo.
• Providing and running standard tests
• How a meta-package should be documented?
• Distributing them for picking up by the GAP package update system
• Presenting them on the GAP website
• Building components with bin/BuildPackages.sh

When: would be desirable to have this in place for GAP 4.11

How: set up a mock meta-package with some mock components, add it for the redistribution with GAP., but exclude from all package archives. Start to adapt existing projects (CAP, Homalg,...) only when this will be well tested.

[fingolfin]

Closing, see #2267 (comment) for an explanation