One opam file for multiple packages

[AltGr]

This could make it much nicer to handle for packages with multiple definitions from the same source.
Quoting @avsm:

name: foo
maintainer: anil
dev-repo: blah.git
build: [ dune blah blah ]
package mirage {
  depends: [...]
}
package unix {
  depends: [...]
}

Instead of having mirage.opam, unix.opam.
The idea could be as simple as "interpret the file as multiple foo.opam files, for foo as each package * section, and the contents the common part + that section"

[samoht]

One of the UI question then becomes: how do you install the sub-package unix? For instance:

$ opam install foo-bar-unix

This could lead to the following resolution scheme:

• check if foo-bar-unix is a concrete package
• if not check if unix is a sub-package of foo-bar
• if not, check if unix is a sub-package of bar, a sub-package in foo
• if not, fail with a nice error message.

[AltGr]

Ah, my understanding of it so far was more limited, here is what I had in mind:

• this is specifically for in-source opam files, not for the repository (indeed that would end up with name clashes)
• there is no automatic prefixing nor package hierarchy: you just specify a list of packages in a shorter way, and in a single file

Basically, it would be like assuming you have a command expand_opams that does

type multi_opam = defs * (pkgname * defs) list
let expand_opams (base_defs, sub_pkgs) =
  let pkgs =
    (if has_field "name" base_defs then [get_field "name" base_defs, base_defs] else []) @
    List.map (fun (pkgname, defs) -> pkgname, base_defs ++ sub_pkgs) sub_pkgs
  in
  List.iter (fun (pkgname, opam) -> write (pkgname ^".opam") opam) pkgs

and that is (virtually) run before opam does the search for package definitions (where ++ is by-field overriding merge).

Would that be sufficient, or at least a good start ?

It may probably be also worth discussing adding hierarchical packages to the repository, but IMHO in that case it would be better to start from the repository shape point of view rather than package definitions. What do you think ?

[samoht]

But what is the concrete syntax to install sub-packages? You mean opam-publish will have to expand these files during the submission?

[AltGr]

yes, I guess referring to the directory or opam file would include them all (for opam or opam-publish), and they would be internally considered as expansed (i.e. publish would submit n packages) ; and you would otherwise refer to them individually by name.

Also, opam pin foo . would extract the definition for foo from the file.

[dbuenzli]

It seems to me that this is significantly complexifying the current opam model (I also already see problem with the install of opam files in lib/PKG/opam which odig uses).

If this is only for pins (as far as I understand). Why don't we rather define a protocol (step) that given the package name allows to generate the opam file from the pinned source to use when you pin a repo.

[dbuenzli]

I have opened #3797 which from a workflow perspective would, if feasible, provide a better answer to the problem than this proposal (and to a few others along the way).