Please break the cyclic build dependency

[infinity0]

It seems there is a cyclic build dependency in this git repository:

    cmdliner
     ^    |
     |    v
{ topkg, topkg-care }

Package managers in general don't support this sort of thing, where a single source repo/tarball contains two sub-packages. The reason is because things can easily go wrong - if the version numbering histories of the sub-packages end up (accidentally or on purpose) with mismatching versions then it can be very confusing to recover the actual source code for a given sub-package at a given version.

opam apparently supports it, but I'd ask you not to take advantage of this even if you think you can do it correctly (by keeping the version numbers in lock-step) because it makes it hard to port to other distros' package managers.

Even if you are disciplined and never make a mistake, the fact that the two packages share version number histories (along with the fact they are stored in the same git repo) suggests "intuitively" that they belong to the same "unit-of-distribution". From this perspective, it is a true circular dependency, topkg.git depends on cmdliner.git and vice versa.

In order to build this in Debian or other distros, we either have to package these things as three separate Debian packages, or one big Debian package. The first option is probably cleaner (in this case) than the second option, but we would still have to duplicate this source package and manually write scripts to build specific parts {topkg, topkg-care} in each duplicated instance. This is not ideal infrastructure-wise, and it also destroys any hope of creating a tool that can automatically convert opam packages to Debian packages, and using it on this package.

For these reasons, I ask either that topkg-care be split off into a separate git repo, or for the cyclic dependency to be broken some other way.

[dbuenzli]

There is no cyclic dependency, there are three packages cmdliner, topkg and topkg-care the last two share the same tarball but have different build instructions see the topkg.opam and topkg-care.opam files. I don't see how this poses a problem for you or how this would prevent to automatically translate opam packages to debian packages.

[dbuenzli]

This seems rather a debian packaging mistake.

[infinity0]

It is a cyclic dependency if we translate packages in the intuitive manner, each "tarball" as a source Debian (or other distros') package. Distribution's package builders build each package individually, on a different machine as the build of another package. There is no way to "interleave" the build of topkg, then run the build of cmdliner, then run the build of topkg-care.

Thus, one option is to split topkg and topkg-care into two Debian packages. This is not ideal because it would involve duplication of the tarball, and manual work as I described before. Yes, it is possible to "make it work", but it is not a good situation and that is why it is not supported in Debian on purpose.

From another "package manager design" perspective: Not only do {topkg, topkg-care} share the same tarball, but they share the same version-numbering histories. Because of this shared version history, calling topkg and topkg-care "separate packages" is not conceptually sound, even if the opam designers thought it was an appropriate thing to support. Normally, truly-separate packages do not have shared version-numbering histories.

Pretending that two packages are separate, but still releasing them in lock-step with each other is a confusing thing to do. OTOH, not releasing them in lock-step, but keeping them in the same git repo, also leads to confusion later. From the description topkg-care does sound like it is supposed to be a truly-separate package from topkg, and the design of it would probably benefit if you did split it off and allowed its version to vary independently of topkg - you would be forced to design the API of topkg more carefully (so that it works with more versions of topkg-care), which would be beneficial as this allows other third-party tools to integrate better with topkg.

[dbuenzli]

It is a cyclic dependency if we translate packages in the intuitive manner, each "tarball" as a source Debian (or other distros') package.

Then don't. Just follow the structure we have in opam it's perfectly sound (and automatable).

It seems to me that you simply have a viewpoint problem. If the current situation worries you too much I suggest you rename the tarball for the topkg-care package to topkg-care.tbz and pretend it's not the same as topkg.tbz.

you would be forced to design the API of topkg more carefully (so that it works with more versions of topkg-care), which would be beneficial as this allows other third-party tools to integrate better with topkg.

If I had more time on my hands I would certainly be very interested in knowing what you think is not carefully designed in topkg and learn about all these benefits you mention but I'm afraid I don't.

Aside, with the general move away of depopts (which I personally don't necessarily see as the long term solution but then...) you should be prepared to see more and more of this kind packages coming from the OCaml world. With appropriate automated tooling for translating them to debian package I still don't really see the problem here.

[infinity0]

There is no problem with automatically translating a single git repo with several opam packages, if they don't introduce cyclic dependencies. Indeed, I agree with this statement from the post you linked:

The majority of sub libraries don’t need their own bug trackers and git histories.

We don't want separate bug tracking pages and Debian package version histories for sub libraries either, we want a single Debian package for a single OCaml project.

OTOH, I am trying to explain to you why the current situation of topkg does not fit "intuitive ideas" about what "a project" is. As a project, a unit of software-development, topkg is cyclicly dependent with cmdliner. I cannot build the whole project, without building cmdliner in the middle of the build; and I cannot build cmdliner by building its dependencies as whole-projects - I have to build a sub-part of the topkg project.

You have worked around this by splitting it into several opam packages, then untying the cyclic dependency in terms of opam package dependencies. But this is only one technical definition of "dependency", in the small viewpoint of opam packages. When developing software projects, other types of dependency exist, and it's these "intuitive ideas" on what a dependency is, that have driven the design and development of distributions' package managers. What is reasonable or intuitive about having to pause the build of one project, start partially building another project, resume the build of the first project, and so on?

[dbuenzli]

So it seems the only problem is that I challenged your intuitions. So be it. There are two projects in this git repo topkg and topkg-care I hope people in charge of packaging on the debian side will be sufficiently smart to realize that.

[infinity0]

"Possible" does not imply easy, good, or worth the cost. Suppose we wanted to persuade someone to program something in OCaml and not assembly. "But you can do anything with assembly code", you reply. "I've challenged your intuitions about what good coding practises are!" But the point is really elsewhere. It's a shame that you're not willing to put in the effort to understand these points, but are only interested in pointing out the fact that my arguments about a non-logical topic aren't fully logically watertight, thinking that this is an adequate defense against criticism.

[dbuenzli]

"Possible" does not imply easy, good, or worth the cost

That's precisely what you don't understand. The cost is much lesser for me that way, I manage approximatively 30 OCaml packages and I need to be efficient in this (see here for more on this). With appropriate tooling on the debian side which is perfectly possible by using the opam v2 API I don't think it will incur a cost there.

It's a shame that you're not willing to put in the effort to understand these points, but are only interested in pointing out the fact that my arguments about a non-logical topic aren't fully logically watertight, thinking that this is an adequate defense against criticism.

That's a very nice sentence but I'm afraid it doesn't make much sense in the context of this discussion. Maybe you should be a little bit more open minded and try to understand why things are done in a certain manner before starting yelling too much and "explain" me things and try to impose your viewpoint about things.

Now I have lost a little bit too much time with you so I'm simply going to lock that issue.

Goodbye.