Provides / Exclusive ports / Alternates

[sdcb]

Some packages shared same header and cannot install them all at once, like:

• OpenSSL/BoringSSL
• libmysql/libmariadb
• GLUT/freeglut

Can we add a definition in CONTROL defines the exclusive port, and give a user-friendly notice when installing the exclusive library.

[Orvid]

As a side question, how should dependencies on one of the conflicting packages be handled? For BoringSSL, perhaps a section in the CONTROL file reading Provides: openssl would make sense? It makes sense for these sets of packages, where one is specifically intended to implement the API of another, what of packages that conflict for completely unrelated reasons? Do such packages exist, and, if they did, would we really want them in the main port repo?

[ras0219-msft]

This is definitely a design decision with several reasonable alternatives.

First, the immediate issue of "package A has the same headers as package B". We have a list of every currently installed file (which powers the vcpkg owns command) and we should use this to provide reasonable errors on installation that the package you're trying to install conflicts with an existing package. In the future, it would be ideal to track and perform the same analysis for DLL exports as well (we currently implicitly require that every symbol is owned by a single library).

Second, is the larger question of what to do about alternatives? The examples @sdcb listed cover the two most interesting scenarios from a library author's point of view:

• OpenSSL/BoringSSL BoringSSL explicitly provides a subset of OpenSSL and it's conceivable that a library will need the full API surface of OpenSSL (which is the default).
• libmysql/libmariadb libmariadb explicitly provides a superset of libmysql, and it's conceivable that a library would require the non-default.

From the final user's perspective, I think there are a few core scenarios.

• Installing a small set of libraries from the top should just work as reliably as possible without bothering me with (to me) irrelevant choices.
• I need the ability to state that "I want to use BoringSSL instead of OpenSSL. Try to build the world with that replacement and I accept any build breaks because of it."

I can see two overarching choices here:

1. Give ports the ability to require particular alternatives. This means that lots of ports will end up being overly restrictive and, in the end, neuter my ability to select what implementation I want for my application. On the other hand, all allowed configurations are more likely to work.
2. Only allow ports to require the overall "contract". Every port will require openssl, and that will be filled by the alternative that the final user chooses (OpenSSL, BoringSSL, LibreSSL, etc). The build will be attempted and I will see build failures when the configuration I've chosen in fact does not work. If upstream updates to support the alternative, these scenarios will start working without any effort on the portfile maintainer's part beyond the usual "update to latest stable".

The core principles of Vcpkg are to serve the final user's wishes as much as possible, even at the expense of expression in the portfiles or static analysis. This is along the same lines as why ports cannot specify version requirements in their dependencies; it is entirely left for the end user to decide what version they will use. Because of this, I am heavily on the side of option 2 above.

I have some ideas for how option 2 could be implemented (e.g. Provides:), but I'd like your feedback on the situation first. Please make sure I haven't omitted anything!

[albertziegenhagel]

I overlooked that issue and posted a new one. Sorry for that. I think most of the ideas I has are very similar to the thinks discussed in here. This is what I originally posted in the other issue:

This is a problem that vcpkg currently does not have, but that may will arise in the near future, so I wanted to start a discussion on it:

Some libraries do have dependencies to not one specific library but rather to a standardized interface that can be implemented by multiple different libraries. Examples are BLAS [1] or MPI [2]. While a library consuming BLAS in general does not care how the required functions are implemented there are many different vendor implementations where each of them has some advantages and disadvantages over the other ones (OpenBLAS, Intel MKL, reference (netlib) BLAS/LAPACK, ...).

Vcpkg should provide support where a user can choose which implementation he wants to use.

I had a quick look at homebrew and at least for MPI they have defined a special requirement :mpi [3] that can be satisfied by both open-mpi and mpich2. But I don't really like this kind of explicit implementation for a specific requirement. Instead I would propose the following:

• A port that depends on MPI would state mpi instead of the specific implementation (currently msmpi) as its dependency.
• In the CONTROL file of a port, we add an additional group called Provides (name can be changed). If that group is not specified a port by default always provides an interface that is called the same as the library. So the msmpi port as currently would provide msmpi only. If the port now additionally states that it provides mpi, the set of provided interfaces by the port would be mpi and msmpi. This way an other port would be able to depend either on the generic mpi or the specific msmpi if needed.
• Checking for dependencies would then be done based on the interface sets instead of library names.
• A user should not be able to install two libraries that provide the same interface. We would not need to state conflicts explicitly.

The only question that remains in my opinion is, if the user wants to install a library that depends on mpi but he does not yet have an mpi implementation installed. homebrew solves this by providing a default mpi implementation (which is open-mpi) that will be installed automatically in those cases. We have two ways to to the same. A port could specify that it wants to be the default implementation for an interface. E.g. in the CONTROL file

Provides: mpi [default]

An other possibility would be to have a separate list of default ports that could look like this:

mpi: msmpi
blas: openblas
lapack: clapack

The benefit of the second possibility is that we could look up the default ports quickly (we do not need to red all CONTROL files) and we guarantee no conflicts (what if two different ports state that they are default?)

The reason why I am bringing this is up is not really MPI (I do not know of any other free MPI implementation on windows than MSMPI) but it is BLAS and LAPACK. Currently we do have a port for BLAS that uses OpenBLAS, but it is compiled without the assembly code enabled, what in my opinion contradicts the idea of BLAS: it should provide bleeding fast linear algebra operations, but the C reference implementation that is currently compiled by vcpkg is the opposite of that. From very quick benchmarks that I run, I saw that the reference BLAS implementation compiled with a fortran compiler is about four times faster. For LAPACK it is similar, since we currently compile an outdated version where the original Fortran code has been converted to C code by a tool. While those libraries are very good as a default implementation (since they to not need a fortran compiler) it would be nice to have the possibility to choose a different implementation, when a fortran compiler is available.

What do you think?

[1] https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms
[2] https://en.wikipedia.org/wiki/Message_Passing_Interface
[3] https://github.com/Homebrew/brew/blob/bbed7246bc5c5b7acb8c1d427d10b43e090dfd39/Library/Homebrew/requirements/mpi_requirement.rb

@ras0219-msft mentioned the following:

One other way to have defaults: have the interface name be the default port. So, the interface would be openssl and it would be additionally provided by libressl and boringssl.

Having the interface name being the same as the default port would not allow a port to specify that it wants explicitly that library. So if port works with openssl only, and not with libressl or boringssl, it would have no way to specify that.

[albertziegenhagel]

@ras0219-msft would you accept a PR with an implementation as proposed by me above? Maybe I can first write a specification in the docs folder as you did in #1205

[ras0219-msft]

A brief spec would be great; I think a lot of wording could be copied from these issues as well to help fill it out quickly!

Having the interface name being the same as the default port would not allow a port to specify that it wants explicitly that library. So if port works with openssl only, and not with libressl or boringssl, it would have no way to specify that.

That's true. I really dread giving libraries the ability to specify incompatibilities like that (A -> B, C -> D, B and D conflict) since it's almost certain to be overly restrictive. I guess we can just rely on our public curation to stop that from happening.

An other possibility would be to have a separate list of default ports that could look like this:
mpi: msmpi
blas: openblas
lapack: clapack

What about making those "interfaces" into just empty ports with a dependency on their "default" implementation? So, for example:

Source: ssl
Version: 1
Description: Metapackage for the OpenSSL interface
Build-Depends: openssl
# Maybe something to specify that this is a dependency-only package?

Then, the other libs can have Provides: ssl. This would

• Not require loading the entire package graph to resolve the default.
• Minimize the amount of "new" syntax and config files -- they're just (minimal) packages!
• Still allow packages to depend directly on openssl.

There might still be some internal "magic" that has to happen when handling these packages though, since I would imagine the naïve implementation to result in

>vcpkg list
boost
ssl
openssl

or

>vcpkg list
boost
libressl

which is arguably not quite right. Maybe whenever you install a package that has Provides: X, the system always generates and installs a virtual X package that has the proper dependency? That would also make removal work correctly (remove libressl -> remove ssl -> remove boost).

[albertziegenhagel]

What about making those "interfaces" into just empty ports with a dependency on their "default" implementation?
[...]

This sound like a very nice solution! I will try to sum everything up so that we have a clear plan on what we want and how to get it.

[ki82]

Any progress on this?

I would like to use boringssl instead of openssl.
Installing boost using vcpkg forces me to install openssl.
Is there any workaround to be able to install a package but not to include dependent package(openssl)?