Shared library version and SONAME/install name

[dalcinl]

Shared library version and name

The current status of the ABI specification defines the following major and minor ABI versions to encode backward-compatibility contracts:

MPI_ABI_VERSION (major)	1
MPI_ABI_SUBVERSION (minor)	0

We should use these numbers do define the shared library version. Shared library versions typically start at version 0, and that's the number we currently generate. However, AFAICT, that's just conventional. Should we instead start at version 1 to match MPI_ABI_VERSION, thus producing an SONAME of libmpi_abi.so.1 on Linux and a install name of @rpath/libmpi_abi.1.dylib?
Alternatively, MPI could start with MPI_ABI_VERSION=0, although that's admittedly awkward. Likewise, having libmpi_abi.so.X with X not matching MPI_ABI_VERSION is also awkward.

Use of libtool

On a related note, the MPI ABI major and minor numbers would most likely be used to define the current/release/age triplet of libtool's -version-info c:r:a. Would the following mapping be appropriate?
I assume here we chose the major library version starting at 1, and not 0 as it is conventional, as per the previous discussion.

libtool	MPI ABI
current	MPI_ABI_VERSION + MPI_ABI_SUBVERSION
revision	0
age	MPI_ABI_SUBVERSION

Which would map to the following shared library file names

Platform	Shared library name & symlink	Shared library full file name
Linux	libmpi_abi.so.${MPI_ABI_VERSION}	libmpi_abi.so.${MPI_ABI_VERSION}.${MPI_ABI_SUBVERSION}
Darwin	libmpi_abi.${MPI_ABI_VERSION}.dylib	libmpi_abi.${MPI_ABI_VERSION}.${MPI_ABI_SUBVERSION}.dylib

EDIT: As per the current ABI proposal, libtool's revision number will always stay zero (is this corrent?). The full shared library filename may have an extra .0 appended on Linux.

[hzhou]

What is the use case for these two macros?

[dalcinl]

The primary use of these macros should be Conditional compilation. For example, I'm about to merge this commit into mpi4py mpi4py/mpi4py@170da9f. I do a few things differently under the ABI. For example, I assume that all MPI symbols are present for a given API version, and I disable or do differently for implementation-specific workarounds.

Irrespective of the macros, Jeff is proposing two version numbers (major and minor) for versioning the ABI. I'm trying to discuss here how these two numbers will map to the shared library filenames and SONAME.

[hzhou]

@dalcinl So you only need #define MPI_ABI? I am concerned that we are going to use header macros to differentiate ABI versions. Sounds like trouble.

[dalcinl]

For the time being yes, my only need is to know whether I'm compiling under the ABI. But that does not mean that in the future having access to the ABI version in the preprocessor will not be useful. What's the problem with differentiating ABI versions? What's exactly your concern?

What I'm trying to discuss in this issue is somewhat orthogonal to the addition of these constants to mpi.h. The ABI will have two version numbers, major and minor, either explicitly, or implicitly. By explicitly, I mean that the standard states these two ABI major/minor numbers for every API major/minor version, and the ABI versions may or may not endup being added to mpi.h. If the Forum decides NOT to define ABI version (which would be IMHO a disservice to the community), then the ABI version will be implicitly defined trough the API version form the set of backward compatible and non-backward compatible additions on each release of the MPI specification.
Therefore, these ABI versions enconde backward compatibility contracs, quite close if not almost identical in concept to libtool interfaces, then we have to use the ABI version to define the libtool triplet cur:rel:age.

PS: Everytime I raise an issue or concern about a technicality and you reply back with backlash I'm left in an awkward position, in the middle of Jeff and you. For quite a bit of time, Jeff had these two constants plus three new ABI-related query routines as part of his proposal. I guess these additions will be discussed in the next Chicago reading, see Jeff's PR about that. My last merged PR added MPI_ABI_(SUB)VERSION to the stubs mpi.h symply because that header is supposed to be more or less in-sync with the proposal, although I have not yet added the three new query routines because I raised concerns about them.
I do have the feeling that two new macros and three new routines for ABI-related queries seems a bit two much. But that discussion should be held elsewhere. A video meeting would be the best starting point.

[hzhou]

PS: Everytime I raise an issue or concern about a technicality and you reply back with backlash I'm left in an awkward position, in the middle of Jeff and you.

Why though? You could either agree with me and argue against Jeff with me. Or you could disagree with me and argue with me with Jeff. Or you could leave the argument for Jeff and me.
Unless you want to stand in the middle and prevent the argument.

FWIW, I don't have a sufficient distinction between an argument and a discussion.

[dalcinl]

Why though?

Because I think most of the times the most productive use of time is is to argument/discuss in group and settle things once and for all. Otherwise we agree in pairs, and then the third-party at a later time makes the whole thing restart all over again.

In the particular subject of whether to expose ABI version info as macros, I think the general answer is yes, why not? But it would be really nice if a single macro would provide all what we need. For example, If a MPI implementation and the included mpi.h supports the ABI , we could have #define MPI_ABI_VERSION {100*major+minor}where major and minor and the major and minor ABI version. Otherwise if the ABI is not supported then#define MPI_ABI_VERSION 0` or some other sentinel value? or perhaps better the macro not defined if not supported? If not defined in C, what do do in Fortran? Those are all relatively trivial things, but we all tend to have strong opinions. What I think the three of us agree is that there is enough mess in the MPI standard to add more messy/sloppy stuff on top of it.

But again, now I'm derailing myself! I opened this issue to discuss how to name and version libraries. Maybe my opening in the description was a bit confusing. I'll try to reword that.

FWIW, I don't have a sufficient distinction between an argument and a discussion.

Don't worry, I understand you use them interchangeably. I do the same all the time.

[hzhou]

Sounds good. So far I only see points to support a mere #define MPI_ABI. This means the API does not have any implementation-dependent extensions. Anything more to bring the ABI into API seems wrong. ABI versioning is controlled by the libtool triplet and the dynamic library versioning. Add 2nd mechanism IMO is unnecessary and harmful.

[dalcinl]

ABI versioning is controlled by the libtool triplet and the dynamic library versioning

An who ultimately decides the value of the triplet to pass to libtool and the dynamic library versioning?

[hzhou]

ABI versioning is controlled by the libtool triplet and the dynamic library versioning

An who ultimately decides the value of the triplet to pass to libtool and the dynamic library versioning?

The libtool triplet is a link-time option for implementers. It doesn't need to be set in header files as macros and especially doesn't need to be consumed by MPI users. Even if you want to put it in the header, it can be directly set to the triplet in a comment rather than set them in misleading macros and imply a formula to derive the triplet.

[dalcinl]

The libtool triplet is a link-time option for implementers

There you have... The whole point of the ABI is interoperability across implementations. We cannot leave library versioning 100% up to the implementers, because they feel kings in their little reigns and don't give a damn about each other. Look what happened to library versions in MPICH and Open MPI. As per my recollection, there are 10 different MPI APIs (one for each MPI standard version), but in only one time (MPI-3.0) there was a backward incompatible change that removed stuff. Yet, MPICH managed to have today libmpi.so.12 (12!) and Open MPI is at libmpi.so.40 (40!). If MPI has had an ABI from the very beginning, today we all should be at libmpi.so.1.

This is the stubs repository. Some day we will properly rename it and maintain it. Third parties will be able to use it to build binary assets without a concrete implementation. I'm already doing that in testing!
To that end, we have to produce here a libmpi_abi.so.X with a SONAME of libmpi_abi.so.X. Where the X number comes from? Who clearly mandates what that number should be? Where is the clear specification of the rules for that number to change if the ABI changes?.
Irrespective of whether the ABI version numbers belong to mpi.h or not, I think the standard still have to formally mandate on ABI version numbers. From these standard numbers, implementors MUST derive the the current and age values for libtool (with release=0 always?). Otherwise, how do we ensure interoperability across implementations?

[jeffhammond]

I want these macros so I know what version of the header I compiled my application or library against, so that I can check it against the SO I'm using at runtime. If this is the wrong way to do this, that's fine - I really don't know what I'm doing w.r.t. ABI-versioned shared library designs.

[jeffhammond]

As far as arguing versus discussing, it is important to get all the objections on the table so we don't make mistakes. At some point, somebody has to decide what to do, and probably somebody isn't perfectly happy with everything, but hopefully the objections are not of the kind "this is garbage that will never work".

[hzhou]

The whole point ...

@dalcinl Calm down :) I was not arguing against your whole point [to fix the so version] at all. I was saying you do not do that via header macros that are meant for MPI users. You just need to write down the exact libtool triplet in the document or as a comment in the header file and require implementers to use it.

I want these macros so I know what version of the header I compiled my application or library against, so that I can check it against the SO I'm using at runtime. If this is the wrong way to do this, that's fine - I really don't know what I'm doing w.r.t. ABI-versioned shared library designs.

A comment line will do. And a text in the document will enforce that.

[hzhou]

Libtool has explicit rules on how to set the version triplet. It does not often align with people's perception of how an API version works. I don't think it is a good idea to invent a separate rule from libtool. Rather, set the version triplet according to the libtool's rule, write that triplet down, and ask implementers to use it.

PS: https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html

[jeffhammond]

ok i'm fine to use libtool's method, but i need to know how to write it into the MPI spec...

[hzhou]

ok i'm fine to use libtool's method, but i need to know how to write it into the MPI spec...

Just write the libtool triplet literally into the spec. Every time we publish a new version of the spec, we update the triplet according to libtool's rule.

[dalcinl]

I want these macros so I know what version of the header I compiled my application or library against, so that I can check it against the SO I'm using at runtime.

A comment line will do. And a text in the document will enforce that.

@jeffhammond I believe I told you in private communication that I was not sure the MPI_ABI_(SUB)?VERSION macros and the query routine MPI_Abi_get_version() where useful. But I'm very aware that I overwhelmed you with so many things going on, so you probably forgot or simply ignored me. @hzhou's comment are more or less in line with these previous comments of mine. The ultimate reason why the ABI version is not that useful is because the MPI API version (from MPI_(SUB)?VERSION and MPI_Get_version()) define the set of additions and removals in each release of the standard, and these additions and removal implicitly define the ABI compatibility.

Guys, we need to reach an agreement on whether we need or not MPI_ABI_(SUB)?VERSION and MPI_Abi_get_version(). Not having them will simplify Jeff's proposal, removing two macro constants and one query routine. Nonetheless, the proposal may still need some more explicit text discussing versioning for the shared library, defining the rule for incrementing the major version (and the initial value of 0 used so far here in the stubs and in MPICH?).

PS: @hzhou I'm sorry, I get easily frustrated. I opened this issue with the intention of settling the versioning of the shared library, and things went sideways discussing the contents of mpi.h. Unfortunately, mpi.h as it is now do have MPI_ABI_(SUB)?VERSION, these macros have been in Jeff's proposal for some time, I just added them to mpi.h to keep the header in sync with the proposal. As a consequence of the recent change, the addition of MPI_ABI_(SUB)?VERSION is also in MPICH after the merge of my recent ABI update PR. I commit myself to implement any fix here and in MPICH once we reach an agreement on what to do exactly.

[hzhou]

@dalcinl No worries :)

@jeffhammond Just a reminder that it is always easy to add to the standard later on but nearly impossible to remove from it. I find it worrisome that you are trying hard to solve all issues beyond the initial scope. It is due diligence to look at and discuss all things, but I would try hard to define the current scope. For example, I think including anything Fortran in the first ABI spec is not a good idea. Those _f2c and MPI_F_xxx can be optional inclusions. I think most consumers of MPI ABI won't be affected by those APIs. We need to address the Fortran binary issue, but I think it is better to tackle that as a separate effort.

Sorry for the side track.

[jeffhammond]

if you read the text, i am defining only what C needs from Fortran, plus stating things that are required for a Fortran implementation like VAPAA to use the C ABI.

[dalcinl]

@jeffhammond I believe your last comment is misplaced. Did you meant to put it in #27?

[jeffhammond]

No, I was responding to Hui's comment immediately prior.